EFW Inc.

 

Decision

Matter of:  EFW Inc.

File:  B-418177; B-418177.2; B-418177.3; B-418177.4; B-418177.5

Date:  January 21, 2020

DIGEST

1.  Protest that agency misevaluated both the protester’s and the awardee’s proposals under the technical and past performance factors is denied where the contemporaneous record shows that the agency’s evaluation was reasonable and consistent with the solicitation.

2.  Protest that agency made an unreasonable cost realism assessment of the awardee’s proposal is denied where the record shows that the agency based its realism analysis on each offeror’s unique approach and made a reasonable cost realism assessment. 

EFW, Inc., of Fort Worth, Texas, protests the award of a contract to Rockwell Collins, Inc., doing business as Collins Aerospace, of Cedar Rapids, Iowa, under request for proposals (RFP) No. N00421-18-R-0091, issued by the Department of the Navy, Naval Air Warfare Center, for binocular helmet-mounted display systems for use by helicopter pilots (often referred to as the enhanced visual acuity system, or EVA).  EFW argues that the Navy misevaluated the proposals and made an unreasonable source selection decision. 

We deny the protest.

BACKGROUND

The RFP, issued on December 7, 2018, sought proposals for a next generation helmet-mounted night vision device and heads-up display in support of the enhanced visual acuity program.  The RFP anticipated the contractor’s efforts would begin with an engineering and manufacturing development (EMD) phase, followed by options for engineering development models, and low-rate initial production.  Agency Report (AR) Tab 1, RFP, at 86.  Award of a single hybrid contract[1] would be made to the offeror whose proposal provided the best value to the agency considering three factors, listed in descending order of importance:  technical approach, past performance, and cost.  Id. at 89. 

Under the technical approach factor, the agency would assess a technical rating (green/acceptable, yellow/marginal, or red/unacceptable) and a separate technical risk rating (low, moderate, high, or unacceptable), and the evaluation would include a technology readiness assessment.  Id. at 92.  The assessment of technical risk would consider the risk associated with the offeror’s technical approach and, in particular, the “potential for disruption of schedule, increase in costs, degradation of performance, the need for increased Government oversight, or the likelihood of unsuccessful contract performance.”  Id.  In that connection, the RFP also encouraged offerors to identify as a separate “risk reducer” each individual element of their proposals that would reduce the technical risk and provide benefit to the agency.  Id. at 63, 65, 93. 

The past performance evaluation would result in a performance confidence assessment rating based on an integrated assessment of all performance areas, with a neutral rating of unknown confidence for an offeror without available performance information.  Id. at 91.  The cost/price evaluation would assess fixed-price contract lines for reasonableness and a “clear understanding of the solicita[t]ion requirements,” while the cost-reimbursement contract lines would be assessed for “realism, completeness, and consistency with respect to the offeror’s technical approach.”  Id.

The Navy received two proposals by the February 19, 2019, closing date:  one from Collins and one from EFW.  On March 22, after evaluating the initial proposals, the Navy formed a competitive range of both offerors’ proposals and initiated discussions.  On August 12, both firms submitted final proposal revisions (FPR).  After evaluating the FPRs, the Navy determined that further discussions were required and reopened discussions.  On September 24, both firms submitted a second round of FPRs.   

After evaluating the second round FPRs, the agency’s final adjectival ratings and evaluated costs for the offerors were as follows:

AR Tab 2, Final Source Selection Evaluation Board (SSEB) Report, at 5, 22, 50, 64, 76, and 83. 

Although the source selection authority (SSA) determined that EFW’s proposal was slightly superior under the technical approach factor, she deemed that advantage insufficient to justify paying its higher evaluated cost.  AR Tab 3, Source Selection Decision Document (SSDD), at 2-4.  On September 30, the contracting officer informed EFW that the agency had selected Collins’s proposal for award.  After receiving an extended debriefing, EFW filed this protest. 

DISCUSSION

EFW raises numerous challenges to the evaluation of both firms’ proposals under the technical and past performance factors, the cost realism assessment, and the resulting source selection decision.  Broadly, EFW argues that the Navy should have rated Collins’s proposal technically unacceptable and its own proposal more favorably, should have rated Collins’s past performance lower and EFW’s higher, and should have found Collins’s costs unrealistic.  We have reviewed all of EFW’s challenges to the evaluation and source selection and conclude that none provides a basis to sustain the protest.  We address the principal arguments in detail below.[2] 

Challenges to Technical Evaluations

Misapplication of RFP Technical Risk Ratings

EFW first argues that the Navy misevaluated Collins’s technical approach by failing to rate its technical risk as unacceptable.  EFW maintains that the RFP’s definitions of moderate and high risk refer to proposals with “a significant weakness or combination of weaknesses,” which must refer to a proposal with no more than a single significant weakness.  As a result, under EFW’s reading, a proposal with more than one significant weakness (such as Collins’s) could not be assessed either of these ratings.  Instead, it must be assessed an unacceptable risk rating, defined as a proposal receiving “a combination of significant weaknesses,” and therefore ineligible for award.  Consolidated Protest at 13-14 (quoting RFP at 90, 92).  

The Navy and Collins respond that EFW mischaracterizes the terms of the RFP, arguing that the RFP language did not require the assessment of risk based simply on the number of weaknesses identified in the technical evaluation.  Instead, they argue, the RFP provided for the technical risk rating to be based on the evaluators’ judgment about the extent to which the weaknesses--in the context of the technical approach--would cause disruption to schedule, increased cost, or degraded performance, and judgment about the likelihood that contractor and government efforts would overcome the difficulties.[3]  AR Contracting Officer’s Statement and Memorandum of Law (COS/MOL) at 14‑16 (quoting RFP at 92); Intervenor’s Comments at 5.  Both argue that EFW’s challenges should be denied because the record shows that the evaluators reasonably considered the risks in Collins’s proposal in detail, including the specific weaknesses identified, and then exercised reasonable judgment, consistent with the RFP’s guidance, in determining that the proposal posed moderate risk.  COS/MOL at 18-19; Intervenor’s Comments at 7-8.   

In reviewing protests against allegedly improper evaluations and source selection decisions, it is not our role to reevaluate proposals; rather, our Office examines the record to determine whether the agency’s judgment was reasonable and in accord with the RFP criteria.  MD Helicopters, Inc.; AgustaWestland, Inc., B-298502 et al., Oct. 23, 2006, 2006 CPD ¶ 164 at 13.  Even where some elements of the proposal do not meet the agency’s standards, the agency is not required to evaluate a proposal as posing unacceptable risk unless the solicitation specifically provides otherwise; the agency may exercise judgment in determining the level of risk posed.  Id. at 25. 

Our review of the record provides a reasonable basis for the Navy’s evaluation of Collins’s proposal as posing a moderate technical risk.  The SSEB explained the nature of the significant weaknesses in Collins’s proposed approach, which had their basis in the lack of the desired level of technological maturity for two components of its proposed system:  [DELETED].  AR Tab 2, Final SSEB Report, at 17-20.  For each element, the SSEB explained the information justifying its conclusion that the component was not at the desired level of technological maturity.  It also explained the remaining technical hurdles, Collins’s techniques to accomplish them, and the risk entailed by seeking to mature the technology during that stage of the contract (the EMD phase).  Id. at 19‑21.  The SSEB also identified two aspects of Collins’s approach that were considered risk reducers:  the firm’s experience in [DELETED], and its experience with [DELETED].[4]  Id. at 16-17.  Considering the relationship of those risk reducers to the risks identified in the significant weaknesses, the SSEB reasoned that the two components could be brought to the agency’s desired level of technological maturity through considerable contractor emphasis and agency oversight.  Id. at 21.  The SSA discussed and concurred with the SSEB’s judgment.  AR Tab 3, SSDD, at 2-3. 

Based on our review, the record provides a detailed explanation for the SSEB’s conclusions about the extent of the technical challenges faced by Collins’s approach and the reasons that the SSEB concluded the firm could overcome them.  In addition, the SSEB explained its reasoning for concluding that the resulting level of risk for Collins’s approach would be acceptable, and thus merited a technical risk rating of moderate.  AR, Tab 2, Final SSEB Report, at 20.  We do not agree with EFW’s selective--and thus, misleading--reading of the RFP to require the SSEB to assess unacceptable risk (or even high risk) once multiple significant weaknesses were identified.  Rather, the SSEB properly exercised its evaluation judgment in rating Collins’s technical risk as moderate, and provided detailed justification for that judgment in the contemporaneous record. 

SSA Authority to Depart from SSEB Evaluation

EFW challenges the SSA’s decision to discount one risk reducer the SSEB found in the protester’s technical approach for the EVA night vision sensor, which the protester argues was done to justify deeming its proposal to have only a slight advantage over Collins’s proposal under the technical factor.  EFW argues that the SSA lacked a valid justification for disregarding the SSEB’s technical assessment.  Consolidated Protest at 16-17, 22. 

In its proposal, EFW identified as one of the technical risk mitigation elements a design that would allow potential substitution of a different [DELETED].  The proposal explained that the technical approach was to implement [DELETED] as the primary [DELETED], but that its design allowed [DELETED] that, while less capable, was still viable.  That approach would mitigate the risk that EFW’s efforts to [DELETED] did not meet the contract objectives.  Document Request No. 1/Folder 2, EFW Technical Proposal (vol. II), at 102.  The SSEB discussed the proposed risk mitigation and credited EFW’s proposal with a risk reducer for this [DELETED].[5]  AR Tab 2, Final SSEB Report, at 26-27, 33‑34.

The SSA acknowledged the SSEB’s judgment in assigning a risk reducer for EFW’s [DELETED], but disagreed with the value of the approach, principally because the [DELETED] was also not [DELETED].  AR Tab 3, SSDD, at 2‑3.  The SSA explained that the potential reduction of risk by [DELETED] would only be obtained if the main risk (the failure of EFW’s efforts to achieve [DELETED]) had manifested.  Id.  The SSA pointed out that the SSEB had assessed the effect of making that change would have minimal schedule impact, but a cost impact “as high as $[DELETED].”  Id.  In contrast to the SSEB, the SSA’s own judgment was that EFW’s [DELETED] would not actually provide an advantage to the agency during contract performance, and so it could not be considered a risk reducer.  Id.  The Navy argues that the SSA’s judgment in disagreeing with the SSEB was reasonable.  COS/MOL at 43-45.

The record supports the reasonableness of the SSA’s judgment that EFW’s [DELETED] would not reduce the risk of the proposed technical approach and thus should not be evaluated as a risk reducer.  We have noted that in a negotiated procurement, the SSA is entitled to use reports and analyses prepared by others, but is still required to exercise independent judgment in making judgments in the source selection.  Consistent with that principle, source selection officials and higher-level agency evaluators may reasonably disagree with the evaluation ratings and results of lower-level evaluators.  Wyle Labs., Inc., B-311123, Apr. 29, 2008, 2009 CPD ¶ 96 at 6.  Here, the SSA reviewed the SSEB’s conclusion that EFW’s [DELETED] reduced the risk of its technical approach but exercised independent judgment in determining that there would be no significant benefit to the Navy in performance for two reasons:  [DELETED], and the benefit would accrue only after EFW had been [DELETED].  While we recognize EFW’s disagreement with that judgment, in our view, the SSA explained a reasoned basis for discounting the risk reducer that the SSEB had assessed. 

Misevaluation of EFW’s Design Tolerance

Next, EFW argues that the SSEB unreasonably failed to recognize that its approach using the [DELETED] would provide an additional risk reducer through its expected performance in [DELETED], which was one of the Navy’s important objectives for the EVA.  A reasonable evaluation, EFW argues, would have recognized that [DELETED], would exceed the agency’s minimum performance threshold by [DELETED] percent (or possibly more)[6] for operation [DELETED].  The approach would thereby have a development margin (or design tolerance) in EFW’s ability to meet the agency’s minimum threshold.  That design tolerance should have been recognized as reducing the risk for EFW’s technical approach and so, deemed to be an additional risk reducer, EFW argues.  Consolidated Protest at 17-20; Protester’s Comments at 21-26. 

The Navy argues that risk reducers were used to assess methods of reducing risk to the EVA program as a whole, and that the agency reasonably concluded that EFW had not substantiated its claims about the performance of the [DELETED].  In the agency’s view, the [DELETED] increased the risk to the program as a whole.  Also, the agency noted that EFW’s proposal only focused on [DELETED] reducing the risk of meeting requirements for [DELETED] did not reduce the risk to the EVA program overall, the agency concluded.  COS/MOL at 21-22. 

Our review of the contemporaneous record provides no basis to question the Navy’s judgment in not assessing an additional risk reducer for the design tolerance of EFW’s proposed [DELETED].  Instead, EFW itself identified the benefit of the expected performance of its [DELETED] as relating to whether it would meet the threshold [DELETED].  Given that the focus of EFW’s claim was on one threshold, not the program, we find reasonable the agency’s assessment that the performance of EFW’s [DELETED], even if successful, would not reduce the technical risk for its performance of the EVA program overall,[7] and so did not merit assessment of an additional risk reducer.   

Misevaluation of EFW’s Sensor Technology

EFW also challenges the assessment of a significant weakness to its proposal for the lack of technical maturity of the [DELETED] sensor technology.  Consolidated Protest at 21‑22.  Rather than a significant weakness, EFW argues that its proposal, as amplified through discussions, demonstrated the feasibility of its [DELETED] and the advantages of its design.  Id. at 18-20.  Additionally, EFW notes that the SSEB assessed two risk reducers for its technical approach:  the [DELETED] of its design, and its proposed approach of having a [DELETED].  Id. at 15-16; see AR Tab 2, Final SSEB Report, at 33-34.  

The RFP provided that the technical evaluation would assess “understanding of, approach to, and ability to meet the solicitation requirements.”  RFP at 89.  The technical evaluation would also assess a list of specific elements, among which were the display field of view and night vision camera field of view, scene display artifacts, night vision, and technical maturity.  As noted above, the technical rating could be acceptable, marginal, or unacceptable.  The accompanying technical risk rating would consider the potential for the offeror’s approach to result in disruption of schedule, increase in costs, degradation of performance, the need for increased government oversight, and the likelihood of unsuccessful contract performance.  Id. at 90. 

The Navy responds that EFW’s argument is based only on its own belief in the superiority of its technical approach, and amounts to disagreement with the agency’s reasoned evaluation judgments.  The agency argues that the agency properly assessed a significant weakness due to the technological immaturity of EFW’s proposed [DELETED] because that approach appreciably increased the risk of cost and schedule overruns.  COS/MOL at 24.  The agency notes that the evaluators recognized that EFW proposed an alternative [DELETED], but that the [DELETED] also lacked technological maturity, which made the assessment of a significant weakness for EFW’s approach was proper.  Id.  Altogether, the Navy and Collins argue that this aspect of the protest should also be denied because the SSEB’s evaluation was reasonable and the SSA exercised appropriate discretion in reaching a differing conclusion, in one respect, than the SSEB.  Id. at 20‑28, 44-45; Intervenor’s Comments at 9-10. 

We agree.  The Navy’s recognition of the technological immaturity and the risk presented by EFW’s approach reflects a reasonable evaluation judgment, which is supported by the record.

Challenges to Past Performance Evaluations

EFW next argues that the Navy misevaluated both firms’ past performance.  First, EFW contends that the Navy ignored Collins’s lack of past performance relevant to providing digital night vision devices for aircrew helmets.  Consolidated Protest at 29-30.  EFW also argues that the Navy concealed from the SSA adverse past performance information about Collins, and that the evaluators favorably considered past performance information about Collins that should have been excluded because it was submitted by an SSEB member.  Consolidated Protest at 29-30; Second Supp. Protest (Nov. 7, 2019) at 4-8. 

The Navy responds that it reasonably considered Collins’s past performance on an integrated basis across all performance areas.  Even accepting EFW’s contention that Collins has not performed a contract for precisely the type of integrated product required here, the Navy disputes EFW’s assertion that the satisfactory confidence rating was therefore improperly applied to Collins’s performance record.  Id. at 31.  Additionally, the Navy replies that the SSA was properly and accurately informed about the negative past performance report for Collins that EFW contends had been concealed and the positive assessment by a member of the SSEB.  Supp. AR at 5 (quoting Supp. AR Tab 5, Enhanced Visual Acuity Final SSA Briefing Slides, at 13).  The agency contends that the SSEB and the SSA reasonably considered Collins’s performance record to merit satisfactory confidence. 

The evaluation of past performance, including an agency’s determination of the relevance and scope of a firm’s performance history to be considered, is a matter of agency discretion, which GAO will not find improper unless unreasonable, inconsistent with the solicitation criteria, or undocumented.  A protester’s disagreement with the agency’s judgment, without more, is insufficient to establish that an evaluation was improper.  Harris IT Servs. Corp., B-406067, Jan. 27, 2012, 2012 CPD ¶ 57 at 10. 

The contemporaneous record here provides no basis to sustain EFW’s challenges to the past performance evaluation of Collins; to the contrary, the record shows that the Navy performed a detailed review of the firm’s past performance record.  The record also supports the Navy’s determination that Collins had relevant past performance that was positive overall, and that an instance of adverse past performance was offset by demonstrated and ongoing systemic improvements.  AR Tab 2, Final SSEB Report, at 50.  The evaluators also accurately relayed the substance of Collins’s past performance to the SSA, including the view of one reference that, if given the option, “knowing what you know today, would you award this contract to this contractor,” that reference responded “probably not.”  Id. at 45-48; Supp. AR Tab 5, Enhanced Visual Acuity Final SSA Briefing Slides, at 13. 

EFW’s contention that the information should have resulted in a lower confidence rating for Collins under the past performance factor reflects disagreement, but does not show that the agency’s evaluation judgment was unreasonable in assessing the satisfactory confidence rating.[8] 

In our view, the record demonstrates that the Navy’s assessment of Collins’s past performance was reasonable.  The record reflects consideration of both the positive and adverse aspects of Collins’s past performance, including the extent to which the firm had implemented systemic improvements.  While recognizing significant positive past performance information, the SSEB also identified adverse performance on a relevant contract affecting both technical and schedule performance.  The SSEB considered the nature of the past performance problems; the firm’s systemic improvement efforts; the differences between Collins’s proposed technical approach for the EVA contract and the techniques used when the adverse performance occurred; and the measures that Collins used to compensate and recover its production schedule.  AR Tab 2, Final SSEB Report, at 45-48.  The record thus supports the Navy’s assessment of a satisfactory confidence rating overall.  Id. at 50-51.  Furthermore, the record confirms that the SSA was properly briefed on the firm’s performance record and reasonably concurred with the SSEB’s assessments.  

In challenging the evaluation of its own past performance, EFW argues that the Navy improperly minimized EFW’s position as the world leader in supplying night vision systems and helmet mounted displays, and EFW’s record of highly relevant contracts with successful performance.  Consolidated Protest at 30‑34.  EFW contends that satisfactory ratings of its prior performance were unfairly treated as adverse, and that its performance record should have resulted in a substantial confidence rating.  Id. at 33.  

The Navy responds that the record of the protester’s relevant past performance showed multiple instances where EFW and its principal entities were rated satisfactory in the area of technical quality, and marginal and satisfactory in the area of schedule.  EFW also received a single satisfactory rating in the area of management performance.  COS/MOL at 32‑33; see AR Tab 2, Final SSEB Report, at 56, 59, 63.  The Navy also acknowledges that the record shows that EFW made systematic improvements in most areas, but in one area, the efforts at improvement were ongoing.  The Navy also noted that the firm had not provided supporting evidence for the success of those systemic improvements.  COS/MOL at 33.  The Navy argues that the record shows that the evaluators carefully considered EFW’s performance record, and made a reasonable judgment that EFW’s performance record overall merited a satisfactory confidence rating.  Id. at 34.

The record shows that the evaluators carefully reviewed the firm’s past performance, along with EFW’s efforts at systemic improvement, and concluded that while the record justified discounting much of the adverse information about EFW, one area of adverse schedule performance remained as a continuing risk that could impair the firm’s performance of the contract.  AR, Tab 2, Final SSEB Report, at 56-61.  Based on the record, we conclude that the assessment of a satisfactory confidence rating for EFW was reasonable.  

Challenges to Cost Realism Evaluation

EFW next argues that the Navy should have recognized that Collins’s lower proposed level of effort resulted in an unrealistically low cost estimate.  The protester also argues that the agency failed to properly consider that Collins had proposed to allocate development costs necessary to performing this contract as independent research and development (IR&D), which will increase the firm’s indirect costs.  Second Supp. Protest (Nov. 7, 2019) at 8-13.  The protester argues that a reasonable cost realism analysis of Collins’s proposal would have identified this issue.

The Navy responds that the agency reasonably analyzed the realism of both offerors’ proposed costs, including ensuring that all elements of the work were adequately addressed and that the proposed costs were consistent with the offeror’s design approach.  Supp. COS/MOL at 11-13.  The evaluators then made probable cost adjustments based on the agency’s analysis, and concluded that the most probable cost of Collins’s proposed approach was $46.5 million.  Id. at 12 n.4.  With respect to the effect of Collins’s proposed reliance on significant IR&D, the Navy argues that it recognized the issue, reviewed the realism of that aspect of the proposal, and concluded that the agreement included IR&D projections for efforts on this contract.  Id. at 15.  The Navy argues that it did not simply accept Collins’s proposed costs, as EFW suggests, but made appropriate realism adjustments to determine Collins’s most probable cost.  Id. at 16. 

The purpose of a cost realism analysis is to determine the extent to which the offeror’s proposed costs are realistic and reasonable.  Such a determination must itself be reasonable.  The cost realism analysis aims to determine what, in the government’s view, it would realistically cost the offeror to perform given the offeror’s own technical approach.  SRS Techs., B-238403, May 17, 1990, 90-1 CPD ¶ 484 at 4 n.1.  The agency is not required to conduct an extensive analysis or to verify all elements supporting the cost realism assessment, and our Office will not disturb an agency’s cost realism analysis unless it is shown to lack a reasonable basis.  Orbital Sci. Corp., B‑414603, B‑414603.2, July 26, 2017, 2017 CPD ¶ 249 at 7.

Here, the record provides a reasonable basis for the Navy’s cost realism review.  EFW’s argument that its own cost estimates--which were, by definition, based on its own unique approach--were higher than Collins’s, does not provide a factual basis to sustain EFW’s challenges to the agency’s cost realism judgments.  See BCF Sols., Inc., B‑409570, June 13, 2014, 2014 CPD ¶ 177 at 13 (denying protest based on mistaken premise that cost realism analysis required agency to compare protester’s costs to awardee’s).  The realism of an offeror’s costs is not assessed by comparing one offeror’s proposed costs to another offeror’s proposed costs, but rather, assessing whether the offeror’s proposed costs are realistic for its unique technical approach.  Onyx-Technica, JV, B-412474, B-412474.2, Feb. 26, 2016, 2016 CPD ¶ 65 at 5.  We also find that the Navy reached reasonable conclusions about Collins’s proposed approach of treating some of its development costs as IR&D.  The assessment of those costs as realistic took into account Collins’s most recent update of its forward pricing rate agreement, which included those costs as IR&D.  The record provides no basis to sustain EFW’s challenges to the agency’s evaluation of Collins’s costs.

Source Selection Rationale

Finally, EFW argues that the SSA failed to “look behind” Collins’s rating of acceptable under the technical approach factor to conclude that EFW’s proposal was superior.  Consolidated Protest at 24-26.  As a result, EFW argues that the Navy effectively abandoned the best-value tradeoff criteria in favor of selecting Collins’s proposal based on price.  Id.; Protester’s Comments at 35-36. 

Our review of the contemporaneous record refutes this allegation.  The SSA discussed significant elements of the technical evaluation, and directly compared the two offerors to support the SSA’s documented tradeoff judgments about whether EFW or Collins had an advantage for each element.  AR Tab 3, SSDD, at 3‑4.  After that element-by-element review, the SSA then reached an overall judgment, which expressed the view that, overall, EFW’s technical approach offered a slight advantage over Collins’s.  Id. at 4. 

The SSA next considered and expressed agreement with the past performance evaluation, and determined that neither offeror had an advantage under that factor.  The SSA then reviewed the cost evaluation and noted the cost realism adjustments that had been made to both offeror’s proposed costs.  Id. at 4.  The SSA then explained that the source selection judgment depended on whether EFW’s slight advantage under the technical factor would justify paying the estimated cost premium of almost $7 million.  The SSA concluded that the tradeoff favored Collins’s lower-cost proposal.  Id. at 5. 

As a whole, the contemporaneous record shows the SSA engaged in a detailed consideration of the evaluation and made a reasoned judgment to select Collins’s lower-cost proposal over EFW’s.  The source selection decision was reasonable and consistent with the RFP evaluation criteria.  The record provides no basis to sustain EFW’s challenge to the source selection rationale. 

The protest is denied. 

Thomas H. Armstrong

General Counsel

 

---