Avionic Instruments LLC

Decision

Matter of:  Avionic Instruments LLC

File:  B-418604; B-418604.2

Date:  June 30, 2020

DIGEST

Protest challenging the agency’s evaluation of technical proposals is sustained because the contemporaneous record does not show that the evaluation was reasonable and in accordance with procurement law and the terms of the solicitation.

Avionic Instruments LLC, of Avenel, New Jersey, protests the award of a contract to Physical Optics Corporation (POC), of Torrance, California, under request for proposals (RFP) No. N00019-19-R-0030, issued by the Department of the Navy, Naval Air Systems Command, to replace analog inverters on the Navy’s UH-1 Y helicopter fleet with digital signal processor (DSP) inverters.  The protester challenges as unreasonable the agency’s evaluation of technical proposals.

We sustain the protest.

BACKGROUND

This procurement was for development and production of a replacement inverter for the UH-1Y helicopter.  Agency Report (AR), Tab 1, Source Selection Plan at 7.  Inverters convert a helicopter’s direct current (DC) into an alternating current (AC) usable for other electronic needs.  The agency intends to replace all existing UH-1Y analog power inverters, which have a history of low reliability rates, with a digital design that offers the potential for improved reliability.  AR, Tab 7, Source Selection Evaluation Board (SSEB) Report at 3.  After conducting market research, the agency decided to procure a new DSP inverter on a full and open competitive basis under Federal Acquisition Regulation part 15.  Id.

The RFP provided for the award of a fixed-price contract with a base award and options.  AR, Tab 2, RFP at 3-6.  The base award included four contract line item numbers for:  non-recurring engineering (NRE) required to satisfy specification requirements; delivery of six flight test assets; delivery of four assets for validation/verification, and testing; and technical and administrative data.  Id.  The options were to manufacture and deliver additional quantities of DSP inverters that meet the configuration approved during the base contract.  Id.

Award would be made to the offeror whose proposal represented the best value to the government, considering two factors--technical and price--with technical being more important.  Id. at 65.  Under the technical factor, the government would evaluate the offeror’s “understanding of, approach to and ability to meet the solicitation requirements.”  Id.  The technical evaluation would include the following elements:  overall design approach; approach to qualification testing; logistics planning; an experience risk assessment of qualification testing and achieved reliability for similar aircraft applications; and small business strategy.  Id. 

The agency would assign each proposal a technical rating and a technical risk rating.[1]  Id. at 65.  The technical rating would be an assessment of compliance with the solicitation requirements, considering “the benefits and detriments related to program performance and operations.”  Id.  The technical risk rating would evaluate “the risk associated with the technical approach in meeting the requirement.”  Id. at 65-66.  The technical risk rating would consider the potential for disruption of schedule, increase in costs, degradation of performance, need for increased government oversight, and likelihood of unsuccessful contract performance.  The experience element would only be evaluated for risk.  Id. at 65. 

With regard to experience, the RFP required offerors to “provide information that describes the Offeror team’s (prime and subcontractors) relevant experience for DSP Inverter designs/applications” or similar devices.  Id. at 56.  The RFP required offerors to describe and compare the type of work, scope of effort, physical location of effort, and the product/technology involved.  An offeror was to substantiate its experience by providing final versions of a statement of work (SOW) or performance work statement (PWS) for existing or previously performed contracts, citing the specific sections of the prior experience that are similar to the solicitation requirements here.  The RFP advised that a “requirement not specifically identified in the SOW/PWS or otherwise demonstrated through documents as discussed here will not be considered.”  Id. at 57.

Avionic and POC both submitted proposals.  POC’s proposal included a major subcontractor, which was proposed to perform approximately [DELETED] percent of the [DELETED] work.  AR, Tab 4, POC Tech. Proposal at 27. 

The agency evaluated Avionic’s proposal as having three risk reducers,[2] and no strengths, weaknesses, significant weaknesses, deficiencies, or uncertainties.  AR, Tab 6, Final Evaluation at 15-16.  The agency assigned Avionic’s proposal the following two risk reducers under the overall design element:  the design approach proposed components that have already been qualified and proven in other products, reducing the risk that performance requirements will not be met; and Avionic’s three-dimensional model of the inverter, showing the layout of the internal components, reduced the risk that the size requirements will not be met.  Id. at 15.  The agency assigned the third risk reducer under the experience element because:  the protester’s proposal demonstrated experience designing, building, qualifying, and fielding a system using DSP technology; and the protester’s familiarity with the necessary design and integration steps reduced program risk.  Id.  The agency evaluated the protester’s proposal as acceptable under the technical factor with a low risk rating.  Id. at 17-18.

The agency evaluated POC’s proposal as having four risk reducers, and no strengths, weaknesses, significant weaknesses, deficiencies, or uncertainties.  Id. at 24‑25.  The agency assigned POC’s proposal a risk reducer under the overall design element for [DELETED]--thus reducing the risk that the electrical performance requirements will not be met.  Id. at 24. 

The agency assigned POC’s proposal two risk reducers under the experience element, the first for experience designing, qualifying, and fielding avionic systems for multiple military aircraft.  The agency noted that “[b]oth [POC and its  subcontractor] have shown they are very experienced and knowledgeable in Avionic system qualification testing, and [DELETED], which serves to reduce the risk to the Government.”  Id.  The agency assigned the second risk reducer under the experience element for “multiple industry-standard manufacturing certifications,” which indicated the awardee was “well versed in manufacturing and delivering a compliant and consistent product” which will reduce “the risk of schedule delays when manufacturing begins of delivering compliant products.”  Id. 

The agency assigned a fourth and final risk reducer, under qualification testing, because [DELETED].  In this regard, the agency concluded that “there may be potential to do testing [DELETED] if schedule necessitates or [DELETED] if [DELETED] problems arise during the qualification process.”  Id.  POC’s proposal received the same evaluation ratings as Avionic’s proposal--acceptable under the technical factor with a low risk rating.  Id. at 26-27.

Avionic’s total evaluated price of $8,014,522 was higher than POC’s total evaluated price of $7,326,521.  Id. at 30.  The agency concluded that both prices were reasonable.

The SSEB chair reviewed the evaluation of Avionic’s and POC’s technical proposals, including the risk reducers assigned by the agency.  AR, Tab 8, Proposal Analysis Report at 4-5.  The SSEB chair also examined the underlying reasons for the ratings and concurred with them.  Id. at 4.  The SSEB chair found that the proposals were “technically similar, with neither having a clear technical advantage over the other.”  Id.  After reviewing the evaluations, the SSEB chair conducted a trade-off analysis.  See id. at 4‑8.  Because the proposals were technically similar, the SSEB chair found that POC’s proposal, which was lower priced, represented the best value to the government and recommended award to POC.  Id. at 8.

After reviewing the underlying evaluations, the source selection authority concurred with the recommendation of the SSEB Chair and selected POC for contract award.  AR, Tab 9, Source Selection Decision Document (SSDD) at 2.  This protest followed.

DISCUSSION

The protester challenges various aspects of the agency’s evaluation of the technical proposals.  Avionic asserts that the agency failed to assess risk in POC’s proposal for lack of experience with all PWS requirements, failed to assess the risk that POC would not properly manage its subcontractor, unreasonably awarded risk reducers to proposals, and overlooked strengths in Avionic’s proposal.  As explained below, we sustain the protest on two bases--that the record contains insufficient support for the agency’s risk assessment with respect to prior experience, and that the agency unreasonably assigned risk reducers.[3]

Failure to Consider Awardee’s Lack of Experience

First, the protester asserts that in assigning POC’s proposal a technical risk rating of low, the agency unreasonably failed to consider that POC and its subcontractor lacked critical experience.  As discussed above, offerors were required to provide a SOW (or PWS) for existing or previously performed contracts and to identify portions of those SOWs that corresponded to the current requirement.  As relevant to the protester’s allegation, the SOW for this competition included four types of required review under the technical reviews task:  systems requirements review (SRR), preliminary design review (PDR), critical design review (CDR), and production readiness review (PRR).  SOW ¶ 3.1.1. 

To satisfy the RFP’s requirement that offerors cite specific sections of SOWs to establish prior experience that is similar to the solicitation requirements, POC’s proposal included a chart listing the current PWS requirements by subparagraphs; for each of those subparagraphs, the protester noted which, if any, of POC’s prior contracts contained similar requirements.  See AR, Tab 4, POC Tech. Proposal at 176.  POC’s proposal also contained an analogous chart mapping the subcontractor’s prior experience to the current PWS requirements.  See id. at 368. 

Avionic argues that POC’s proposal failed to identify prior experience for POC or its subcontractor under several PWS requirements.  As a result, Avionic contends it was unreasonable for the agency not to assign risk to POC’s proposal for its lack of experience.  Supp. Protest at 3.  For example, the protester asserts that neither POC nor its subcontractor had experience with three of the four required types of review identified in the solicitation--SRR, CDR, and PRR--or with First Article Inspection, Qualification Test Plan, or Peculiar Support Equipment.  Id. at 3, citing AR, Tab 4, POC Tech. Proposal at 176-77, 368-69.  In response, the agency claims that the SOWs submitted by POC demonstrated the required experience, notwithstanding the content of the charts in the proposal.  See Supp. AR at 3-5.  The agency argues that “it would be unreasonable for the Agency to conclude POC had no experience with SRR, CDR, and PRR solely because POC’s cross reference matrix did not indicate that experience.”  Agency Response to Comments at 16.

In reviewing a protest challenging an agency’s evaluation of proposals, our Office will not reevaluate proposals; rather, we review the record to determine whether the agency’s evaluation was reasonable, consistent with the stated evaluation criteria, adequately documented, and consistent with applicable procurement statutes and regulations.  National Gov’t Servs. Inc., B-401063.2 et al., Jan. 30, 2012, 2012 CPD ¶ 59 at 5.  While we will not substitute our judgment for that of the agency, we will sustain a protest where the agency’s conclusions are inconsistent with the solicitation’s evaluation criteria, undocumented, or not reasonably based.  VariQ Corp., B‑414650.11, B-414650.15, May 30, 2018, 2018 CPD ¶ 199 at 6-7.

Here, the record contains no contemporaneous agency evaluation of whether Avionic’s or POC’s proposals met the SOW requirements.  In fact, the only contemporaneous discussion of this topic in the record is each offeror’s self-identification of its prior experience which was summarized in its proposal.  POC’s proposal did not claim experience for POC with any of the four types of reviews noted above.  See AR, Tab 4, POC Tech. Proposal at 176.  POC’s proposal claimed experience for its subcontractor in only one of the four types of review--PDR.  Id. at 368. 

In response to this issue, the agency submitted a supplemental report arguing that it concluded from other parts of POC’s proposal that the company had the review experience required by the solicitation--even though POC’s proposal did not claim experience in these areas.  Supp. AR at 3-5.  In addition, the agency provided an affidavit from the team leader in charge of the technical evaluation, who is also the aerospace engineer responsible for the helicopter program that includes the helicopter in this procurement.  AR, Tab 17, Aff. of Technical Team Leader at ¶ 2.  The engineer explained that, notwithstanding the content of the charts in the awardee’s proposal, as well as the rest of the proposal, it was his judgment that the SOWs demonstrated the experience not claimed in the chart.  Id. at ¶¶ 5-6.  The engineer mentions a document referenced by POC in its proposal that the engineer explains is evidence that POC has the required experience.  Id. at ¶ 6.

Our Office will not limit its review of an agency’s evaluation to contemporaneously documented evidence, but instead will consider all the information provided, including a party’s arguments and explanations.  CRAssociates, Inc., B-418194, Jan. 23, 2020, 2020 CPD ¶ 80 at 5.  Post-protest explanations that provide a detailed rationale for contemporaneous conclusions, and simply fill in previously unrecorded details, will generally be considered in our review as long as those explanations are credible and consistent with the contemporaneous record.  Id.  When an agency’s post-protest defense of its evaluation is not supported by the contemporaneous record, or is inconsistent with the record, such explanations are unpersuasive and will be afforded little weight.  Celta Servs., Inc., B‑411835, B‑411835.2, Nov.2, 2015, 2015 CPD ¶ 362 at 8-9.

Here, the record contains no contemporaneous agency evaluation of whether proposals satisfied the many prior experience requirements in the PWS.  See Supp. AR at 4-9 (referencing no contemporaneous agency evaluation in response to the protester’s claim that the agency failed to identify the lack of POC’s experience as a risk); Agency Response to Protester’s Comments at 16-19 (same).  The team leader’s affidavit provides the only support for the agency’s evaluation of these requirements.  Because the team leader’s affidavit, created post-protest, is the sole explanation of why the agency credited POC with experience beyond that claimed by POC in its proposal, we accord it little weight.  We thus have an insufficient basis to conclude that the Navy reasonably considered the risk of POC’s lack of experience--as reflected in POC’s own proposal.  Accordingly, we sustain this argument.

Risk Associated with POC’s Management of Its Subcontractor

Avionic also contends that the agency unreasonably failed to assign risk to POC’s proposal for its heavy reliance on its subcontractor.  The protester lists numerous factors that it contends required the agency’s assignment of risk to POC’s proposal, including that POC itself does not have experience with inverters, and that POC’s subcontractor is proposed to perform [DELETED] percent of the requirement.  Comments at 6.  The Navy argues that the RFP did not require the evaluation of an offeror’s experience managing subcontractors, its usage of subcontractors, or its planned approach to managing subcontractors on this contract.  Supp. AR at 9-10.  The protester does not challenge the agency’s assertion that the RFP did not provide for such an evaluation.  See Comments at 7.

In the absence of an express solicitation provision that the agency would assess risk based on the use of subcontractors, we will not find unreasonable an agency’s failure to assign such a risk.  Sigmatech, Inc., B-415028.3, B-415028.4, Sept. 11, 2018, 2018 CPD ¶ 336 at 7.  A protester’s disagreement with the agency’s judgment as to the risk attendant to the use of subcontractors does not provide a basis to sustain the protest.  Id.

The RFP did not mandate that the agency evaluate the risk associated with an offeror’s use of, or management of, its subcontractors.  Without such a requirement, the record provides no basis to conclude that the agency unreasonably failed to evaluate the ability of the awardee to manage its subcontractor or unreasonably failed to assign the awardee’s proposal increased risk for the use of a subcontractor.  Sigmatech, Inc., supra.  As a result, this allegation provides no basis on which to sustain the protest.

Awardee’s Risk Reducer for [DELETED]

Next, Avionic contends that the Navy unreasonably assessed risk reducers in POC’s proposal for [DELETED] when POC lacks experience in conducting the testing.  Comments at 8.

The Navy explains that this “risk reducer was for [DELETED] that could enable testing to be done [DELETED] if schedule necessitates or that testing could continue to be completed [DELETED].”  Supp.  AR at 11; see AR, Tab 6, Final Evaluation at 24 (noting that POC’s proposal received a risk reducer because [DELETED], so there may be potential to do testing [DELETED] if schedule necessitates or [DELETED] if [DELETED] problems arise during the qualification process”).  The agency awarded the risk reducer for two reasons:  the potential to do testing [DELETED] and [DELETED] if problems arise during the testing. 

Avionic contends that, since POC lacks experience with qualification testing for inverters, the agency unreasonably assigned this risk reducer, because it is not clear that POC could effectively [DELETED] in the performance of the contract.  Comments at 8.  We note that it is not clear from the record that [DELETED], which would require [DELETED].  It seems reasonable, however, for the agency to recognize some schedule risk reduction in the [DELETED].  We thus find no merit to the allegation that the agency unreasonably assigned POC’s proposal a risk reducer for offering [DELETED].[4]

Awardee’s Risk Reducer for Multiple Manufacturing Certificates

Avionic also argues that the agency unreasonably failed to award its proposal a risk reducer for demonstrating that it can timely deliver a product manufactured in its own facilities.  In the protester’s view, the agency unreasonably assigned a risk reducer to the awardee’s proposal for having multiple manufacturing certifications, but did not assign a risk reducer for the protester’s experience manufacturing and making timely deliveries.  Comments at 9. 

The agency claims, correctly, that the awardee had multiple manufacturing certifications, and the protester did not.  See Supp. AR at 14-15; Agency Response to Comments at 12.  The RFP, however, did not require manufacturing certificates, and the risk reducer here was not assigned simply for possession of those certificates.  Instead, the risk reducer was assigned because the multiple manufacturing certifications “indicate[d] [the awardee was] well versed in manufacturing and delivering a compliant and consistent product,” and, thus, “the risk of schedule delays when manufacturing begins” is reduced.”  AR, Tab 6, Final Evaluation at 24.

Lessening schedule delays as part of a reduction of risk was an announced evaluation criterion.  The risk reducer at issue here was awarded because, in the agency’s view, POC’s possession of multiple manufacturing certifications lessened the risk of schedule delays during the manufacturing phase of contract performance.  During the pendency of this protest, the agency failed to address or rebut the protester’s assertion that Avionic’s proposal should have been awarded a risk reducer for its manufacturing experience and making timely deliveries.  The agency argues instead that the protester did not also possess multiple manufacturing certifications; this argument, however, fails to address Avionic’s assertion. 

In our view, performance experience can provide a basis for reducing risk.  The Navy provides no rationale for why possessing manufacturing certifications is better evidence of being “well versed in manufacturing and delivering a compliant and consistent product,” than a track record of successful manufacturing at one’s own facility--which Avionic has.  In summary, whether the flaw is described as disparate treatment (which Avionic argues) or an unreasonable evaluation, the record here does not provide sufficient support for the reasonableness of the agency’s evaluation.  Accordingly, we sustain the protest on this basis, as well.

Overlooked Strengths in Protester’s Proposal

Finally, Avionic contends that the agency unreasonably failed to identify numerous strengths in its proposal.  As noted above, the agency awarded neither proposal strengths.  For example, Avionic argues that its proposal met the definition of a strength for its approach to qualification and the maturity of its design.

The RFP defined a strength as “[a]n aspect of an Offeror’s proposal that has merit or exceeds specified performance or capability requirements in a way that will be advantageous to the Government during contract performance.”  RFP at 68. Avionic argues that its proposal met the definition of a strength for its approach to qualification and the maturity of its design.  Protest at 12-13.  Avionic also notes that the agency assigned its proposal two risk reducers under overall design.  Id. at 12.  The protester argues, however, that the agency erred in concluding that a design that was already qualified and shown that it could meet requirements was merely a risk reducer and not a strength.  Id.  Given that the maturity of its DSP design “would unquestionably ‘be advantageous to the Government during contract performance,’ the definition of ‘strength’ in the RFP,” the protester claims the agency unreasonably failed to evaluate its proposal as having a strength.  Id. at 13.  That, in turn, the protester asserts, resulted in an unreasonably low adjectival rating for its technical proposal.  Id.

The agency demonstrated in its evaluation that it recognized and appreciated the advantages in the protester’s proposal.  The source selection authority was aware of and considered the positive aspects of the protester’s design.  AR, Tab 9, SSDD at 1-2.  The protester’s challenge is not that the agency failed to identify advantages in Avionic’s proposal; rather, Avionic contests the weight accorded those advantages. 

The protester’s challenge to the agency’s evaluation amounts to mere disagreement with the weight the agency assigned to Avionic’s design.  AdvancedMedcorp.; TrustSolutions, LLC, B-404910.4 et al., Jan. 17, 2012, 2012 CPD ¶ 25 at 21 (noting that, for the most part, a protester’s disagreement with the weight or importance attached to particular proposal benefits provides no basis on which to sustain a protest); Kathryn Huddleston & Assocs., Ltd., B-294035, July 30, 2004, 2004 CPD ¶ 142 at 2 (noting that, while the protester disagreed with the weight that a favorable finding was given in the technical evaluation, such disagreement, without more, does not provide a basis for our Office to object to the agency’s evaluation). 

Moreover, the record does not support a contention that, had the agency awarded the protester’s proposal a strength rather than two risk reducers, the agency would necessarily have evaluated the protester’s proposal as good, rather than acceptable.[5]  A proposal that was awarded a strength may have been evaluated as good, but was not required to be.  In any event, a source selection authority must look behind adjectival ratings and consider the substance of competing proposals, notwithstanding the ratings assigned.  Biswas Info. Tech. Solutions, Inc., B-414760.3, B-414760.4, Oct. 5, 2018, 2018 CPD ¶ 332 at 9 (noting that source selection must look behind the adjectival ratings, and must reflect the qualitative assessment of the underlying differences among competing offers); Protection Strategies, Inc., B-414573.3, Nov. 9, 2017, 2017 CPD ¶ 348 at 6 (same).  The record here demonstrates that the source selection authority did exactly this.  For these reasons, we find the record does not support the protester’s contention that the agency unreasonably failed to assign Avionic’s proposal strengths instead of risk reducers.

CONCLUSION AND RECOMMENDATION

Because the contemporaneous record does not support a finding that the evaluation was reasonable and in accordance with procurement law and the terms of the solicitation, we sustain Avionic’s protest.  We recommend that the agency reevaluate proposals, make a new best-value tradeoff decision, and document its conclusions in a manner consistent with the discussion above.  In addition, we recommend that the Navy reimburse Avionic the costs associated with filing and pursuing its protest, including reasonable attorneys’ fees.  4 C.F.R. § 21.8(d)(1).  Avionic should submit its claim for costs, detailing and certifying the time expended and costs incurred, to the contracting agency within 60 days after receipt of this decision.  4 C.F.R. § 21.8(f)(1).

The protest is sustained.

Thomas H. Armstrong
General Counsel

---