Matter of: Brunswick Defense File: B-255764 Date: March 30, 1994 94-1 CPD Para. 225

Matter of: Brunswick Defense File: B-255764 Date: March 30, 1994 94-1 CPD Para. 225

PROCUREMENT Competitive Negotiation Contract awards Administrative discretion Cost/technical tradeoffs Technical superiority Agency reasonably determined that the protester's lower-priced proposal did not represent the best value to the government in a procurement for a quantity of target missiles, where the agency, having considered all relevant and available data, evaluated the protester's proposal as unacceptable and high risk under the past performance and systemic improvement evaluation factor, as compared to the awardee's highly satisfactory and low risk assessment under this factor, and reasonably concluded that the protester's unacceptable performance likely would ultimately cost the agency more than the awardee's price premium.

Attorneys

DECISION

We deny the protest.

The AQM-37C target missile is a supersonic, high altitude, air-launched, expendable target that simulates enemy threats. Beech was the original manufacturer and for nearly 30 years the sole qualified supplier of this missile. In 1988, the agency, seeking a second source for the missile, awarded contract No. N00019-88-C-0340 (contract -0340) to Brunswick for 10 missiles. In 1991, Brunswick successfully completed qualification testing. Thereafter, awards were made to Beech and Brunswick for the Navy's 1991 and 1992 production requirements, under contract Nos. N00019-91-C-0163 (contract -0163) and N00019-92-C-0155 (contract -0155), respectively. The agency decided to make a single award for its 1993 missile requirements.

The RFP, issued March 9, 1993, contemplated the award of a fixed-price contract for 50 to 120 missiles per year for a base and 4 option years. Detailed performance, design and function specifications were provided for the missile. In pertinent part, the specifications required a minimum flight reliability rate of 90 percent, excluding flight failures caused by government furnished parts or systems.

The RFP established a "best value" evaluation scheme, based upon three factors in descending order of importance:

A. Price B. Technical C. Past performance and systemic improvement

Of these criteria, price was stated to be significantly more important than the technical factor, which was moderately more important than the past performance and systemic improvement factor.[1] The RFP also warned that the low-priced, acceptable proposal might not receive the award, if a higher-priced proposal was sufficiently superior in the other evaluation areas to justify the price premium.

Detailed proposal preparation instructions explained how the Navy would evaluate proposals under the past performance and systemic improvement factor, the evaluation area relevant to this protest. For the past performance element, offerors were informed that the Navy would gauge the offeror's "past and present performance by review of data presented by the offeror, data in existing government data bases, data from cognizant procuring and contract administration offices, and data from on-site surveys." This data would be used in a performance risk assessment, which would consider the effectiveness of the offeror's actions in resolving any problems uncovered by the data, but would presume that any problem not addressed by the offeror was still in existence.

For the second evaluated element of this factor, systemic improvement, offerors were asked to demonstrate "the ability to isolate past and present problems down to a root cause and to take systemic improvement management actions to resolve the root cause of the problems." Specifically, the offeror was to propose a corrective action plan to resolve the particular performance problems uncovered by the data, since "past and present performance is not presumed to be perfect."

Brunswick and Beech submitted initial proposals by the April 5 closing date, and discussions were conducted with each firm. Best and final offers (BAFO) were received on May 5 and evaluated by the technical evaluation team in accordance with the stated solicitation criteria and the agency's source selection plan for this procurement. Under this plan, each proposal factor or subfactor was evaluated as either outstanding, highly satisfactory, satisfactory, marginal or unacceptable, and risk for each factor and subfactor was assessed as high, medium, or low.

Brunswick's BAFO price was approximately $73 million, inclusive of options, while Beech's price was approximately $91 million. Brunswick's and Beech's technical proposals were evaluated as satisfactory overall, although Beech's proposal earned a low risk rating, as opposed to Brunswick's medium risk rating. There was a dramatic disparity between the two offerors' past performance/systemic improvement factor ratings. Here, Beech was rated highly satisfactory, with a low performance risk, while Brunswick was rated unacceptable, with a high performance risk. The primary discriminators in the past performance/systemic improvement evaluation were each offeror's flight reliability rate and Brunswick's inability to achieve systemic improvement.

Brunswick's flight reliability rate was based upon flights of missiles produced under its production contract -0163.[2] Of the initial five flights, three flights suffered from "low thrust," failing to accelerate in accordance with the RFP guaranteed performance specifications, and were classed as failures. The launch of an additional two missiles, which implemented the protester's corrective action plan for remedying the thrust problem, resulted in one of its "corrected" missiles suffering from low thrust. This gave Brunswick a 43 percent flight reliability rate (three successes in seven flights), as compared to Beech's 96 percent flight reliability rate for missiles flown over the past 11 quarters.

In summarizing Brunswick's performance in the initial five flights, the Navy's evaluators expressed concern that Brunswick had not determined the precise cause of the thrust failure in the three unsuccessful flights, and had based its corrective action plans on hypotheses alone. Concerning the additional two "corrected" missile flights, the evaluators noted that Brunswick "had two years to fix" this recurrent low thrust problem, and its failure to do so evidenced "a lack of effective corrective action."

Brunswick's unreliable flight rate, combined with its failure to correct performance problems, persuaded the Navy's procurement review board (PRB) to unanimously recommend award to Beech, despite Brunswick's price advantage of 18 percent. The PRB concluded that Brunswick's unacceptable performance eclipsed any price advantage associated with its proposal.

On June 3, the PRB briefed the source selection authority (SSA) on the technical evaluation and the PRB's award recommendation. The SSA decided to defer a selection decision until Brunswick had launched additional missiles and developed a broader performance base. Brunswick launched three additional missiles following the PRB's award recommendation; two of these succeeded and one failed, because it accelerated considerably faster than specified and burned out early, i.e., a "high thrust problem." This gave Brunswick an overall 50 percent flight reliability rate under contract -0163 (10 flights, 5 successes), and prompted the Navy to issue a breach of warranty notice under that contract on July 16 because the protester had failed to achieve the required 90 percent reliability rate specified in its contract.

The agency decided to conduct further discussions with the offerors; this would permit the agency's evaluators to review Brunswick's most recent flight tests and afford the protester a chance to address its performance problems. By the time the agency issued the discussion questions, Brunswick had launched an additional two missiles. One was a success, but the other's engine failed to ignite and it glided to the ground, an "unpowered" flight. Accordingly, during discussions, the agency advised Brunswick that 6 of the 12 missiles launched under Brunswick's contract -0163 did not meet the AQM-37C specification requirements. The Navy asked Brunswick to describe its failure analysis for these flights and to explain how it proposed to meet the specification requirements in the future.

Four of Brunswick's six flight failures suffered from "low thrust problems," and the protester explained during discussions that its investigation of these problems had isolated "the Brunswick-manufactured portion of the system upstream of the engine since Beech was not experiencing similar low thrust problems."[3] Brunswick identified two separate hardware deficiencies that may have produced the low thrust problem: (1) a leak in the missile's nitrogen pressurization system and/or (2) clogging in the missile's filter system. Brunswick attributed the low thrust encountered in the four unsuccessful flights to one or both of these hardware deficiencies.

With respect to the "high thrust" flight, Brunswick stated that an idiosyncrasy in the engine selector knob was the most likely cause of the problem. A slight misalignment of that knob, Brunswick explained, produces a higher thrust rate than expected. The protester stated that the problem with the selector knob was characteristic of the AQM-37C missile design, and was not a Brunswick hardware deficiency, and recommended that the Navy update its preflight procedure to explain how to set the knob. With respect to the final unsuccessful flight, Brunswick had not yet developed a failure analysis, but speculated that the missile's loss of power may have resulted from "an electrical anomaly . . . not related to the previous problems." In assuring the Navy that future target flights would meet the specification requirements, Brunswick stated that, "to the best of its knowledge [it] complies with the [missile's] specification requirements--test, reliability, quality, etc.," and that its corrective action plans had effectively eliminated each of the problems encountered.

The agency thoroughly reviewed Brunswick's performance record and discussion responses, regarding the protester's unacceptable past performance/systemic improvement factor rating. Several performance areas were considered--the protester's performance during qualification testing under contract -0340, its performance under a software development contract,[4] its flight reliability rate under contract -0163, and its failure analysis of the unsuccessful flights under that contract.

In considering the protester's flight record, the agency noted that 4 of the protester's 10 qualification flights had failed, and that 6 of the protester's 12 production flights had failed. In summarizing the protester's record, the agency stated that,

"[w]hat is most disturbing about Brunswick['s] failures . . . is the significant number of `isolated' anomalies. During the Brunswick qualification program, the [missile] exhibited at least five unrelated anomalies . . . . And during flight operations of production vehicles, at least four unrelated anomalies were discovered."

The agency stated that Brunswick's investigation into these anomalies proved completely inconclusive in some cases, while, in other cases, Brunswick's corrective action plan was based on "mere hypothesis, not concrete evidence or scientific testing." Most significant, in the agency's opinion, was that Brunswick had not adopted an engineering approach that anticipated potential problems in the system before a failure occurred, but, in all cases, reacted to the anomaly after flight failure and discontinued its investigation after identifying a single "likely" cause. The agency concluded that, "as long as Brunswick adheres to this `fix it when it breaks' engineering philosophy, we can continue to expect more `isolated' failures on future production targets."

The agency's conclusions in this regard were supported with a detailed critique of the protester's failure analysis of the four "low thrust" production missiles. The Navy's critique identified numerous inconsistencies, flaws in logic, and inadequate testing methods in Brunswick's failure analysis, which it stated were characteristic of the protester's basic systems engineering approach. For example, Brunswick identified a clogged filter in the missile system as responsible for the low thrust in one, and possibly two, unsuccessful launches; its investigation disclosed that the aluminum tip of the filter does not withstand simulated fuel flow conditions. The agency noted that the aluminum tip was a design change which Brunswick introduced into the original technical data package and which should have been subjected to flow tests before the design change was made. In addition, the agency faulted Brunswick for investigating only a single failure point and for ignoring the possibility of multiple failure points, the type of isolated thinking that had allowed the aluminum tip deficiency to escape Brunswick's attention during its earlier low thrust investigations.

Based upon its evaluation of Brunswick's complete performance record, the agency concluded that the protester's past performance/systemic improvement factor rating remained unacceptable with high risk. On September 23, the PRB again recommended that award be made to Beech as representing the best value to the government. Specifically, the PRB stated:

"Despite the fact that the Brunswick BAFO is $17.3 [million] lower than Beech's for equal target quantities and price is the most important evaluation factor, Brunswick's low (50 [percent]) production vehicle flight success rate to date, when applied to their potential contract quantity of 546 targets, yields 103 fewer specification compliant targets than would a Beech award[5] . . . . The PRB cannot find evidence in either Brunswick's offer nor in the Government data bases that would indicate Brunswick possesses the organic systems engineering and systemic improvement problem solving capabilities and processes which could improve this flight success rate to any reasonable reliability level. It is for this reason, combined with Brunswick's `unacceptable' Past Performance and Systemic Improvement rating and the analysis stated herein that the PRB cannot find that Brunswick represents best value to the Government."

The PRB found that this difference in flight reliability (Brunswick's 50 percent reliability rate as compared to Beech's 96 percent rate) carried with it significant costs, including not only the missile replacement cost, but the range and fleet operation costs, and the lost training opportunities. Taking into account only the missile replacement costs and the range operation costs, which could be easily quantified, the PRB estimated that Brunswick's low reliability rate would cost the agency approximately $33 million.[6] Thus, although Brunswick's price was approximately $17 million, or 18 percent, less than Beech's, the PRB concluded that Brunswick's unacceptable performance, combined with its inability to effect systemic improvement, overshadowed its price advantage.[7]

On September 28, the SSA adopted the PRB's reasoning and recommendation and selected Beech for award. Over the ensuing month, the award determination was briefed to a number of agency management officials, and award was made to Beech on November 5. Shortly before award, on November 1, Brunswick successfully launched its first four missiles under contract -0155. The SSA learned of the successful launches on November 3, but did not reopen the evaluation process, since he had already made the selection decision on September 28 and concluded that the new information did not affect his decision to proceed with award.[8]

Brunswick protests that the Navy lacked a rational basis to determine that its proposal was unacceptable for the past performance and systemic improvement factor. The protester principally contends that the agency should have used a higher percentage of successful flights in calculating its flight reliability rate. Had the agency done so, the protester claims that it, and not Beech, would have received the contract award.

Source selection officials in negotiated procurements have broad discretion in determining the manner and extent to which they will make use of the technical and cost evaluation results. Grey Advertising, Inc., 55 Comp. Gen. 1111 (1976), 76-1 CPD Para. 325. Agencies may make cost/technical tradeoffs in deciding between competing proposals; the propriety of such a tradeoff turns not on the difference in technical scores or ratings, per se, but on whether the selection official's judgment concerning the significance of that difference was reasonable and adequately justified in light of the RFP evaluation scheme. DynCorp, 71 Comp.Gen. 129 (1991), 91-2 CPD Para. 575; Wyle Labs., Inc.; Latecoere Int'l, Inc., 69 Comp.Gen. 648 (1990), 90-2 CPD Para. 107. A protester's mere disagreement with the agency's evaluation determination does not demonstrate that the evaluation is unreasonable. Universal Technologies, Inc.; Spacecraft, Inc., B-248808.2, et al., Sept. 28, 1992, 92-2 CPD Para. 212. Rather, we will review the agency's justification based upon its rationality and consistency with the established evaluation factors. Wyle Labs., Inc.; Latecoere Int'l, Inc., supra; DynCorp, supra.

Brunswick argues that the agency improperly understated its flight reliability rate, and that, under a properly estimated flight reliability rate, its lower-priced proposal would have been found to represent the best value. With respect to its production contract -0163, the protester believes that it was responsible for only four unsuccessful flights, the "low thrust" flights. With respect to the "high thrust" flight and the "unpowered" flight, Brunswick cites either government launch personnel or a defect in the AQM-37C design specifications as the source of failure, which allegedly should not have been counted against Brunswick's flight reliability rate. In addition, the protester claims that the Navy could have easily reopened the evaluation to consider Brunswick's four successful flights on November 1, 1 month after the selection decision was made. Finally, the protester claims that the Navy understated the number of successful flights it maintained under its qualification contract -0340 and that these flights should have been used in calculating its flight reliability rate. Had all this been done in the manner suggested by the protester, its flight reliability rate would have been 77.3 percent. The protester states that, under the PRB's cost/technical tradeoff, a success rate greater than 75 percent would have turned the selection decision in favor of Brunswick.[9]

Contrary to the protester's arguments, we find that the agency was justified in calculating Brunswick's flight reliability rate based upon the 12 production missiles that were launched during the evaluation period under contract -0163 and was not required to consider Brunswick's post-selection flights or qualification flights in the award selection process.

With respect to the four successful flights launched under contract -0155, these flights took place more than 1 month after the agency had concluded its evaluation of proposals and the SSA had made his selection decision. Contrary to the protester's arguments, the RFP did not contemplate that award would be delayed with each new flight launched under the offerors' ongoing missile production contracts.[10] Moreover, the SSA had already extended the evaluation once, notwithstanding the PRB's June recommendation of an award to Beech, so that Brunswick might improve its unacceptable past performance/systemic improvement rating through the benefit of some additional flights, as well as discussions. We do not think that the SSA was required to delay the award for however long it might take Brunswick to improve its flight record to an acceptable level, particularly given Beech's expiring BAFO. See Roarda, Inc., B-204524.5, May 7, 1982, 82-1 CPD Para. 438.

Brunswick also argues that the Navy unreasonably excluded the flights launched under its qualification contract in determining its flight reliability rate; under Brunswick's interpretation, it would have achieved a reliability rate of 72 percent, considering both the qualification flights and the production flights under contract -0163.[11] Brunswick argues that there is no reasonable difference between the production missiles and the qualification missiles to warrant their exclusion from its flight reliability rate, since all missiles were built to the same specifications.

What Brunswick ignores is that the testing standards applicable to the qualification contract were more lenient than those applicable to the production contract, which will apply to missiles procured under this RFP. For example, in the qualification contract, it appears that the Navy relieved Brunswick from testing the dive angle of its flights at the more severe angles required by the AQM-37C guaranteed performance specifications. Brunswick recognizes that the two contracts were distinguished by "[agency]-provided changes associated with testing," but argues that these changes "did not affect performance or design." [Emphasis in original.] However, it is impossible to tell whether the relaxation of the testing standards applicable to the qualification contract affected performance or design, since the qualification missiles were not tested for compliance against each guaranteed performance specification. Moreover, as Brunswick's expert recognizes, the manufacture of missiles for qualification purposes is different than that ultimately employed in the production mode. Accordingly, we think that the Navy could reasonably view Brunswick's flights under its qualification contract as not being a particularly relevant measure of flight reliability as compared to the flights under its production contract, which employed the same standards and manufacturing methods that will apply to this production contract.

Brunswick accepts responsibility for the 4 "low thrust" failures of the 12 flights launched under its production contract -0163, but disclaims responsibility for the other two flight failures. While the expert employed by Brunswick in connection with this protest hypothesizes alternate causes for the 2 flight failures, he does not demonstrate that the agency's attribution of fault to Brunswick was unreasonable.[12] Indeed, the record still does not substantiate Brunswick's assertions that the "high thrust" and "unpowered" failures were not its fault, but contains sufficient evidence from which the agency could conclude that Brunswick was responsible for its poor flight reliability rate. See Pannesma Co. Ltd., B-251688, Apr. 19, 1993, 93-1 CPD Para. 333. Based upon our review, the agency could reasonably attribute responsibility for these flight failures to Brunswick, and, even if it could not, Brunswick was not prejudiced. Specifically, the protester admittedly can only approach the 75 percent reliability rate by counting the qualification and post-selection flights, which we have already found should not be counted.

In focusing on its flight reliability rate, Brunswick largely ignores the agency's legitimate concerns about its systemic improvement capability, which was one of the two elements supporting the protester's unacceptable rating for the past performance/systemic improvement factor. Nor does Brunswick address whether it could ever achieve the 90 percent reliability rate mandated by the RFP. In our view, the Navy was justifiably concerned that Brunswick, given its apparently passive systems engineering approach, would not be able to produce operable missiles with the required 90 percent reliability rate, and Brunswick has not alleged otherwise. For example, even under the protester's interpretation of its flight record, Brunswick's flight reliability did not improve, but deteriorated, from its qualification contract -0340 to its production contract -0163. In addition, Brunswick demonstrated a continuing inability to remedy a "low thrust" problem in its missiles even after implementing a corrective action plan, and unrelated anomalies continued to surface in successive flight failures. Thus, we think that the Navy, having reviewed the protester's entire flight record and discussion responses, could reasonably judge Brunswick's systemic improvement capability as unacceptable with high risk, and this concern is not assuaged by Brunswick's attempt to boost its flight reliability rate to a 77.3 percent figure.

In summary, our review of the record reflects that the Navy's past performance/systemic improvement evaluation comported with the RFP evaluation scheme; that it relied upon all information then available and relevant to Brunswick's performance; and that it amply supported the protester's unacceptable performance rating and the tradeoff decision in favor of a more reliable, higher-priced proposal. Under the circumstances, given Brunswick's unacceptable past performance/systemic improvement rating, the agency reasonably chose to pay the associated price premium in making award to Beech, which had a historical reliability rate well in excess of the contract requirements.

The protest is denied.

1. The technical factor included four weighted subfactors: (i) Compliance, (ii) Systems engineering, (iii) Schedule/Management, and (iv) Supportable system.

2. All flight tests performed during the proposal evaluation period for this procurement involved Brunswick missiles supplied under contract -0163.

3. Brunswick supplemented its discussion response with the fault analysis it developed pursuant to the Navy's breach of warranty notice.

4. The contract awarded under this RFP also requires the development of some new software to update the AQM-37C missile.

5. It appears that the agency improperly calculated that Brunswick would produce 103 fewer operable targets than Beech, assuming a production quantity of 546 targets. In fact, by our calculations, an award to Brunswick for this quantity would yield 251 fewer operable targets than would an award to Beech, based upon the two offerors' respective reliability rates.

6. The PRB estimated that 392 missiles would be funded during the contract. Applying Beech's 96 percent reliability rate to this quantity yields 376 operable missiles; applying Brunswick's 50 reliability rate yields 196 operable missiles, 180 fewer missiles than Beech or 48 percent less. The agency estimated that these 180 inoperable Brunswick missiles would cost $23 million to replace, and $9 million in wasted range operations, for a total of $33 million. Meanwhile, Beech's price for 392 missiles was only $15 million more than Brunswick's.

7. The PRB considered the technical proposals of both offerors to be roughly comparable, so that this evaluation factor did not sway the tradeoff.

8. The BAFOs of both offerors were set to expire on November 6, and Beech advised during discussions in September that it could not extend its BAFO after this date because of expiring subcontractor commitments.

9. Based upon the PRB's hypothetical tradeoff analysis outlined in footnote 6 infra, a Brunswick success rate of 75 percent would reduce its replacement costs for inoperable targets from $33 million to $15 million. If this figure were used in the analysis, Brunswick's low-priced proposal would be considered most cost effective, even assuming its lower reliability rate.

10. Recognizing that Brunswick had an ongoing production contract and that its flight record might improve, the PRB did recommend that the agency reevaluate Brunswick's performance before it decided to exercise Beech's contract options.

11. The Navy disputes Brunswick's interpretation of its flight record under both these contracts. In any event, even Brunswick concedes that a 72 percent reliability rate would not have swayed the evaluation in its favor.

12. Brunswick did not present the agency during discussions or in response to the breach of warranty notice with the analysis that it now offers to disclaim its responsibility for one of the two contested flight failures--the "unpowered" flight.