DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Lockheed Martin Maritime Systems & Sensors

File: B-299766; B-299766.2

Date: August 10, 2007

DIGEST

Protest is sustained where solicitation provided for tow-off testing of prototype arrays according to a procedure in which each offeror was explicitly permitted to participate in installation and check-out of its prototype, and then granted a single opportunity to repair any failure of its prototype immediately prior to towed testing, but the agency did not notify the protester that its prototype had already failed before arrival at the testing site, and did not allow the protester to repair its prototype, and as a result, the agency was unable to evaluate significant aspects of the performance of the protester's array.

Lockheed Martin Maritime Systems & Sensors protests the award of a contract to Chesapeake Sciences Corporation by the Department of the Navy, Naval Sea Systems Command (NAVSEA), under request for proposals (RFP) No. N00024'05'R'6259, issued as a follow-on to development contracts awarded under a broad agency announcement to procure production towed-arrays for the TB-34 Next Generation Fat-Line Towed Array program. Lockheed argues that the Navy failed to properly remedy a breach of procurement integrity, did not follow the procedure for testing and evaluating prototype arrays specified in the RFP, and made a defective source selection decision.

We sustain the protest.

BACKGROUND

The Navy conducted this competition for the production of towed arrays[1] between the two contractors that had delivered prototypes of their own design under separate contracts. Thus, the procurement process began in September 2004 when, after competition among four firms under a broad agency announcement, the Navy awarded separate contracts to Chesapeake and Lockheed for each to develop a prototype of a next-generation towed array, for a subsequent head-to-head competition for a production contract. Under the prototype contracts, each contractor designed and delivered a prototype of its array in October 2005.

The Navy experienced problems with both prototypes in mid-March 2006, and returned each to the respective manufacturer for repairs from March 13 to March 19, 2006.[2] Shortly thereafter, the Navy issued the RFP here on March 23 to Chesapeake and Lockheed, requesting production offers. While the RFP requested a technical proposal, the evaluation of the performance of each offeror's prototype array, as discussed below, formed the basis for a significant part of the technical evaluation. In this regard, it should be noted that certain testing of the prototype arrays was ongoing before and after issuance of the RFP for this competition.

The RFP described the non-price evaluation in terms of three factors: technical performance, contractor statement of work, and past performance.[3] RFP at 178. Of these three, the RFP specified that the technical performance factor would be significantly more important than the contractor statement of work and past performance factors combined. In addition, the non-price factors were significantly more important than cost/price. The RFP also made clear that the operation of the prototypes would be a significant aspect of the technical performance evaluation. RFP at 170-71.

The RFP described several stages of the evaluation of the protoypes, including acoustic calibration, tow-off testing, and anechoic chamber testing. These were described in a separate section of the RFP labeled –Evaluation Plan for the Next-Generation Fat-Line Towed Array Testing.— RFP, Evaluation Plan, at 1. Initial acoustic calibration took place at the NavalUnderseaWarfareCenter's Underwater Sound Reference Detachment, near Leesburg, Florida, while towed testing was planned for LakePend Oreille in Idaho using the Research Vessel Chinook. RFP, Evaluation Plan, at 20. Regarding the towed testing, the RFP provided for vendor involvement in a pierside checkout, at the outset, thus:

The array and tow cable being tested will be installed aboard the Chinook winch and integrated with the shipboard laboratory equipment by vendor personnel using appropriate procedures. The Navy personnel, and vendor personnel if available, will power the array and verify proper operation in accordance with vendor guidelines. After proper operation is established, Navy personnel and/or vendor personnel will command the array to turn on each electrical CAL[ibration] signal in sequence . . . .

Id.[4]

The RFP Evaluation Plan also incorporated a document titled –Evaluation Ground Rules,— that described two phases in the tow-off. After installation and pier-side checkout, –phase 1— would include operation of the array with the vendor present, while –phase 2— would be conducted without the vendor present. The ground rules explained the purpose of each phase:

C.2 TOW-OFF EVALUATION RULES

* * * * *

Array vendors are welcome to participate in the phase-1 testing. . . .

In the event of an array failure during phase 1, the array vendor will have no more than 7 days to make repairs to the array. There will not be a second phase-1 test period after a failure. Once the array is repaired, it will be in control of the government and will proceed to phase 2.

The evaluation team will conduct the second phase of the tow-off without vendor participation, but the vendors are welcome to leave their troubleshooting equipment aboard the research vessel during phase 2 for the evaluation team's use in the event of a failure. Phase 2 is expected to require several days of towing testing. Daily briefings will not be scheduled during phase 2. The evaluation team will contact a vendor if necessary.

RFP, Evaluation Plan, app. C, at C-2 to C-3.

The Evaluation Plan also noted that –[t]his document lists the evaluation testing approach and actual tests that may be performed in the design competition; any of these tests may be performed, but the government may not complete all the tests listed.—[5] RFP, Evaluation Plan, at 2.

Beginning on March 27, the Navy began conducting a multi-day process of calibration with the arrays at the Leesburg facility, starting with Chesapeake's array. On March 29 the Navy began calibration of Lockheed's array, and encountered a second failure of the device.[6] The Navy explains that

[o]n the late afternoon of 30 March 2006, the evaluation team was approximately three-quarters through the calibration process at Leesburg on the [Lockheed] TB-34 prototype when the processing system stopped receiving data. The next morning, using troubleshooting procedures previously provided by [Lockheed] . . . prior to release of the solicitation, the evaluation team isolated the failure to the [Lockheed] array's [deleted]. Only the remaining wideband calibration was able to be completed. As a result of the failure in the [deleted], the array was not able to process and transmit any data from the array's sensors.

Agency Report (AR) at 19.

At the end of the calibration procedure in Florida, both prototypes were moved to LakePend Oreille in Idaho. From April 10 through 14, the Navy installed, checked, and towed Chesapeake's array from the Chinook, recording the array's data as provided in the test protocol. From April 17 through 19, the Navy installed, checked, and towed Lockheed's array. The towing process for Lockheed took less time, because, as the Navy knew, the Lockheed array had been unable to transmit data since the failure that had occurred during the calibration process at the Navy's Leesburg facility, and thus data handling steps were unnecessary. Neither Lockheed nor Chesapeake participated in the installation or check-out of its array.[7]

On May 8, both offerors submitted proposals in response to the RFP. The Navy opened discussions on August 28. While the discussions dealt with technical evaluation issues, the Navy did not inform Lockheed that its array had not functioned during towing. Next, the Navy received and evaluated revised written proposals from both offerors. At this point, the technical evaluators produced an extensive narrative of strengths, weaknesses, and risks associated with each proposal, as well as a numerical score for each subfactor (to which the evaluators would later apply weighting factors, reflecting the significance of each factor and subfactor described in the RFP). The numerical scores assigned to each subfactor are set forth below:

As these unweighted scores illustrate, the evaluators concluded that the lack of towing data resulted in a score of zero on Lockheed's rating under the acoustic self-noise subfactor. However, the technical panel report explained that in other respects –the array was evaluated based on design data— that had been delivered under the prototype contract. AR, Tab 8, Technical Evaluation Review Panel Report (Aug. 10, 2006), at 2. After weighting the subfactors, the failure of Lockheed's array had a fairly minor impact on the point scores reflected in the technical panel report, and the panel concluded that the proposals were technically equivalent. The weighted scores after this evaluation are set forth below:

AR, Tab 9, Contract Award Review Panel Report Addendum, Oct. 12, 2006, at 4.

The source selection authority cited the reports of the technical and price evaluators, noted the identical total weighted scores under the non-price factors, and selected Lockheed because its evaluated cost was 23% lower than Chesapeake's. AR, Tab 10, Initial Source Selection Memorandum, Nov. 7, 2006, at 1. On November 22, the Navy announced the award of the contract to Lockheed. Chesapeake promptly requested a post-award debriefing.

Several days before the scheduled date for the debriefing, a corporate executive at Chesapeake received a telephone call that is relevant to this dispute. According to the executive, the caller (who Chesapeake asserts is not a government employee, but who is not otherwise identified in the record) used words to the effect of: –Sorry you lost, but at least yours worked.—[8] Chesapeake understood the caller to be saying that Lockheed's array had failed to function during towed testing. On December 1, Chesapeake received its debriefing, where it inquired of the Navy whether the information Chesapeake had received was true. According to Chesapeake, the Navy personnel expressed discomfort at the inquiries, and responded only that –[i]t depends...— Chesapeake Protest, exh. G, Declaration of Chesapeake Executive, at 2.

On December 6, Chesapeake filed a protest with our Office, challenging the award of the production contract to Lockheed, and alleging that the Navy could not reasonably have selected Lockheed over Chesapeake, given that Lockheed's array had failed to operate during the tow-off testing.[9] Chesapeake also argued that the Navy had used improper or unstated evaluation criteria. Before the due date for an agency report in that protest, the Navy elected to reopen the procurement to conduct discussions, seek revised proposals, and make a new best value determination. Our Office dismissed Chesapeake's protest on the basis of that corrective action. Chesapeake Scis. Corp., B-299234, Dec. 20, 2006.

After that earlier protest had been dismissed, and as a part of reopening the procurement, the Navy provided Lockheed with a debriefing on January 4, 2007. See Federal Acquisition Regulation (FAR) sect. 15.507(b)(2). Lockheed states that it was during this debriefing that it learned that Chesapeake had been told of the failure of Lockheed's array via an unauthorized disclosure, rather than the debriefing process. Protest at 27.

By letter dated January 17, Lockheed advised the Navy that the firm considered the disclosure to Chesapeake that Lockheed's array had failed during testing to be a violation of procurement integrity provisions. By that time, however, the contracting officer (CO) had, on his own, concluded that the disclosure was a violation of procurement integrity provisions, and had already memorialized his conclusion in writing. In this document, the CO concluded that the disclosure had not affected the competition, and that the recompetition should go forward. Agency Report, Tab 13, Memorandum from the CO, Dec. 20, 2006, at 4. Shortly after Lockheed submitted its allegation of a procurement integrity violation, the NAVSEA executive director of contracts reviewed the CO's conclusions and authorized the recompetition to go forward on the basis that the disclosure –did not prejudice or impact the Source Selection Process.— Agency Report, Tab 14, Memorandum from Executive Director, Jan. 10, 2007, at 1. The CO informed Lockheed on January 25 that –any procurement integrity act violation that might have occurred . . . in no way precludes a fair and equitable competition.— AR, Tab 16, Letter from CO to Lockheed, Jan. 25, 2007, at 1.

The Navy then opened discussions by providing each offeror with its section of the technical panel report, which contained the numerical ratings and the narrative of evaluated strengths, and weaknesses, for all subfactors. The Navy also requested that both offerors prepare final proposal revisions. See generally AR, Tab 17, Discussions Letters to Lockheed and Chesapeake. As part of its reevaluation, the Navy decided that it would limit its consideration of data analysis or modeling, thus making the results of towed testing significantly more important to the evaluation.

Upon receipt of final proposal revisions, the Navy evaluated the proposals as follows:

The record reflects that the decreases in Lockheed's technical scores were largely driven by the revision in the evaluation scheme, under which the Navy looked to data from the tow-off testing in order for the offerors to receive full credit under several criteria. As a result, the weighted scoring was significantly affected by the failure of Lockheed's array to produce any data during the towing testing. The final weighted scores and evaluated prices are set forth below:

After noting the technical panel report had found significant differences between the offerors as reflected in their numerical scores, the source selection authority reversed the initial decision selecting Lockheed, and instead selected Chesapeake for award. The source selection authority identified the key contributors to his decision as the higher overall and subfactor ratings achieved by Chesapeake, the higher tactical and wideband frequency bandwidth of Chesapeake's array, and the low ratings of Lockheed under the passive localization and ranging, wideband capability, shallow water capability, and acoustic self-noise criteria. The source selection authority concluded that these differences were significant enough to justify selecting Chesapeake at its higher evaluated cost. AR, Tab 21, Final Source Selection Decision, Apr. 20, 2007, at 2.

DISCUSSION

Lockheed challenges the handling of the procurement integrity violation, the testing of its prototype and the evaluation of proposals. With respect to procurement integrity, Lockheed argues that the Navy failed to properly investigate and consider the impact of the violation in determining what corrective action to take, and consequently allowed Chesapeake to benefit from the violation. With respect to the conduct of the procurement, Lockheed argues that the Navy failed to follow the RFP ground rules that provided the vendors an opportunity to be present at the pierside installation--and if the need arose, to repair the array--before towing testing at Lake Pend Oreille. Finally, Lockheed also argues that in the course of corrective action, the Navy conducted misleading discussions, changed the evaluation to increase Chesapeake's score while decreasing Lockheed's, and relied excessively on those point scores in selecting Chesapeake for award.

The Navy responds that notwithstanding the procurement integrity violation, it properly determined to continue with the corrective action, and that Lockheed's protest on this basis is untimely. The Navy also argues that the offerors were already provided an opportunity to repair their arrays (in mid-March, before the RFP was issued), and there is no requirement that they be given a second repair opportunity. In addition, the Navy contends that Lockheed can only speculate that its repairs would have been successful. Finally, the Navy argues that discussions were adequate, the final evaluation was fair and consistent with the RFP, and the source selection decision properly considered the differences between the offerors which were signified by the point scores.

Procurement Integrity Issues

In its initial protest, Lockheed argues that the agency did not properly address the procurement integrity violation Lockheed alleged on January 17. In its supplemental protest, upon reviewing the Navy's documents regarding this matter, Lockheed alleges that the Navy's consideration of the violation did not meet the procedural requirements of FAR sect. 3.104-7. In particular, Lockheed argues that the Navy only considered the impact of the violation on the initial source selection, but did not consider any impact on the reopened competition. In answer, the agency and intervenor argue that Lockheed did not timely raise its procurement integrity allegation with the agency within 14 days of learning of it, as required by section 21.5(d) of our Bid Protest Regulations. 4 C.F.R. sect. 21.5(d) (2007). The Navy also argues that its consideration of the matter was reasonable, and consistent with the applicable regulations.

Lockheed's response to the agency and intervenor's timeliness arguments is that it did not learn that the disclosure to Chesapeake (about the operation of Lockheed's device during testing) occurred outside the debriefing process until the firm received its own debriefing on January 4. Lockheed also explains that it was only during its debriefing that it learned that the Navy considered the information to be source selection information. In Lockheed's view, it had no reason to suspect a procurement integrity violation until it learned that the Navy considered the test failure to be source selection information. Letter from Lockheed Counsel to GAO, July 13, 2007, at 2-3.

We will not conclude, on this record, that Lockheed failed to timely allege a procurement integrity violation. On the other hand, we think Lockheed's response to the timeliness arguments suggests that it saw no harm to the procurement process as a result of this disclosure. The record here shows that Lockheed learned that Chesapeake had been advised that Lockheed's array did not work during testing when it intervened in Chesapeake's protest to our Office late in 2006. There is no contemporaneous evidence suggesting that the content of this information was viewed by Lockheed as competitively harmful. In addition, it was not the content of the disclosure that led to the allegation that the procurement had been compromised, but the later-learned knowledge that the Navy was not the source of the disclosure. In fact, Lockheed's apparent lack of concern upon originally learning of the disclosure suggests that it believed that the information could have been provided to Chesapeake during that firm's debriefing. Moreover, the Navy has argued that even though it did not disclose this information to Chesapeake, it would have been permissible to do so during Chesapeake's debriefing. Supp.AR at 10.

The record shows that the CO concluded that the disclosure here constituted a procurement integrity violation, and that he took steps to memorialize this conclusion and his views of its impact on the procurement. In addition, the CO prepared his memorandum almost a month before Lockheed filed its allegation. The record also shows that both the CO and the NAVSEA executive director of contracts were aware of both the content of the disclosure and the fact that the competition was being reopened when they considered the impact of the violation. We think that the CO's handling of this matter, and his decision to proceed with the reopened competition despite the earlier disclosure, were reasonable. We also find reasonable the actions of NAVSEA's executive director of contracts when he confirmed the CO's judgments and memorialized his own consideration of the matter.[10]

Deviation from Evaluation Ground Rules

With respect to the testing of the array prototypes, Lockheed argues that it was improperly denied the opportunity specified in the RFP to repair its array before phase 2 of the towed testing. The Navy responds that Lockheed had already been provided an opportunity to repair its array before the RFP was issued, and the RFP had warned offerors that –the government may not complete all the tests listed— and thus the Navy had the discretion to omit phase 1,[11] and even if the firms had been given a repair opportunity, Lockheed might not have been able to successfully repair its array.

We have held that where an agency provides in a solicitation for the evaluation of a product sample--a situation analogous to the prototypes being tested here--that evaluation must be fair, reasonable, and consistent with the evaluation criteria. Design Contempo, Inc., B'252589.2, Aug. 11, 1993, 93-2 CPD para. 90 at 4. This includes an obligation to perform product tests as described in a solicitation. Mine Safety Appliances Co., B'238597.2, July 5, 1990, 90'2 CPD para. 11 at 11. Thus, while our Office will not make an independent determination of the merits of an offeror's proposal or, in the case of a product demonstration, the performance of the offeror's product, we will review the evaluation record, including the results of any test demonstration, to ensure that the agency's technical judgment has a rational basis and is consistent with the stated evaluation criteria. Exploration Prods., B-279251.2, B-279251.3, June 1, 1998, 98'2 CPD para. 15 at 5.

The RFP provisions quoted above anticipated the possible failure of this equipment before the beginning of phase 2 towed testing, and provided certain repair rights to each vendor in such a situation. Specifically, the RFP provided that in phase 1 the offeror would be given a single repair opportunity of up to 7 days in which to return a failed prototype array to a working state. Lockheed was denied this opportunity the RFP –ground rules— had promised.

Additionally, given the specific language of the RFP here, we think it is not reasonable for the Navy to argue that it was allowed to omit the phase 1 repair opportunity--particularly where that omission brought about exactly the type of harm that the RFP seemed to be designed to avoid. The Navy emphasizes that phase 1 was also intended, in part, to allow the offeror to train Navy personnel in the proper handling and operation of the array, and we accept the Navy's argument that its personnel needed no further training. Nevertheless, the phase 1 testing plainly promised a repair opportunity, apparently in an effort to ensure that the prototype arrays would be operational at the start of phase 2 towed testing.

The Navy points out that both offerors repaired their arrays prior to issuance of the RFP, and argues that Lockheed was not entitled to a second repair opportunity. In our view, however, the earlier repair opportunity cannot be relied upon by the Navy to meet a promise it had not yet made. In any event, the mid-March repairs (made before the prototypes were shipped from Florida to Idaho) cannot reasonably be seen as a substitute for the RFP-provided right to restore a failed array to an operational status at the most crucial time under the RFP, that is, pierside at LakePend Oreille, immediately before the beginning of the towing testing.

We also are not persuaded by the Navy's speculative assertions that Lockheed might have been unable to repair its array, if it had been offered that opportunity. On this subject, Lockheed answers with its own speculation that it could have repaired its array quickly and easily using spares that it had on hand. Our process is ill-suited to deciding protests on the basis of speculation--by either party. In our view, we need not determine that Lockheed would have successfully repaired its array; we need only conclude that it was denied the opportunity to do so immediately before phase 2 testing, and that this was contrary to the ground rules specified in the RFP.[12]

RECOMMENDATION

We sustain the protest because we conclude that the Navy did not conduct the testing of Lockheed's prototype in accordance with the process set forth in the RFP, and this failure prejudiced Lockheed in this competition. We recommend that the Navy either follow the ground rules in its RFP by allowing Lockheed an opportunity to repair its array immediately before towed testing, or revise the evaluation ground rules by an amendment to the solicitation and allow the offerors an opportunity to compete under the revised RFP. In either case, the Navy should prepare a new evaluation and source selection decision. If this process results in selection of Lockheed, the Navy should terminate the contract with Chesapeake. Finally, we recommend that the protester be reimbursed its costs of filing and pursuing the protest, including reasonable attorneys' fees. Bid Protest Regulations, 4 C.F.R. sect. 21.8(d)(1). The protester should submit its certified claim, detailing the time expended and costs incurred, directly to the contracting agency within 60 days of receiving this decision. 4 C.F.R. sect. 21.8(f)(1).

The protest is sustained.

Gary L. Kepplinger

General Counsel