B-299766; B-299766.2, Lockheed Martin Maritime Systems & Sensors, August 10, 2007
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.
Protest is sustained where solicitation provided for tow-off testing of prototype arrays according to a procedure in which each offeror was explicitly permitted to participate in installation and check-out of its prototype, and then granted a single opportunity to repair any failure of its prototype immediately prior to towed testing, but the agency did not notify the protester that its prototype had already failed before arrival at the testing site, and did not allow the protester to repair its prototype, and as a result, the agency was unable to evaluate significant aspects of the performance of the protester’s array.
Lockheed Martin Maritime Systems & Sensors protests the award of a contract to Chesapeake Sciences Corporation by the Department of the Navy, Naval Sea Systems Command (NAVSEA), under request for proposals (RFP) No. N00024‑05‑R‑6259, issued as a follow-on to development contracts awarded under a broad agency announcement to procure production towed-arrays for the TB-34 Next Generation Fat-Line Towed Array program. Lockheed argues that the Navy failed to properly remedy a breach of procurement integrity, did not follow the procedure for testing and evaluating prototype arrays specified in the RFP, and made a defective source selection decision.
Navy conducted this competition for the production of towed arrays
between the two contractors that had delivered prototypes of their own design
under separate contracts. Thus, the
procurement process began in September 2004 when, after competition among four
firms under a broad agency announcement, the Navy awarded separate contracts to
Navy experienced problems with both prototypes in mid-March 2006, and returned
each to the respective manufacturer for repairs from March 13 to
The RFP described the non-price evaluation in terms of three factors: technical performance, contractor statement of work, and past performance. RFP at 178. Of these three, the RFP specified that the technical performance factor would be significantly more important than the contractor statement of work and past performance factors combined. In addition, the non-price factors were significantly more important than cost/price. The RFP also made clear that the operation of the prototypes would be a significant aspect of the technical performance evaluation. RFP at 170-71.
RFP described several stages of the evaluation of the protoypes, including
acoustic calibration, tow-off testing, and anechoic chamber testing. These were described in a separate section of
the RFP labeled “Evaluation Plan for the Next-Generation Fat-Line Towed Array
Testing.” RFP, Evaluation Plan, at
1. Initial acoustic calibration took
place at the
The array and tow cable being tested will be installed aboard the Chinook winch and integrated with the shipboard laboratory equipment by vendor personnel using appropriate procedures. The Navy personnel, and vendor personnel if available, will power the array and verify proper operation in accordance with vendor guidelines. After proper operation is established, Navy personnel and/or vendor personnel will command the array to turn on each electrical
[ibration] signal in sequence . . . . CAL
The RFP Evaluation Plan also incorporated a document titled “Evaluation Ground Rules,” that described two phases in the tow-off. After installation and pier-side checkout, “phase 1” would include operation of the array with the vendor present, while “phase 2” would be conducted without the vendor present. The ground rules explained the purpose of each phase:
C.2 TOW-OFF EVALUATION RULES
* * * * *
Array vendors are welcome to participate in the phase-1 testing. . . .
In the event of an array failure during phase 1, the array vendor will have no more than 7 days to make repairs to the array. There will not be a second phase-1 test period after a failure. Once the array is repaired, it will be in control of the government and will proceed to phase 2.
The evaluation team will conduct the second phase of the tow-off without vendor participation, but the vendors are welcome to leave their troubleshooting equipment aboard the research vessel during phase 2 for the evaluation team’s use in the event of a failure. Phase 2 is expected to require several days of towing testing. Daily briefings will not be scheduled during phase 2. The evaluation team will contact a vendor if necessary.
RFP, Evaluation Plan, app. C, at C-2 to C-3.
The Evaluation Plan also noted that “[t]his document lists the evaluation testing approach and actual tests that may be performed in the design competition; any of these tests may be performed, but the government may not complete all the tests listed.” RFP, Evaluation Plan, at 2.
on March 27, the Navy began conducting a multi-day process of calibration with
the arrays at the Leesburg facility, starting with
[o]n the late afternoon of
30 March 2006, the evaluation team was approximately three-quarters through the calibration process at Leesburg on the [Lockheed] TB-34 prototype when the processing system stopped receiving data. The next morning, using troubleshooting procedures previously provided by [Lockheed] . . . prior to release of the solicitation, the evaluation team isolated the failure to the [Lockheed] array’s [deleted]. Only the remaining wideband calibration was able to be completed. As a result of the failure in the [deleted], the array was not able to process and transmit any data from the array’s sensors.
Agency Report (AR) at 19.
the end of the calibration procedure in
On May 8, both offerors submitted proposals in response to the RFP. The Navy opened discussions on August 28. While the discussions dealt with technical evaluation issues, the Navy did not inform Lockheed that its array had not functioned during towing. Next, the Navy received and evaluated revised written proposals from both offerors. At this point, the technical evaluators produced an extensive narrative of strengths, weaknesses, and risks associated with each proposal, as well as a numerical score for each subfactor (to which the evaluators would later apply weighting factors, reflecting the significance of each factor and subfactor described in the RFP). The numerical scores assigned to each subfactor are set forth below:
these unweighted scores illustrate, the evaluators concluded that the lack of
towing data resulted in a score of zero on Lockheed’s rating under the acoustic
self-noise subfactor. However, the
technical panel report explained that in other respects “the array was
evaluated based on design data” that had been delivered under the prototype
contract. AR, Tab 8, Technical
Evaluation Review Panel Report (
Tab 9, Contract Award Review Panel Report Addendum,
source selection authority cited the reports of the technical and price
evaluators, noted the identical total weighted scores under the non-price
factors, and selected Lockheed because its evaluated cost was 23% lower than
days before the scheduled date for the debriefing, a corporate executive at
December 6, Chesapeake filed a protest with our Office, challenging the award
of the production contract to Lockheed, and alleging that the Navy could not
reasonably have selected Lockheed over Chesapeake, given that Lockheed’s array
had failed to operate during the tow-off testing.
that earlier protest had been dismissed, and as a part of reopening the
procurement, the Navy provided Lockheed with a debriefing on
letter dated January 17, Lockheed advised the Navy that the firm considered the
Navy then opened discussions by providing each offeror with its section of the
technical panel report, which contained the numerical ratings and the narrative
of evaluated strengths, and weaknesses, for all subfactors. The Navy also requested that both offerors
prepare final proposal revisions. See
generally AR, Tab 17, Discussions Letters to Lockheed and
Upon receipt of final proposal revisions, the Navy evaluated the proposals as follows:
The record reflects that the decreases in Lockheed’s technical scores were largely driven by the revision in the evaluation scheme, under which the Navy looked to data from the tow-off testing in order for the offerors to receive full credit under several criteria. As a result, the weighted scoring was significantly affected by the failure of Lockheed’s array to produce any data during the towing testing. The final weighted scores and evaluated prices are set forth below:
noting the technical panel report had found significant differences between the
offerors as reflected in their numerical scores, the source selection authority
reversed the initial decision selecting Lockheed, and instead selected
challenges the handling of the procurement integrity violation, the testing of
its prototype and the evaluation of proposals.
With respect to procurement integrity, Lockheed argues that the Navy
failed to properly investigate and consider the impact of the violation in
determining what corrective action to take, and consequently allowed
The Navy responds that notwithstanding the procurement integrity violation, it properly determined to continue with the corrective action, and that Lockheed’s protest on this basis is untimely. The Navy also argues that the offerors were already provided an opportunity to repair their arrays (in mid-March, before the RFP was issued), and there is no requirement that they be given a second repair opportunity. In addition, the Navy contends that Lockheed can only speculate that its repairs would have been successful. Finally, the Navy argues that discussions were adequate, the final evaluation was fair and consistent with the RFP, and the source selection decision properly considered the differences between the offerors which were signified by the point scores.
Procurement Integrity Issues
In its initial protest, Lockheed argues that the agency did not properly address the procurement integrity violation Lockheed alleged on January 17. In its supplemental protest, upon reviewing the Navy’s documents regarding this matter, Lockheed alleges that the Navy’s consideration of the violation did not meet the procedural requirements of FAR sect. 3.104-7. In particular, Lockheed argues that the Navy only considered the impact of the violation on the initial source selection, but did not consider any impact on the reopened competition. In answer, the agency and intervenor argue that Lockheed did not timely raise its procurement integrity allegation with the agency within 14 days of learning of it, as required by section 21.5(d) of our Bid Protest Regulations. 4 C.F.R. sect. 21.5(d) (2007). The Navy also argues that its consideration of the matter was reasonable, and consistent with the applicable regulations.
response to the agency and intervenor’s timeliness arguments is that it did not
learn that the disclosure to
We will not conclude, on this
record, that Lockheed failed to timely allege a procurement integrity
violation. On the other hand, we think
Lockheed’s response to the timeliness arguments suggests that it saw no harm to
the procurement process as a result of this disclosure. The record here shows that Lockheed learned
The record shows that the CO concluded that the disclosure here constituted a procurement integrity violation, and that he took steps to memorialize this conclusion and his views of its impact on the procurement. In addition, the CO prepared his memorandum almost a month before Lockheed filed its allegation. The record also shows that both the CO and the NAVSEA executive director of contracts were aware of both the content of the disclosure and the fact that the competition was being reopened when they considered the impact of the violation. We think that the CO’s handling of this matter, and his decision to proceed with the reopened competition despite the earlier disclosure, were reasonable. We also find reasonable the actions of NAVSEA’s executive director of contracts when he confirmed the CO’s judgments and memorialized his own consideration of the matter.
Deviation from Evaluation Ground Rules
With respect to the testing of the array prototypes, Lockheed argues that it was improperly denied the opportunity specified in the RFP to repair its array before phase 2 of the towed testing. The Navy responds that Lockheed had already been provided an opportunity to repair its array before the RFP was issued, and the RFP had warned offerors that “the government may not complete all the tests listed” and thus the Navy had the discretion to omit phase 1, and even if the firms had been given a repair opportunity, Lockheed might not have been able to successfully repair its array.
have held that where an agency provides in a solicitation for the evaluation of
a product sample--a situation analogous to the prototypes being tested
here--that evaluation must be fair, reasonable, and consistent with the
evaluation criteria. Design Contempo,
The RFP provisions quoted above anticipated the possible failure of this equipment before the beginning of phase 2 towed testing, and provided certain repair rights to each vendor in such a situation. Specifically, the RFP provided that in phase 1 the offeror would be given a single repair opportunity of up to 7 days in which to return a failed prototype array to a working state. Lockheed was denied this opportunity the RFP “ground rules” had promised.
Additionally, given the specific language of the RFP here, we think it is not reasonable for the Navy to argue that it was allowed to omit the phase 1 repair opportunity--particularly where that omission brought about exactly the type of harm that the RFP seemed to be designed to avoid. The Navy emphasizes that phase 1 was also intended, in part, to allow the offeror to train Navy personnel in the proper handling and operation of the array, and we accept the Navy’s argument that its personnel needed no further training. Nevertheless, the phase 1 testing plainly promised a repair opportunity, apparently in an effort to ensure that the prototype arrays would be operational at the start of phase 2 towed testing.
Navy points out that both offerors repaired their arrays prior to issuance of
the RFP, and argues that Lockheed was not entitled to a second repair
opportunity. In our view, however, the
earlier repair opportunity cannot be relied upon by the Navy to meet a promise
it had not yet made. In any event, the
mid-March repairs (made before the prototypes were shipped from
We also are not persuaded by the Navy’s speculative assertions that Lockheed might have been unable to repair its array, if it had been offered that opportunity. On this subject, Lockheed answers with its own speculation that it could have repaired its array quickly and easily using spares that it had on hand. Our process is ill-suited to deciding protests on the basis of speculation--by either party. In our view, we need not determine that Lockheed would have successfully repaired its array; we need only conclude that it was denied the opportunity to do so immediately before phase 2 testing, and that this was contrary to the ground rules specified in the RFP.
sustain the protest because we conclude that the Navy did not conduct the
testing of Lockheed’s prototype in accordance with the process set forth in the
RFP, and this failure prejudiced Lockheed in this competition. We recommend that the Navy either follow the
ground rules in its RFP by allowing Lockheed an opportunity to repair its array
immediately before towed testing, or revise the evaluation ground rules by an
amendment to the solicitation and allow the offerors an opportunity to compete
under the revised RFP. In either case,
the Navy should prepare a new evaluation and source selection decision. If this process results in selection of
Lockheed, the Navy should terminate the contract with
The protest is sustained.
 In simple terms, a towed array is a sophisticated listening/sensing device, which is intended to be deployed behind a submarine by a towing cable, and which produces data to assist in locating and identifying other vessels and underwater features, particularly near shore.
 After repairing its array, Lockheed sent a letter to the Navy advising that its array had failed because of the improper application of electrical power, and offering to assist the Navy in testing the array.
 The subfactors under the technical performance factor, in descending order of importance, were: handler and processing capability, array reliability, life-cycle cost, array performance (further subdivided into shallow water capability, passive localization and ranging, wideband sensor, and acoustic self-noise), and suitability for production. The subfactors under the contractor statement of work factor were: approach/oral presentation, firm fixed-price data deliverables, and cost-plus-fixed‑fee data deliverables. There were no subfactors under the past performance factor. RFP at 178.
 In contrast to the government-only testing that would follow, the evaluation plan provided that “[t]he array vendors are welcome to participate in the installation, pierside checkout, and initial (phase 1) portions of the towing tests . . .” RFP, Evaluation Plan, app. A, at A-3.
 For example, while the RFP listed a number of performance measurement tests, from calibration and a tow-off using the Chinook, to “at-sea testing aboard an attack submarine,” the record does not indicate that any submarine-based at-sea testing was performed as part of the evaluation. RFP, Evaluation Plan, at 1.
 As mentioned above, the first failure of the device had led to its return to Lockheed for repairs in mid-March, which Lockheed completed shortly before the issuance of the RFP on March 23.
On April 18, while in the midst of towing
Lockheed’s inoperative array, the Navy sent a letter to Lockheed advising it
that the Navy did not expect to need any assistance with towing testing. Neither the fact that towing testing was
already underway at
The record contains two separate declarations from
Lockheed intervened and was provided a redacted copy of the protest on
December 12 or 13, 2006, which contained the protest allegation that
“Chesapeake, on good information, believes that the Lockheed prototype failed
to function, and hence, the Navy was unable to collect data from it during the
To the extent that Lockheed argues that a sentence in
While the Navy also argues that the failure of Lockheed’s array occurred during
phase 2, we believe the Navy’s argument in this respect relies on a strained
reading of the RFP. Read as a whole, we
agree with Lockheed that the RFP contemplated that phases 1 and 2 of the
evaluation would occur at the location of the towed testing, which was
 As noted above, Lockheed also argues that the evaluation was improper, discussions were misleading, and the source selection decision was superficial. In light of our recommendation that will result in, among other things, a new source selection decision, we find it unnecessary to address these grounds of protest.