DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Sikorsky Aircraft Company; Lockheed Martin Systems Integration‑Owego

File: B-299145; B-299145.2; B-299145.3

Date: February 26, 2007

DIGEST

Protest is sustained where (1) solicitation for combat search and rescue aircraft provided that cost/price would be calculated on the basis of Most Probable Life Cycle Cost, including both contract and operations and support (O&S) costs, (2) solicitation requested detailed information quantifying required maintenance for proposed aircraft, and (3) agency nevertheless normalized cost of maintenance when calculating O&S costs, thereby ignoring potentially lower cost of asserted low maintenance helicopters; once offerors are informed of criteria against which proposals will be evaluated and award made, agency must adhere to those criteria.

Sikorsky Aircraft Company and Lockheed Martin Systems Integration‑Owego (LMSI) protest the Department of the Air Force’s award of a contract to The Boeing Company, under request for proposals (RFP) No. FA8629-06-R-2350, for the Combat Search and Rescue Replacement Vehicle (CSAR-X). Sikorsky and LMSI challenge the evaluation of proposals and resulting source selection.

We sustain the protests.

The HH-60 helicopter currently serves as the agency’s combat search and rescue aircraft, used to recover downed aircrew and isolated personnel. However, as indicated in the contemporaneous record, and confirmed at the hearing our Office conducted in this matter, the Air Force has determined that its aging fleet of HH-60s has several shortfalls, including: insufficient combat radius; a cabin and payload that are too small; inadequate survivability, including inadequate defensive systems and armament; inadequate adverse weather capability; inadequate battlespace awareness; and, as an aircraft nearing the end of its service life, difficulties in maintaining the aircraft and inadequate availability. Source Selection Decision (SSD) at 2; RFP sect. L 1.1; CSAR-X Final Evaluation Brief, Oct. 21, 2006, at 11; Hearing Transcript (Tr.) at 15-16, 208-09, 240-49. The solicitation, issued on October 5, 2005, provided for the acquisition of a non-developmental item, medium lift, vertical take‑off and landing aircraft that, with minimal development, could achieve a combat ready CSAR-X Block 0 aircraft Initial Operational Capability (IOC) by Fiscal Year (FY) 2012 and replace the HH‑60 aircraft. The contemplated effort included the development and demonstration (on a cost-plus-award-fee-with-incentive-fee basis) of the initial Block 0 aircraft, fixed-price production lots for a total of 141 aircraft, associated trainers, spares and support, and the cost-plus-award-fee development and demonstration of the ultimate, fully capable CSAR X Block 10 aircraft.

Award was to be made to the offeror whose proposal represented the “best value” based on consideration of four evaluation factors: (1) mission capability, including (in descending order of importance) subfactors for Block 0 aircraft performance, system architecture and software, systems engineering, product support, management/schedule and Block 10 aircraft performance; (2) proposal risk, assessed at the subfactor level, and focusing on weaknesses associated with an offeror’s proposed approach and the resulting potential for disruption of schedule, increased cost, degradation of performance, the need for increased government oversight, and the likelihood of unsuccessful contract performance; (3) past performance; and (4) cost/price. Of particular importance here, the solicitation provided that, for evaluation purposes, cost/price would be calculated on the basis of the Most Probable Life Cycle Cost (MPLCC) for the aircraft, comprised of the cost/price of: (1) system development and demonstration (SDD); (2) the award fee; (3) production; (4) operations and support (O&S); (5) other government costs; and (6) proposal risk adjustments, based on a quantification in terms of schedule and cost of the impact of the technical, programmatic and schedule risk of the offeror’s proposed approach. Mission capability, proposal risk and past performance were of equal importance and, when combined, were “significantly more important than” cost/price. RFP sect. M 8.

Boeing, Sikorsky and LMSI submitted initial proposals by the November 5, 2005 closing time. Boeing proposed the twin-rotor HH‑47 helicopter, an updated version of the MH-47G helicopter, part of a family of helicopters that have been in production and flying for over 40 years. LMSI proposed the single-rotor US101 helicopter, a version of the EH-101, military versions of which are in military service with the United Kingdom. Sikorsky proposed the single-rotor S-92 helicopter, currently in commercial use to ferry personnel to and from off-shore oil platforms and used by the United Kingdom, Canada and several other governments for maritime search and rescue.

All three proposals were included in the competitive range. After conducting several rounds of written and oral discussions--including several face-to-face “debriefings” during which offerors were advised of the agency’s current evaluation of their technical proposals and past performance--and after a flight evaluation to assess the current capability of the offerors’ baseline aircraft, the Air Force requested final proposal revisions (FPR) by September 18, 2006. The Air Force evaluated the FPRs as follows:

	LMSI	Sikorsky	Boeing
Mission Capability/ Proposal Risk
Block 0 Performance	Acceptable/High	Acceptable/Low	Exceptional/Low
Architecture/Software	Acceptable/ Moderate	Acceptable/Moderate	Acceptable/Moderate
Systems Engineering	Acceptable/Moderate	Acceptable/Moderate	Acceptable/Moderate
Product Support	Exceptional/Low	Exceptional/Low	Acceptable/Low
Management/Schedule	Acceptable/Moderate	Exceptional/Moderate	Exceptional/Moderate
Block 10 Performance	Acceptable/Moderate	Acceptable/Moderate	Exceptional/Moderate
Past Performance	Little Confidence	Satisfactory Confidence	Satisfactory Confidence
Price/Cost	($ million)	($ million)	($ million)
SDD Proposed	[REDACTED]	[REDACTED]	[REDACTED]
SDD Adjustment	[REDACTED]	[REDACTED]	[REDACTED]
Production Proposed	[REDACTED]	[REDACTED]	[REDACTED]
Production Adjusted	[REDACTED]	[REDACTED]	[REDACTED]
OGC	[REDACTED]	[REDACTED]	[REDACTED]
Subtotal	[REDACTED]	[REDACTED]	[REDACTED]
O&S	[REDACTED]	[REDACTED]	[REDACTED]
MPLCC	$35,853.7	$38,531.8	$38,891.5

Based on the evaluation results, the source selection authority (SSA) determined that Boeing’s proposal represented the best value. As an initial matter, according to the SSD, because cost/price “was so close between the offerors” and was significantly less important than the other evaluation factors combined, “Cost/Price ratings did not provide a great deal of weight” in the SSA’s source selection decision; rather, “Mission Capability/Proposal Risk Subfactors and Past Performance became the basis for discriminating between the offerors.” SSD at 6. In this regard, the SSA noted that LMSI’s proposal had received a high risk rating under the Block 0 performance subfactor due to [REDACTED]. According to the SSA, since “[REDACTED],” he “saw no reason to take on that HIGH level of risk.” Id. In addition, the SSA noted that LMSI had received a little confidence rating for past performance due to unsatisfactory performance under its current contract for the VH-71 Presidential helicopter, which was evaluated as highly relevant to this procurement. According to the SSA, LMSI had “show[n under that contract] that it could not reliably meet important schedule requirements and had difficulty in systems engineering flow-downs to their subcontractors.” Id. at 7. The SSA, noting the “stronger proposals offered by Boeing and Sikorsky,” concluded that the “modest” MPLCC savings associated with LMSI’s proposal were outweighed by LMSI’s high risk rating under the Block 0 performance subfactor and little confidence past performance rating. Id. at 6-7.

The SSA also determined that Boeing’s proposal was more advantageous than Sikorsky’s lower cost proposal. In this regard, while acknowledging Sikorsky’s advantage under the product support subfactor, where it was rated exceptional while Boeing’s was rated only acceptable, the SSA emphasized that Boeing’s was the only proposal that received an excellent rating under the Block 0 performance subfactor, having received significantly more strengths (including strengths for mission load and occupant protection) than Sikorsky’s proposal. The SSA also noted Boeing’s advantage under the Block 10 performance subfactor (the least important subfactor), where Boeing’s proposal was rated exceptional and Sikorsky’s only acceptable. The SSA concluded that Boeing’s greater mission capability strengths outweighed Sikorsky’s “slightly lower” MPLCC. Id. at 10.

Upon learning of the resulting award to Boeing (on November 9, 2006), and after being debriefed, Sikorsky and LMSI filed these protests with our Office. In their protests, Sikorsky and LMSI challenge the agency’s evaluation of proposals.

In reviewing protests against allegedly improper evaluations, it is not our role to reevaluate proposals. Rather, our Office examines the record to determine whether the agency’s judgment was reasonable and in accord with the RFP criteria and applicable procurement statutes and regulations. See Rolf Jensen & Assocs., Inc., B‑289475.2, B-289475.3, July 1, 2002, 2002 CPD para. 110 at 5. In this regard, an agency may not announce in the solicitation that it will use one evaluation plan and then follow another; once offerors are informed of the criteria against which their proposals will be evaluated and the source selection decision made, the agency must adhere to those criteria or inform all offerors of significant changes. Remington Arms Co., Inc., B‑297374, B‑297374.2, Jan. 12, 2006, 2006 CPD para. 32 at 15. Here, based upon our review of the record, including a hearing conducted by our Office, we find that the Air Force’s evaluation of O&S costs was inconsistent with the RFP.

As noted above, the solicitation provided that, for evaluation purposes, cost/price would be calculated on the basis of the MPLCC for the CSAR-X aircraft, including (among other costs) O&S costs. In this regard, the solicitation included RFP attachment 13, “O&S Data Form,” the “primary purpose” of which was to “capture all relevant CSAR-X Operating and Support (O&S) costs”; offerors were required to complete the attachment with estimates of specified O&S costs for their proposed aircraft through FY 2029, when the last of the new CSAR-X aircraft were to become fully operational. The agency then extended all O&S costs from FY 2029 to generate a complete life cycle for each aircraft, resulting in an overall CSAR-X life cycle through FY 2039. Contracting Officer’s Statement (Sikorsky) at 22; CSAR-X O&S Cost FPR Evaluation at 3.

Attachment 13 listed the following O&S cost categories: (1) Unit Mission Personnel, including active duty, Air National Guard and Air Force reserve personnel; (2) Unit Level Consumption, including Fuel, Consumables, Repair Costs (Repair Material, Repair Labor and Replenishment Spares), and Training Munitions; (3) Depot Maintenance; (4) Contractor Support, including Training System and Other Contractor Costs (Material and Labor); (5) Sustaining Support, including Support Equipment, Sustaining Engineering and Software Maintenance; and (6) Indirect Support, including active duty, Air National Guard and Air Force reserve personnel. Offerors were required to estimate the O&S costs of their proposed aircraft in these categories through FY 2029, with the exception that attachment 13 stated as follows: “Unit Mission Personnel, Training Munitions, and Indirect Support will be calculated by the government team. Leave blank.”

In addition, offerors were required to comply with, and furnish “sufficient detail for effective evaluation and for substantiating the validity of stated claims” with respect to, various requirements in the CSAR-X System Requirements Document (SRD) quantifying the amount of maintenance the proposed aircraft would need during their service life. RFP sect. L 2.0. For example, offerors were required to submit the maintenance manhours per flight hour (MMH/FH) their aircraft required; the CSAR-X SRD established as a threshold, or minimum, requirement that the CSAR-X aircraft not require more than 20 MMH/FH and, as an objective, that it not require more than 10 MMH/FH. SRD sect. 3.1.4.2.4. Other examples include requirements for offerors to specify: (1) mean time to repair their aircraft, with the SRD specifying a threshold requirement of no more than 1.5 hours and an objective of 0.5 hours, SRD sect. 3.1.4.2.1; (2) time required for Mid Term Scheduled Inspections, with a threshold requirement of 20 manhours and an objective of 10 manhours, SRD sect. 3.1.5.3.2; (3) time to repair a failed system after a sortie, with a threshold requirement of completion within 12 hours 80 percent of the time and an objective of completion within 12 hours 90 percent of the time, SRD sect. 3.1.4.2.1; and (4) time to complete any required long‑term scheduled inspections and repair, with a threshold requirement of completion using no more than 960 manhours and an objective of completion using no more than 480 manhours, SRD sect. 3.1.5.3.3.

The Air Force calculated the O&S cost of Boeing’s proposal as $[REDACTED] billion in “Then Year” (TY) (inflated) dollars, Sikorsky’s as $[REDACTED] billion, and LMSI’s as $[REDACTED] billion. In calculating these O&S cost estimates, the Air Force used the same costs for Unit Mission Personnel, Training Munitions, MER Contractors and Indirect Support for each offeror. The government-generated personnel-based costs for Unit Mission Personnel ($9.611 billion) and Indirect Support ($3.609 billion) accounted for most of the calculated O&S cost totals for each offeror--approximately [REDACTED] and [REDACTED] percent, respectively, of LMSI’s evaluated O&S cost; [REDACTED] and [REDACTED] percent of Sikorsky’s O&S cost, and [REDACTED] and [REDACTED] percent of Boeing’s O&S cost.[1]

Sikorsky and LMSI assert that the agency’s calculation of O&S costs using the same estimated cost for Unit Mission Personnel and Indirect Support for all proposals, irrespective of the aircraft offered, unreasonably failed to account for the reduced maintenance required by their generally newer design, smaller helicopters. They claim that this was inconsistent with the RFP’s requirement for detailed information regarding O&S costs generally, and maintenance costs specifically. The protesters read this information requirement, together with the MPLCC evaluation approach, as indicating that the O&S cost evaluation would take into account the helicopter offered.

We agree with the protesters. It is true that the solicitation provided that “Unit Mission Personnel . . . and Indirect Support will be calculated by the government team,” Attach. 13, and similar language in a different context could mean that the agency would use normalized, i.e.,“plug,” numbers for all offerors. Here, however, the RFP also stated the agency’s intention to “capture all relevant CSAR-X Operating and Support (O&S) costs,” id., and requested the submission of various measures and information intended to quantify the amount of maintenance the proposed aircraft would need during their service life. The solicitation further indicated through its inclusion in the SRD of threshold requirements and objectives with respect to various maintenance measures (such as MMH/FH), that it valued aircraft that require less maintenance. In the face of these solicitation provisions, we think the statement that the government would calculate the Unit Mission Personnel and Indirect Support costs reasonably could only be interpreted as indicating that the agency would perform those calculations taking into account the unique aspects of the proposed aircraft (including maintenance requirements) as identified in the required information. Only in this way would the agency’s evaluation “capture all relevant CSAR-X Operating and Support (O&S) costs,” as it announced was its intent.

We further agree that the agency’s methodology for calculating O&S costs did not reasonably account for each offeror’s unique technical approach, including very different proposed helicopters, and thus was inconsistent with the stated evaluation scheme. In this regard, the Air Force based the evaluated cost for Unit Mission Personnel‑‑that is, the primarily base-level active duty, Air National Guard and Air Force reserve personnel in the contemplated 13 CSAR-X squadrons‑‑and for MER Contractors on the staffing set forth in the Air Force’s MER for the Replacement Personal Recovery Vehicle (CSAX), March 30, 2006 version, issued by the Air Force Special Operations Command (the warfighting command previously responsible for the combat search and rescue mission). The MER specifies that the maximum total staffing requirement “under the current concept of operations,” including both government and contractor personnel, will be 6,232 manyears (or 5,539 manyears excluding contractors and certain technicians), more than half of which were listed under the maintenance function. MER at 2, 16; Tr. at 37-42, 378-81, 1181-82; Agency Comments, Feb. 3, 2007, at 17-18. In addition, the agency derived the evaluated cost for Indirect Support, which included such indirect costs as training and change of station, based on the stafffing specified in the MER. Tr. at 1133-34. As indicated in the MER itself, and as conceded by agency witnesses in hearing testimony, the MER reflected the staffing and maintenance concepts only for the current HH-60 helicopter. MER at 12‑14; Tr. 38, 377-78, 1182.[2]

The Air Force maintains that it was reasonable to base its O&S cost analysis on the MER staffing because the MER represented the official agency staffing for the contemplated CSAR-X squadrons. However, the fact that the MER may have been considered the official staffing level is irrelevant here. The question is not whether the MER could be used in the agency’s analysis; rather, the question is whether the agency reasonably could rely on the MER to the exclusion of the information provided by offerors regarding the characteristics of their particular aircraft, especially since the MER is based on the current helicopter, not those proposed, and may bear only a limited relationship to the ultimate future maintenance staffing needs for the replacement helicopter. The fact that the MER represented the agency’s current official staffing level does not override what we have found was the evaluation approach reasonably indicated by the RFP--that the maintenance characteristics of the proposed helicopters would be considered in the O&S cost evaluation.

Reliance on the MER is all the more problematic given that the current HH‑60, which is nearing the end of its service life, has “very high” maintenance requirements, requiring “upward around” 25‑30 MMH/FH. Tr. at 681. In contrast, as discussed above, the CSAR-X SRD established as a minimum that the CSAR-X aircraft shall not require more than 20 MMH/FH, and stated as an objective that the aircraft should not require more than 10 MMH/FH. SRD sect. 3.1.4.2.4. Consistent with the SRD, all three offerors estimated significantly lower MMH/FH for their proposed helicopters than is required for the current HH‑60. Boeing’s proposal, generally referring to extensive flying experience for the H-47 family of aircraft during Operation Enduring Freedom and Operation Iraqi Freedom, and specifically relying for part of its calculation upon experience with the MH-47E helicopter, generally referred to an estimated MMH/FH for the CSAR-X of [REDACTED] hours. (We note that the specific calculations supporting this number actually indicated an estimated MMH/FH of [REDACTED] hours.) Boeing Mission Capability Proposal, 1(d) Product Support, sect. 4.2, at 46, 50-51, 62-63. LMSI, noting that its current EH‑101 platforms had accumulated over [REDACTED] flight hours, and had specifically demonstrated an MMH/FH of below [REDACTED] hours during combat operations in Iraq, estimated the MMH/FH for its proposed CSAR-X as [REDACTED] hours. LMSI FPR Executive Summary; LMSI FPR Mission Capability Proposal, 1(d) Product Support, sect. 1.4, at 18‑19, 27-30, 50-52. Sikorsky, noting that its aircraft had achieved sustained [REDACTED] percent availability, flying [REDACTED] hours a day, during commercial offshore oil operations in harsh environments, and citing an “[i]ndependent analysis” by an aircraft consulting firm specializing in calculating aircraft operating costs, estimated the MMH/FH of its proposed CSAR-X aircraft as fewer than [REDACTED] hours. Sikorsky FPR Executive Summary; Sikorsky Mission Capability Proposal, 1(d) Product Support, June 2006, sect. 3.1.1, at 42-43, 52; see Tr. at 651.

Further, it is clear that the MER will change to reflect the characteristics, including unique maintenance requirements, of the selected CSAR-X helicopter. Not only does the MER itself acknowledge that it is a “living document” but, in fact, there already have been three editions of the MER during the course of this procurement, including the March 30, 2006 edition used by the evaluators and two subsequent editions (July 17, 2006 and August 30, 2006) that were not used in the evaluation. MER at 2; Tr. at 1088. Both the SSA and the contracting officer anticipated that the MER would be changed after award to reflect the characteristics and requirements of the selected CSAR-X helicopter. Tr. at 97, 384-85, 388-89, 392-93. Although the SSA testified that the MER would be updated only after the agency has 2-3 years of operational experience with the new CSAR-X aircraft, Tr. at 216-17, 385, we note that the agency’s June 2005 Capability Development Document for Combat Search and Rescue Replacement Vehicle (CSAR-X)/Personnel Recovery Vehicle (PRV), June 16, 2005, prepared prior to the current litigation, stated simply that the “initial Manpower Estimate Report . . . will be updated following source selection.” Capability Development Document at 61. In any case, even accepting the SSA’s chronology, the MER could be changed as early as 2014, that is, 2 years after the scheduled initial operational capability in 2012. Since the Air Force purported to evaluate O&S costs through FY 2039, we see no basis in the record for its evaluation to ignore the most likely maintenance requirements for the new CSAR-X for nearly all of its operational life and to rely instead on a maintenance approach (as reflected in the March 2006 MER) based on the current high maintenance aircraft.

The agency asserts that consideration in the O&S cost evaluation of the offerors’ predicted MMH/FH would have been inappropriate. In this regard, the agency argues that only the offerors’ contractually binding commitments, as set forth in each offeror’s proposed Weapon System Specification (WSS)--which was to be incorporated into the offeror’s contract, RFP sect. L-2--were meaningful, and notes that [REDACTED].

We find the agency’s position unpersuasive. First, again, whether or not the agency believed considering the characteristics of the offerors’ proposed aircraft was inappropriate, the fact is that, as already discussed, the RFP indicated that those characteristics would be considered. Further, we note that the agency’s current reliance on the WSS numbers represents a departure from its approach during the O&S cost evaluation; the WSS numbers simply were not used in the O&S cost evaluation. Testimony from agency witnesses indicates that the offerors’ WSS numbers were irrelevant to the O&S cost calculations, such that if one offeror had included an MMH/FH of 19 hours in its WSS and another had included an MMH/FH of 1 hour, the difference would not have affected the evaluated O&S costs. Tr. at 594-99.[3]

The agency also questions the protesters’ MMH/FH predictions, focusing in particular on [REDACTED] estimate that its proposed aircraft would have an MMH/FH of fewer than [REDACTED] hours. (As noted above, [REDACTED] predicted an MMH/FH of [REDACTED] hours.) Specifically, the agency claims that neither Sikorsky’s nor LMSI’s proposals included sufficient data to support their claimed MMH/FH numbers. Again, the contemporaneous record does not support the agency’s position. While the agency now focuses on the reliability of the offerors’ predictions, and one of the evaluators testified that he questioned during the evaluation whether [REDACTED] could achieve its predicted MMH/FH, it is clear from the hearing testimony that the agency did not calculate a probable MMH/FH for the aircraft, but instead simply determined that each proposed aircraft could achieve the SRD objective. Tr. at 42‑46, 77, 99‑101, 654‑57, 660‑61, 979. Thus, to the extent that the agency now questions Sikorsky’s and LMSI’s predictions that their aircraft would require significantly less maintenance than would Boeing’s aircraft, see Agency Comments, Feb. 3, 2007, at 25 n.2, the agency’s position is based on an analysis performed in the course of the protest. We give little weight to analyses performed in the heat of litigation. Boeing Sikorsky Aircraft Support, B‑277263.2, B‑277263.3, Sept. 29, 1997, 97-2 CPD para. 91 at 15 (GAO accords little weight to agency efforts to defend, in the face of a bid protest, a prior source selection through the submission of new analyses, because such reevaluations and redeterminations prepared in the heat of the adversarial process may not represent the fair and considered judgment of the agency); see, e.g., Advanced Sys. Dev., Inc., B‑298411, B‑298411.2, Sept. 19, 2006, 2006 CPD para. 137 at 18; Remington Arms Co., Inc., supra, at 12.

We recognize that the Air Force now appears to be of the view that “trying to base the Air Force’s projected manpower costs on anything other than what is known, i.e., the MER . . . would amount to nothing more than rank speculation.” Agency Comments, Feb. 3, 2007, at 25. We are not questioning the agency’s position in this regard. Our decision here is simply that the agency may not announce in the solicitation that it will use one evaluation plan and then follow another. Remington Arms Co., Inc., supra, at 15. Here, the agency advised offerors that it would evaluate O&S costs, including “captur[ing] all relevant CSAR-X Operating and Support (O&S) costs,” RFP attachment 13, and further indicated through its inclusion in the SRD of threshold requirements and objectives with respect to various maintenance measures (such as MMH/FH), that it valued aircraft that require less maintenance. Given this solicitation advice, the agency could not ignore the differences between the proposed aircraft with respect to required maintenance.

We therefore sustain the protests on the basis that the Air Force, by ignoring differences among the proposed aircraft that could have a material impact on likely O&S costs, departed from its stated evaluation approach. Further, the evaluation of O&S costs based on each offeror’s proposed helicopter could have resulted in lower evaluated total costs for both of the protesters, which could have resulted in a different source selection.

RECOMMENDATION

We recommend that the Air Force amend the solicitation to clarify its intent with respect to the evaluation of O&S costs, reopen discussions with offerors consistent with our conclusions above, and then request revised proposals. If the evaluation of revised proposals results in a determination that Boeing’s proposal no longer represents the best value to the government, the agency should terminate its contract. We also recommended that Sikorsky and LMSI be reimbursed the costs of filing and pursuing their protests, including reasonable attorneys’ fees. 4 C.F.R. sect. 21.8(2)(1). In accordance with 4 C.F.R. sect. 21.8(f)(1), the protesters’ certified claims for such costs, detailing the time expended and costs incurred, must be submitted directly to the agency within 60 days after receipt of this decision.

The protests are sustained.

Gary L. Kepplinger
General Counsel