B-299145; B-299145.2; B-299145.3, Sikorsky Aircraft Company; Lockheed Martin Systems Integration‑Owego, February 26, 2007
![[Select for PDF file]](/graphics/pdf.gif)
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.
Decision
Matter of: Sikorsky Aircraft Company; Lockheed Martin Systems Integration‑Owego
File: B-299145; B-299145.2; B-299145.3
John W. Chierichella, Esq., Jonathan S. Aronie, Esq., Anne B. Perry, Esq., Keith R. Szeliga, Esq., and Jesse J. Williams, Esq., Sheppard Mullin, for Sikorsky Aircraft Company; Marcia Madsen, Esq., David F. Dowd, Esq., Michael E. Lackey, Jr., Esq., William L. Olsen, Esq., and Luke Levasseur, Esq., Mayer, Brown, Rowe & Maw, and Bucky P. Mansuy, Esq., Lockheed Martin Corporation, for Lockheed Martin Systems Integration‑Owego, the protesters.
Paul
F. Khoury, Esq., Scott M. McCaleb, Esq., William J. Colwell, Esq.,
Nicole Owren‑Wiest, Esq., Kevin Plummer, Esq., and Rand L. Allen,
Esq., Wiley Rein LLP, and Mark W. Reardon, The Boeing Company, for The Boeing Company, the intervenor.
Michael
O’Farrell, Esq., and Bridget E. Lyons, Esq., Department of the Air Force, for the agency.
David A. Ashen, Esq., and John M. Melody, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.
DIGEST
Protest is sustained where (1)
solicitation for combat search and rescue aircraft provided that
cost/price would be calculated on the basis of Most Probable Life Cycle Cost,
including both contract and operations and support (O&S) costs,
(2) solicitation requested detailed information quantifying required
maintenance for proposed aircraft, and (3) agency nevertheless normalized cost
of maintenance when calculating O&S costs, thereby ignoring potentially
lower cost of asserted low maintenance helicopters; once offerors are informed
of criteria against which proposals will be evaluated and award made, agency
must adhere to those criteria.
DECISION
Sikorsky Aircraft Company and Lockheed Martin Systems Integration‑Owego (LMSI) protest the Department of the Air Force’s award of a contract to The Boeing Company, under request for proposals (RFP) No. FA8629-06-R-2350, for the Combat Search and Rescue Replacement Vehicle (CSAR-X). Sikorsky and LMSI challenge the evaluation of proposals and resulting source selection.
The HH-60 helicopter currently serves as the agency’s
combat search and rescue aircraft, used to recover downed aircrew and isolated
personnel. However, as indicated in the
contemporaneous record, and confirmed at the hearing our Office conducted in
this matter, the Air Force has determined that its aging fleet of HH-60s has
several shortfalls, including:
insufficient combat radius; a cabin and payload that are too small;
inadequate survivability, including inadequate defensive systems and armament;
inadequate adverse weather capability; inadequate battlespace awareness; and,
as an aircraft nearing the end of its service life, difficulties in maintaining
the aircraft and inadequate availability.
Source Selection Decision (SSD) at 2; RFP sect. L 1.1; CSAR-X Final
Evaluation Brief,
Award was to be made to the offeror whose proposal
represented the “best value” based on consideration of four evaluation
factors: (1) mission capability,
including (in descending order of importance) subfactors for Block 0 aircraft
performance, system architecture and software, systems engineering, product
support, management/schedule and Block 10 aircraft performance; (2) proposal
risk, assessed at the subfactor level, and focusing on weaknesses associated
with an offeror’s proposed approach and the resulting potential for disruption
of schedule, increased cost, degradation of performance, the need for increased
government oversight, and the likelihood of unsuccessful contract performance;
(3) past performance; and (4) cost/price.
Of particular importance here, the solicitation provided that, for
evaluation purposes, cost/price would be calculated on the basis of the Most
Probable Life Cycle Cost (MPLCC) for the aircraft, comprised of the cost/price
of: (1) system development and
demonstration (SDD); (2) the award fee; (3) production; (4) operations and
support (O&S); (5) other government costs; and (6) proposal risk
adjustments, based on a quantification in terms of schedule and cost of the
impact of the technical, programmatic and schedule risk of the offeror’s
proposed approach.
Boeing, Sikorsky and LMSI submitted initial proposals by
the
All three proposals were included in the competitive range. After conducting several rounds of written and oral discussions--including several face-to-face “debriefings” during which offerors were advised of the agency’s current evaluation of their technical proposals and past performance--and after a flight evaluation to assess the current capability of the offerors’ baseline aircraft, the Air Force requested final proposal revisions (FPR) by September 18, 2006. The Air Force evaluated the FPRs as follows:
LMSI |
Sikorsky |
Boeing |
|
Proposal Risk |
|||
Block
0 Performance |
Acceptable/High |
Acceptable/Low |
Exceptional/Low |
Architecture/Software |
Acceptable/ Moderate |
Acceptable/Moderate |
Acceptable/Moderate |
Systems
Engineering |
Acceptable/Moderate |
Acceptable/Moderate |
Acceptable/Moderate |
Product
Support |
Exceptional/Low |
Exceptional/Low |
Acceptable/Low |
Management/Schedule |
Acceptable/Moderate |
Exceptional/Moderate |
Exceptional/Moderate |
Block
10 Performance |
Acceptable/Moderate |
Acceptable/Moderate |
Exceptional/Moderate |
Past
Performance |
Little Confidence |
Satisfactory
Confidence |
Satisfactory
Confidence |
Price/Cost |
($ million) |
($
million) |
($
million) |
SDD Proposed |
[REDACTED] |
[REDACTED] |
[REDACTED] |
SDD Adjustment |
[REDACTED] |
[REDACTED] |
[REDACTED] |
Production Proposed |
[REDACTED] |
[REDACTED] |
[REDACTED] |
Production Adjusted |
[REDACTED] |
[REDACTED] |
[REDACTED] |
OGC |
[REDACTED] |
[REDACTED] |
[REDACTED] |
Subtotal |
[REDACTED] |
[REDACTED] |
[REDACTED] |
O&S |
[REDACTED] |
[REDACTED] |
[REDACTED] |
MPLCC |
$35,853.7 |
$38,531.8 |
$38,891.5 |
Based on the evaluation results, the source selection
authority (SSA) determined that Boeing’s proposal represented the best
value. As an initial matter, according
to the SSD, because cost/price “was so close between the offerors” and was
significantly less important than the other evaluation factors combined,
“Cost/Price ratings did not provide a great deal of weight” in the SSA’s source
selection decision; rather, “Mission Capability/Proposal Risk Subfactors and
Past Performance became the basis for discriminating between the
offerors.” SSD at 6. In this regard, the SSA noted that LMSI’s
proposal had received a high risk rating under the Block 0 performance
subfactor due to [REDACTED]. According to
the SSA, since “[REDACTED],” he “saw no reason to take on that HIGH level of
risk.”
The SSA also determined that Boeing’s proposal was more
advantageous than Sikorsky’s lower cost proposal. In this regard, while acknowledging
Sikorsky’s advantage under the product support subfactor, where it was rated
exceptional while Boeing’s was rated only acceptable, the SSA emphasized that
Boeing’s was the only proposal that received an excellent rating under the
Block 0 performance subfactor, having received significantly more strengths
(including strengths for mission load and occupant protection) than Sikorsky’s
proposal. The SSA also noted Boeing’s
advantage under the Block 10 performance subfactor (the least important subfactor),
where Boeing’s proposal was rated exceptional and Sikorsky’s only
acceptable. The SSA concluded that
Boeing’s greater mission capability strengths outweighed Sikorsky’s “slightly
lower” MPLCC.
Upon learning of the resulting award to Boeing (on
In reviewing protests against allegedly improper
evaluations, it is not our role to reevaluate proposals. Rather, our Office examines the record to
determine whether the agency’s judgment was reasonable and in accord with the
RFP criteria and applicable procurement statutes and regulations. See Rolf Jensen & Assocs., Inc.,
B‑289475.2, B-289475.3,
As noted above, the solicitation provided that, for evaluation purposes, cost/price would be calculated on the basis of the MPLCC for the CSAR-X aircraft, including (among other costs) O&S costs. In this regard, the solicitation included RFP attachment 13, “O&S Data Form,” the “primary purpose” of which was to “capture all relevant CSAR-X Operating and Support (O&S) costs”; offerors were required to complete the attachment with estimates of specified O&S costs for their proposed aircraft through FY 2029, when the last of the new CSAR-X aircraft were to become fully operational. The agency then extended all O&S costs from FY 2029 to generate a complete life cycle for each aircraft, resulting in an overall CSAR-X life cycle through FY 2039. Contracting Officer’s Statement (Sikorsky) at 22; CSAR-X O&S Cost FPR Evaluation at 3.
Attachment 13 listed the following O&S cost categories: (1) Unit Mission Personnel, including active duty, Air National Guard and Air Force reserve personnel; (2) Unit Level Consumption, including Fuel, Consumables, Repair Costs (Repair Material, Repair Labor and Replenishment Spares), and Training Munitions; (3) Depot Maintenance; (4) Contractor Support, including Training System and Other Contractor Costs (Material and Labor); (5) Sustaining Support, including Support Equipment, Sustaining Engineering and Software Maintenance; and (6) Indirect Support, including active duty, Air National Guard and Air Force reserve personnel. Offerors were required to estimate the O&S costs of their proposed aircraft in these categories through FY 2029, with the exception that attachment 13 stated as follows: “Unit Mission Personnel, Training Munitions, and Indirect Support will be calculated by the government team. Leave blank.”
In addition, offerors were required to comply with, and furnish “sufficient detail for effective evaluation and for substantiating the validity of stated claims” with respect to, various requirements in the CSAR-X System Requirements Document (SRD) quantifying the amount of maintenance the proposed aircraft would need during their service life. RFP sect. L 2.0. For example, offerors were required to submit the maintenance manhours per flight hour (MMH/FH) their aircraft required; the CSAR-X SRD established as a threshold, or minimum, requirement that the CSAR-X aircraft not require more than 20 MMH/FH and, as an objective, that it not require more than 10 MMH/FH. SRD sect. 3.1.4.2.4. Other examples include requirements for offerors to specify: (1) mean time to repair their aircraft, with the SRD specifying a threshold requirement of no more than 1.5 hours and an objective of 0.5 hours, SRD sect. 3.1.4.2.1; (2) time required for Mid Term Scheduled Inspections, with a threshold requirement of 20 manhours and an objective of 10 manhours, SRD sect. 3.1.5.3.2; (3) time to repair a failed system after a sortie, with a threshold requirement of completion within 12 hours 80 percent of the time and an objective of completion within 12 hours 90 percent of the time, SRD sect. 3.1.4.2.1; and (4) time to complete any required long‑term scheduled inspections and repair, with a threshold requirement of completion using no more than 960 manhours and an objective of completion using no more than 480 manhours, SRD sect. 3.1.5.3.3.
The Air Force calculated the O&S cost of Boeing’s proposal as $[REDACTED] billion in “Then Year” (TY) (inflated) dollars, Sikorsky’s as $[REDACTED] billion, and LMSI’s as $[REDACTED] billion. In calculating these O&S cost estimates, the Air Force used the same costs for Unit Mission Personnel, Training Munitions, MER Contractors and Indirect Support for each offeror. The government-generated personnel-based costs for Unit Mission Personnel ($9.611 billion) and Indirect Support ($3.609 billion) accounted for most of the calculated O&S cost totals for each offeror--approximately [REDACTED] and [REDACTED] percent, respectively, of LMSI’s evaluated O&S cost; [REDACTED] and [REDACTED] percent of Sikorsky’s O&S cost, and [REDACTED] and [REDACTED] percent of Boeing’s O&S cost.[1]
Sikorsky and LMSI assert that the agency’s calculation of O&S costs using the same estimated cost for Unit Mission Personnel and Indirect Support for all proposals, irrespective of the aircraft offered, unreasonably failed to account for the reduced maintenance required by their generally newer design, smaller helicopters. They claim that this was inconsistent with the RFP’s requirement for detailed information regarding O&S costs generally, and maintenance costs specifically. The protesters read this information requirement, together with the MPLCC evaluation approach, as indicating that the O&S cost evaluation would take into account the helicopter offered.
We agree with the protesters. It is true that the solicitation provided that “Unit Mission Personnel . . . and Indirect Support will be calculated by the government team,” Attach. 13, and similar language in a different context could mean that the agency would use normalized, i.e.,“plug,” numbers for all offerors. Here, however, the RFP also stated the agency’s intention to “capture all relevant CSAR-X Operating and Support (O&S) costs,” id., and requested the submission of various measures and information intended to quantify the amount of maintenance the proposed aircraft would need during their service life. The solicitation further indicated through its inclusion in the SRD of threshold requirements and objectives with respect to various maintenance measures (such as MMH/FH), that it valued aircraft that require less maintenance. In the face of these solicitation provisions, we think the statement that the government would calculate the Unit Mission Personnel and Indirect Support costs reasonably could only be interpreted as indicating that the agency would perform those calculations taking into account the unique aspects of the proposed aircraft (including maintenance requirements) as identified in the required information. Only in this way would the agency’s evaluation “capture all relevant CSAR-X Operating and Support (O&S) costs,” as it announced was its intent.
We further agree that the agency’s methodology for
calculating O&S costs did not reasonably account for each offeror’s unique
technical approach, including very different proposed helicopters, and thus was
inconsistent with the stated evaluation scheme.
In this regard, the Air Force based the evaluated cost for Unit Mission
Personnel‑‑that is, the primarily base-level active duty, Air
National Guard and Air Force reserve personnel in the contemplated 13 CSAR-X
squadrons‑‑and for MER Contractors on the staffing set forth in the
Air Force’s MER for the Replacement Personal Recovery Vehicle (CSAX), March 30,
2006 version, issued by the Air Force Special Operations Command (the
warfighting command previously responsible for the combat search and rescue
mission). The MER specifies that the
maximum total staffing requirement “under the current concept of operations,” including
both government and contractor personnel, will be 6,232 manyears (or 5,539
manyears excluding contractors and certain technicians), more than half of
which were listed under the maintenance function. MER at 2, 16; Tr. at 37-42, 378-81, 1181-82; Agency
Comments,
The Air Force maintains that it was reasonable to base its O&S cost analysis on the MER staffing because the MER represented the official agency staffing for the contemplated CSAR-X squadrons. However, the fact that the MER may have been considered the official staffing level is irrelevant here. The question is not whether the MER could be used in the agency’s analysis; rather, the question is whether the agency reasonably could rely on the MER to the exclusion of the information provided by offerors regarding the characteristics of their particular aircraft, especially since the MER is based on the current helicopter, not those proposed, and may bear only a limited relationship to the ultimate future maintenance staffing needs for the replacement helicopter. The fact that the MER represented the agency’s current official staffing level does not override what we have found was the evaluation approach reasonably indicated by the RFP--that the maintenance characteristics of the proposed helicopters would be considered in the O&S cost evaluation.
Reliance on the MER is all the more problematic given that
the current HH‑60, which is nearing the end of its service life, has
“very high” maintenance requirements, requiring “upward around” 25‑30
MMH/FH. Tr. at 681. In contrast, as discussed above, the CSAR-X
SRD established as a minimum that the CSAR-X aircraft shall not require more
than 20 MMH/FH, and stated as an objective that the aircraft should not
require more than 10 MMH/FH. SRD
sect. 3.1.4.2.4. Consistent with the
SRD, all three offerors estimated significantly lower MMH/FH for their proposed
helicopters than is required for the current HH‑60. Boeing’s proposal, generally referring to
extensive flying experience for the H-47 family of aircraft during Operation
Enduring Freedom and Operation Iraqi Freedom, and specifically relying for part
of its calculation upon experience with the MH-47E helicopter, generally
referred to an estimated MMH/FH for the CSAR-X of [REDACTED] hours. (We note that the specific calculations
supporting this number actually indicated an estimated MMH/FH of
[REDACTED] hours.) Boeing Mission
Capability Proposal, 1(d) Product Support, sect. 4.2, at 46, 50-51,
62-63. LMSI, noting that its current EH‑101
platforms had accumulated over [REDACTED] flight hours, and had
specifically demonstrated an MMH/FH of below [REDACTED] hours during combat
operations in
Further, it is clear that the MER will change to reflect
the characteristics, including unique maintenance requirements, of the selected
CSAR-X helicopter. Not only does the MER
itself acknowledge that it is a “living document” but, in fact, there already
have been three editions of the MER during the course of this procurement,
including the
The agency asserts that consideration in the O&S cost evaluation of the offerors’ predicted MMH/FH would have been inappropriate. In this regard, the agency argues that only the offerors’ contractually binding commitments, as set forth in each offeror’s proposed Weapon System Specification (WSS)--which was to be incorporated into the offeror’s contract, RFP sect. L-2--were meaningful, and notes that [REDACTED].
We find the agency’s position unpersuasive. First, again, whether or not the agency believed considering the characteristics of the offerors’ proposed aircraft was inappropriate, the fact is that, as already discussed, the RFP indicated that those characteristics would be considered. Further, we note that the agency’s current reliance on the WSS numbers represents a departure from its approach during the O&S cost evaluation; the WSS numbers simply were not used in the O&S cost evaluation. Testimony from agency witnesses indicates that the offerors’ WSS numbers were irrelevant to the O&S cost calculations, such that if one offeror had included an MMH/FH of 19 hours in its WSS and another had included an MMH/FH of 1 hour, the difference would not have affected the evaluated O&S costs. Tr. at 594-99.[3]
The agency also questions the protesters’ MMH/FH
predictions, focusing in particular on [REDACTED] estimate that its proposed
aircraft would have an MMH/FH of fewer than [REDACTED] hours. (As noted above, [REDACTED] predicted an
MMH/FH of [REDACTED] hours.)
Specifically, the agency claims that neither Sikorsky’s nor LMSI’s
proposals included sufficient data to support their claimed MMH/FH
numbers. Again, the contemporaneous record
does not support the agency’s position.
While the agency now focuses on the reliability of the offerors’
predictions, and one of the evaluators testified that he questioned during the
evaluation whether [REDACTED] could achieve its predicted MMH/FH, it is clear
from the hearing testimony that the agency did not calculate a probable MMH/FH
for the aircraft, but instead simply determined that each proposed aircraft
could achieve the SRD objective. Tr.
at 42‑46, 77, 99‑101, 654‑57, 660‑61, 979. Thus, to the extent that the agency now
questions Sikorsky’s and LMSI’s predictions that their aircraft would require
significantly less maintenance than would Boeing’s aircraft, see Agency
Comments,
We recognize that the Air Force now appears to be of the
view that “trying to base the Air Force’s projected manpower costs on anything
other than what is known, i.e., the MER . . . would amount to nothing
more than rank speculation.” Agency
Comments,
We therefore sustain the protests on the basis that the Air Force, by ignoring differences among the proposed aircraft that could have a material impact on likely O&S costs, departed from its stated evaluation approach. Further, the evaluation of O&S costs based on each offeror’s proposed helicopter could have resulted in lower evaluated total costs for both of the protesters, which could have resulted in a different source selection.
RECOMMENDATION
We recommend that the Air Force amend the solicitation to clarify its intent with respect to the evaluation of O&S costs, reopen discussions with offerors consistent with our conclusions above, and then request revised proposals. If the evaluation of revised proposals results in a determination that Boeing’s proposal no longer represents the best value to the government, the agency should terminate its contract. We also recommended that Sikorsky and LMSI be reimbursed the costs of filing and pursuing their protests, including reasonable attorneys’ fees. 4 C.F.R. sect. 21.8(2)(1). In accordance with 4 C.F.R. sect. 21.8(f)(1), the protesters’ certified claims for such costs, detailing the time expended and costs incurred, must be submitted directly to the agency within 60 days after receipt of this decision.
The protests are sustained.
Gary L. Kepplinger
General Counsel
[1]
The agency calculated the cost for MER Contractors, the staffing for which was
listed in the MER, as $591.4 million, which was added to each offeror’s MPLCC.
[2] The Air Force maintains that LMSI’s and Sikorsky’s protests of the agency’s approach to using the MER in the calculation of O&S costs are untimely. In this regard, our Bid Protest Regulations provide that protests of alleged solicitation improprieties must be filed prior to the next closing time for receipt of proposals, and other protests shall be filed not later than 10 days after the basis for protest is known or should have been known. 4 C.F.R. sect. 21.2(a) (2006). According to the agency, RFP attachment 13--which provided that “Unit Mission Personnel, Training Munitions, and Indirect Support will be calculated by the government team”--in conjunction with the fact that during the course of the procurement it presented a slide stating that the “Manpower Estimate Report (MER) [dated] Mar 06 will be used for Unit Mission Personnel,” placed offerors on notice of the agency’s intended evaluation approach. We disagree. While offerors were on notice that the agency intended to use the MER in evaluating O&S costs, this did not reasonably advise them that the agency would ignore the actual proposed aircraft and technical approaches when calculating most of the O&S costs, and that the agency thereby was abandoning its stated approach of “captur[ing] all relevant CSAR-X Operating and Support (O&S) costs.” RFP attach. 13.
At the pre-hearing conference conducted by our Office,
the agency for the first time also asserted that the cost team lead had advised
Sikorsky (but not LMSI) during a February debriefing that the same manpower
levels (with no consideration of the proposed aircraft’s reliability) would be
used in calculating the O&S cost associated with each offeror’s proposal. However, the agency did not identify the
Sikorsky employee allegedly provided the information, and all six Sikorsky
executives who attended the debriefing (which was actually conducted in March
and not February) have submitted sworn declarations stating that, while Sikorsky
was advised that the MER would be used in the calculating O&S costs, it was
not advised that the MER would be used in lieu of consideration of the
significant characteristics of the proposed aircraft. Sikorsky Comments,
[3]
This testimony is consistent with the fact that the contemporaneous evaluation
documentation gives no indication that the O&S calculations took into
account such differences in the maintenance‑related measures included in
offerors’ WSSs as: [REDACTED]. CSAR-X Final Evaluation Brief,