DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Navistar Defense, LLC; BAE Systems, Tactical Vehicle Systems LP

File: B-401865; B-401865.2; B-401865.3; B-401865.4; B-401865.5; B-401865.6;B-401865.7

Date: December 14, 2009

DIGEST

1. Protest challenging evaluation of awardee's technical proposal is sustained where agency did not reasonably apply a solicitation criterion which stated that existing production capabilities will be viewed as having less risk than those that do not currently exist.

2. Protest challenging evaluation of protester's past performance is sustained where the agency cannot produce a record that demonstrates the basis for the evaluation.

3. Protest challenging evaluation of awardee's proposed price is denied where the depth and detail of the agency's price realism analysis was a matter within its discretion, and the record shows that the evaluation was reasonable.

Navistar Defense, LLC, of Warrenville, Illinois, and BAE Systems, Tactical Vehicle Systems LP, of Sealy, Texas, protests the award of a contract to Oshkosh Corporation, of Oshkosh, Wisconsin, under request for proposals (RFP) No. W56HZV-09-R-0083, issued by the Department of the Army, U.S. Army Tank-Automotive and Armaments Command, for production of the family of medium tactical vehicles (FMTV). Navistar and BAE challenge the Army's evaluation of the offerors' technical and price proposals, and contend that the selection decision was flawed.

We sustain the protests.

BACKGROUND

The FMTV is a group of diesel engine, all-wheel drive trucks and companion trailers. The FMTV program began in 1991 and replaced a number of different vehicles with a "family" of trucks and trailers based on a common cab and chassis, using a set of common parts. The commonality of designs and parts is intended to simplify production and maintenance, and to reduce costs. The FMTV consists of 14 truck and 3 trailer variants which vary by load capacity and mission requirements. As part of the Army's long-term armor strategy (LTAS), an armored version of the FMTV cab was developed with the capability of being fitted with additional armor, known as a "B-Kit." BAE is the incumbent contractor for production of the FMTV.

The solicitation was issued on February 27, 2009, and sought proposals to provide the follow-on production for the FMTV program. The RFP anticipated award of a 5-year requirements contract with fixed unit prices for the trucks and trailers, certain fixed-price options such as B-Kit armor and alternate paint, and a cost-plus-fixed-fee option for system technical support. The estimated quantities for the Army's requirements are 12,415 trucks and 10,926 trailers. The trucks and trailers are to be built in accordance with a technical data package (TDP) which is owned by the government, and which was provided to offerors. The design for the cab portion of the trucks is a proprietary design owned by BAE; certain drawings for this cab were provided to offerors.

The RFP identified three evaluation factors: (1) capability, (2) past performance and small business participation, and (3) price.[1] RFP sect. M.6.2. The capability and price factors were of equal weight, and were each more important than the past performance and small business participation factor.

The capability factor had two subfactors, production capability, and technical; the production capability subfactor was "significantly more important" than the technical subfactor. Id. Within the production capability subfactor, the RFP identified seven elements of different weights. Specifically, the RFP stated that "the Manufacturing Facilities, Time Phased Critical Path, and Key Tooling/Equipment Elements are equal and individually more important than the Quality System and Subcontractor Letters of Commitment Elements which are equal and individually more important than Production Approach and Workforce/Manpower Planning Elements which are equal in importance." Id. In addition, the RFP advised that a production capability that "currently exists" would be evaluated as having a lower risk than a production capability "that does not currently exist." RFP sect. M.9.1.

Within the significantly less important technical subfactor, the RFP identified two subfactors and advised that "the Proposed Cab Design Element is more important than the Innovation Experience Element." Id. Within the past performance and small business participation factor, past performance was "significantly more important" than small business participation. Id.

Under the price factor, the solicitation stated that the agency would evaluate the reasonableness and realism of the Offeror's total evaluated price. RFP sect. M.8.1. The RFP instructed offerors to provide a cost buildup for their fixed prices, with data regarding profit and costs such as material, labor, overhead, testing, engineering, other direct, and general and administrative. RFP sect. L.2.4. As relevant here, the RFP also required offerors to submit a bill of materials for direct cost items, as follows:

Material Cost. Offeror will provide the total direct material per vehicle. Provide a link to a separate bill of material spreadsheet for items with a unit price greater than $500 per vehicle. On this bill of material include description, vendor name, part number, quantity and unit of measure (per vehicle), price per item and extended [price] per vehicle, basis of price (quote, purchase order, estimate, etc.)

RFP sect. L.2.4.1.

The RFP identified an estimated vehicle and trailer delivery schedule for the 5-year contract period for proposal planning and evaluation purposes. RFP sections L.3.1.1, L.3.1.2. The schedule set forth initial delivery requirements of 24 trucks and 6 trailers in May and June 2010 for first article testing (FAT). Id. No deliveries were due from June to September, and full production deliveries were to begin in October 2010. Id.

The Army received initial proposals from three offerors by the closing date of May 27: Navistar, BAE, and Oshkosh. As relevant here, Oshkosh stated in its proposal for the facilities element of the production capability subfactor that the company did not currently have one of the required facilities, an e-coat facility.[2] Navistar Agency Report (AR) [3], Supp. Documents, Oshkosh Initial Proposal, Vol. 2, at 1-109. Instead, Oshkosh stated that it would utilize subcontractors to perform the required e-coating for the early production requirements in May and June 2010, and would complete its own e-coat facility by July 2010. As also relevant here, Oshkosh stated that it would develop its own designs and production approach for the cab based on the proprietary BAE data in the TDP. Navistar AR, Supp. Documents, Oshkosh Initial Proposal, Vol. 2, at 2-9. As part of its approach to the cab design element, Oshkosh proposed to build a prototype cab in order to "eliminate[] production schedule risks associated with the cab TDP and tooling." Id.

The agency convened separate evaluation teams to assess proposals under the capability, past performance/small business participation, and price factors. The agency also convened a source selection evaluation board (SSEB) to review the evaluation findings, and brief the source selection authority (SSA). The Army concluded that all offerors' proposals were within the competitive range. Navistar AR, Tab 11, Quality of Proposals Briefing, at 25. The agency conducted discussions with each offeror through a series of written items for discussions (IFDs) which requested written responses. All three offerors submitted timely final proposals.

The Army's final evaluation ratings, as developed by the SSEB, were as follows:[4]

	NAVISTAR	BAE	OSHKOSH
CAPABILITY FACTOR	EXCELLENT/ VERY LOW RISK	EXCELLENT/ VERY LOW RISK	EXCELLENT/ VERY LOW RISK
PRODUCTION CAPABILITY SUBFACTOR	EXCELLENT/ VERY LOW RISK	EXCELLENT/ VERY LOW RISK	EXCELLENT/ VERY LOW RISK
Manufacturing Facilities	Excellent/ Very Low Risk	Excellent/ Very Low Risk	Excellent/ Very Low Risk
Time Phased Critical Path	Excellent/ Very Low Risk	Excellent/ Very Low Risk	Excellent/ Very Low Risk
Key Tooling & Equipment	Excellent/ Very Low Risk	Excellent/ Very Low Risk	Excellent/ Very Low Risk
Quality System	Good/ Low Risk	Adequate/ Moderate Risk	Good/ Low Risk
Subcontractor Letters of Commitment	Excellent/ Very Low Risk	Excellent/ Very Low Risk	Excellent/ Very Low Risk
Production Approach	Excellent/ Very Low Risk	Excellent/ Very Low Risk	Excellent/ Very Low Risk
Workforce/Manpower	Excellent/ Very Low Risk	Excellent/ Very Low Risk	Excellent/ Very Low Risk
TECHNICAL SUBFACTOR	GOOD/ LOW RISK	EXCELLENT/ VERY LOW RISK	EXCELLENT/ VERY LOW RISK
Cab Design	Good/Low Risk	Excellent/ Very Low Risk	Excellent/ Very Low Risk
Innovative Engineering Change Experience	Excellent/ Very Low Risk	Excellent/ Very Low Risk	Excellent/ Very Low Risk
PAST PERFORMANCE AND SMALL BUSINESS PARTICIPATION FACTOR	GOOD/ LOW RISK	GOOD/ LOW RISK	GOOD/ LOW RISK
PAST PERFORMANCE SUBFACTOR	GOOD/ LOW RISK	GOOD/ LOW RISK	GOOD/ LOW RISK
SMALL BUSINESS PARTICIPATION SUBFACTOR	GOOD/ SOME RISK	EXCELLENT/ LOW RISK	GOOD/ VERY LOW RISK
TOTAL PRICE	[deleted]	$3,462,891,786	$3,023,192,525

Navistar AR, Tab 14, SSD, at 3.

In the selection decision, the SSA found no basis to distinguish between the proposals under any of the evaluation factors, subfactors, or elements where the proposals received equal ratings. Navistar AR, Tab 14, SSD at 3-6. The SSA noted that the offerors had different ratings in three areas: the quality systems element of the production capability subfactor, the technical subfactor (based on a difference in ratings for the cab design element of the technical subfactor), and the small business participation subfactor. Id. at 3-5. Despite these differences, the SSA concluded that "I find all three offerors' ratings in both the Capability Factor and the Past Performance/Small Business Participation Factor" equal, and based her selection decision on the Total Evaluated Price. Id. at 6. After concluding that each offeror's proposed price was reasonable and realistic, the SSA selected Oshkosh's proposal for award because its price was approximately $439.7 million less than BAE's price, and $[deleted] less than Navistar's price. Id.

The Army awarded the contract to Oshkosh on August 26, and provided debriefings to Navistar and BAE on September 2 and 3, respectively. These protests followed.

DISCUSSION

Navistar and BAE each challenge the Army's evaluation under the capability and price factors, and Navistar challenges the agency's evaluation of its own past performance. First, we conclude that the agency's evaluation was unreasonable under the key tooling and equipment element of the production capability subfactor. In this regard, the RFP set forth very clear and specific groundrules about how the agency would evaluate risk under the production capability subfactor, and the selection decision was inconsistent with those groundrules. Second, because the Army was unable to produce critical relevant documents supporting its evaluation of Navistar's past performance, our Office has no basis on this record to conclude that the evaluation was reasonable.

As a preliminary matter, this decision relies, in part, upon the testimony of Army witnesses provided during a hearing conducted by our Office on November 9 and 10, 2009. We heard testimony from the SSA, the CO, the SSEB chair, the price evaluation team leader, the capability factor team leader, the production capability subfactor team leader, and the cab design evaluator. In reviewing an agency's evaluation of offerors' proposals, we do not limit our consideration to contemporaneously-documented evidence, but instead consider all the information provided, including the parties' arguments, explanations, and any hearing testimony. Remington Arms Co., Inc., B-297374, B-297374.2, Jan. 12, 2006, 2006 CPD para. 32 at 10. While we generally give little or no weight to reevaluations and judgments prepared in the heat of the adversarial process, Boeing Sikorsky Aircraft Support, B-277263.2, B-277263.3, Sept. 29, 1997, 97-2 CPD para. 91 at 15, post-protest explanations that provide a detailed rationale for contemporaneous conclusions, and simply fill in previously unrecorded details, will generally be considered in our review of the rationality of selection decisions--so long as those explanations are credible and consistent with the contemporaneous record. NWT, Inc.; PharmChem Labs., Inc., B-280988, B-280988.2, Dec. 17, 1998, 98-2 CPD para. 158 at 16.

Key Tooling and Equipment Element Evaluation

Navistar and BAE first argue that the Army unreasonably evaluated Oshkosh's proposal as excellent/low risk under the key tooling and equipment element of the production capability subfactor. The protesters contend that the Army's selection decision unreasonably concluded that, contrary to the record, Oshkosh had demonstrated that all of the required key tooling and equipment currently existed, and that there was no basis to distinguish between the competing proposals in this area.

During the hearing, the SSA conceded that that selection decision incorrectly stated that Oshkosh possessed all of the required key tooling and equipment. Hearing Transcript (Tr.) at 701:19-703:12. Nonetheless, the Army contends that the solicitation did not require the agency to assess risk based on whether an offeror currently had all key tooling and equipment. Instead, the agency argues that the RFP only required the agency to evaluate an offeror's proposed approach to acquire the required items. In our view, the agency's evaluation of Oshkosh's proposal under the key tooling and equipment element was inconsistent with the plain language of the solicitation.

The evaluation of an offeror's proposal is a matter within the agency's discretion. IPlus, Inc., B-298020, B-298020.2, June 5, 2006, 2006 CPD para. 90 at 7, 13. In reviewing a protest against an agency's evaluation of proposals, our Office will not reevaluate proposals but instead will examine the record to determine whether the agency's judgment was reasonable and consistent with the stated evaluation criteria and applicable procurement statutes and regulations. See Shumaker Trucking & Excavating Contractors, Inc., B-290732, Sept. 25, 2002, 2002 CPD para. 169 at 3. A protester's mere disagreement with the agency's judgment in its determination of the relative merit of competing proposals does not establish that the evaluation was unreasonable. VT Griffin Servs., Inc., B-299869.2, Nov. 10, 2008, 2008 CPD para. 219 at 4. While we will not substitute our judgment for that of the agency, we will question the agency's conclusions where they are inconsistent with the solicitation criteria, undocumented, or not reasonably based. Sonetronics, Inc., B-289459.2, Mar. 18, 2002, 2002 CPD para. 48 at 3.

The RFP required offerors to provide the following information regarding key tooling and equipment in their proposals:

Identify the extent to which the Key Tooling and Equipment currently exists and are configured to produce the FMTV. For any new Key Tooling and Equipment proposed, provide a milestone schedule to obtain any new Key Tooling and Equipment. Identify any impact that acquiring of the new Key Tooling and Equipment may have on the Government's required delivery date or to meet surge and other vehicle or major component production requirements.

RFP sect. L.3.1.3.5.3.

The RFP expressly anticipated making use of the requested information about whether key tooling and equipment currently existed. For example, it stated that the seven elements within the production capability subfactor, including the element of key tooling and equipment, would be evaluated based on the following understanding of risk for each element:

A production capability that (a) currently exists, (b) is producing items which are the same or similar in complexity to the FMTV, and (c) has excess production capacity sufficient to support the anticipated FMTV delivery schedule reflected in paragraphs L.3.1.1 & L.3.1.2, will be considered a lower risk than a proposed production capability that does not currently exist, is not producing items that are the same or similar to the FMTV, or currently has no excess production capacity . . . .

RFP sect. M.9.1.

Within the production capability subfactor, the RFP also provided more detailed information applicable to each of the seven elements. Specifically, the RFP stated that the agency would evaluate offerors' proposals under the key tooling and equipment element of the production capability subfactor as follows:

Key Tooling and Equipment Element. (See L.3.1.3.5 through L.3.1.3.5.3) The SSEB will assess the risk of the offerors plan to acquire Key Tooling and Equipment needed to support production of FMTV trucks and trailers as specified in the delivery schedules in L.3.1.1 & L.3.1.2). The Government will also assess the risk that the milestone schedule for acquiring Key Tooling and Equipment provided by the offeror will reduce or increase the risk of complying with the timely production of key components, FMTV trucks and FMTV trailers.

RFP sect. M.9.1.b.

In answering the RFP, Oshkosh's proposal set forth numerous charts detailing whether the required key tooling and equipment was "on hand" or whether it would be obtained later. Navistar AR, Oshkosh Final Proposal, Vol. 2, at 1-142-1-147, 1-150-1-151, 1-156-1-197. For equipment not yet "on hand," the proposal included a schedule for when the items would be procured, installed, and finally "proved out" for production. Id. As relevant here, Oshkosh's proposal listed over 100 items of key tooling and equipment to be obtained; these items included welding fixtures for the chassis (which were due to be ready by November 2009), welding robots for the cab (which were due to be ready by February 2009), and all of the required items for its proposed e-coat facility (which was due to be completed by July 2010). Id. at 1-146, 1-166, 1-169.

With respect to the cab design element, Oshkosh stated that it had developed a prototype cab "to determine what tooling is necessary, and identify the best way to configure such tooling." Id. at 2-9c. In its final proposal, the awardee explained that due to difficulties encountered in the development of its prototype cab, "[a]dditional tooling will be created to facilitate the production welding process." Id. at 2-9d. The awardee also stated that it had entered into contracts with a vendor to "produce the stampings required for the FMTV LTAS cab." Id. at 2-9.

In assessing Oshkosh's proposal, the Army's evaluators concluded that the proposal "reflects an excellent approach which will clearly support the attainment of contract requirements." Navistar AR, Tab 20, Oshkosh Interim Key Tooling Evaluation, at 3. Moreover, the agency evaluated the awardee's key tooling and equipment under each of six areas identified in section L of the RFP, and concluded that the items existed, had been procured, or were on schedule to be obtained. Id. at 4. In summary, the agency evaluators rated Oshkosh excellent/very low risk overall under the key tooling and equipment element of the production subfactor based on their conclusion that the proposal presented no weaknesses and had one strength, i.e., "Offeror has procured LTAS cab stamping dies in advance." Id. The SSEB final report repeated the same conclusions. Navistar AR, Tab 13, SSEB Final Report, at 38.

In the selection decision, the SSA stated that the offerors each merited excellent/very low risk ratings under the key tooling and equipment element based on her conclusion that all offerors had the needed items as follows:

In evaluating Key Tooling & Equipment, I find that BAE has all necessary tooling and equipment on hand and in good condition. I find that Navistar has all necessary tooling and equipment on hand and in good condition at the Springfield Assembly Plant line that will be specific to the FMTV and that their trailer manufacturer has all the necessary tooling and equipment from their commercial trailer operation. I find that Oshkosh has all the necessary key tooling and equipment, including procurement of stamping dies for their LTAS cab in advance of award to mitigate risk.

Navistar AR, Tab 14, SSD at 4.

During the hearing, however, the SSA testified that the above quote (from the SSD) was inaccurate, and did not reflect her understanding about whether Oshkosh and Navistar had all of the necessary key tooling and equipment. Tr. at 701:19-703:12. Instead, the SSA testified that she understood that neither Oshkosh nor Navistar had all of the required key tooling and equipment at the time of the selection decision, even though this understanding is not reflected in the decision document. Id.

The agency argues that the SSA's judgment here was reasonable because the solicitation required only that offerors demonstrate their plans to acquire key tooling and equipment. The agency argues that its intent in drafting the "currently exists" clause was merely to limit competition to a company that has the ability to manufacture trucks and trailers similar to the FMTV, as opposed to, for example, a smaller company without such abilities. See Agency Post-Hearing Comments at 7-8. In this regard, the SSA testified that she understood the RFP as merely requiring offerors to demonstrate a "credible plan" for obtaining the required key tooling and equipment. Id. at 703:4-7.

While the solicitation anticipated that the agency would evaluate an offeror's "plan to acquire Key Tooling and Equipment," it also states that an offeror whose key tooling and equipment does not exist will be viewed as presenting more risk than an offeror that has such items on hand. We see no contradiction in the language of the two provisions, and we think they must be read together in a way that gives meaning to both provisions.

As discussed above, Section L of the RFP instructed offerors to [i]dentify the extent to which the Key Tooling and Equipment currently exists and are configured to produce the FMTV." RFP para. L.3.1.3.5.3. The "currently exists" clause stated that an offeror will be evaluated as presenting lower risk if its production capability has all three of the following characteristics: (1) it exists, (2) it is producing similar items, and (3) it has excess capacity. Conversely, an offeror could expect a higher risk rating if its production capacity: (1) did not exist, (2) was not producing similar items, or (3) did not have excess capacity. RFP sect. M.9.1.

We think that the solicitation's ground rules for evaluating the key tooling element clearly intended to differentiate between existing and non-existing production capabilities. Navistar AR, Tab 20, Oshkosh Interim Key Tooling Evaluation, at 4. The SSEB chair and the SSA each stated that they understood the clause to refer to an offeror's factory or facility, rather than the whole of the production capability subfactor. See Tr. at 606:5-606:9 (testimony of the SSEB chair); 705:19-706:16 (testimony of the SSA). The RFP, however, states that the agency will consider the risk associated with an offeror's "production capability"--under which the RFP identified seven elements--and not just the element that considered an offeror's manufacturing facilities. We think the agency's interpretation of the solicitation's "currently exists" language, as related by agency witnesses is inconsistent with the plain language of the RFP.

The record shows that the Army failed to reasonably consider the comparative risk in Oshkosh's ability to procure, install, and "prove out" the required key tooling and equipment.[5] As discussed above, Oshkosh's proposal identified more than 100 items of key tooling and equipment that were not on hand--including items in categories where the agency incorrectly concluded that all necessary items already existed. See Navistar AR, Tab 20, Oshkosh Interim Key Tooling Evaluation, at 3. On this record, we think that the agency's evaluation was inconsistent with the terms of the solicitation, and we sustain the protest on this basis.[6]

Past Performance Subfactor Evaluation

Navistar next argues that the Army unreasonably evaluated its past performance as good/low risk, as opposed to excellent/very low risk. It also contends that the record here does not contain the documents relied upon by the agency for its evaluation, and therefore does not support the rating. We agree with Navistar. Since this record provides no basis for our Office to determine whether the evaluation was reasonable in this area, we sustain this challenge.

The RFP required offerors to submit references for up to five recent and relevant commercial or government contracts. RFP sect. L.4.1. The RFP advised that the agency would evaluate offerors' past performance based on three primary areas: technical performance, delivery, and business relations. RFP sect. M.10.1.1. The RFP provided a questionnaire for past performance references which listed 20 questions addressing technical, delivery, and customer satisfaction performance. Offerors were required to "send a copy of the past performance questionnaire directly to the appropriate Contracting Officer Representative (COR), PCO and relevant customers." RFP sect. L.4.1.4.

Navistar identified two prior contracts for past performance references: a Marine Corps contract for production of mine-resistant ambush protected (MRAP) vehicles, and an Army contract for manufacture, test and support of medium tactical vehicles (MTVs) under a Foreign Military Sales contract.[7] Navistar AR, Final Proposal, Vol. 3, at 8-15. The agency received questionnaire responses from four officials familiar with Navistar's performance on the MRAP contract: the acquisition CO, the program manager, a contracting specialist, and a representative of the Defense Contract Management Agency (DCMA). Navistar AR, Tab 30, Navistar Interim Past Performance Evaluation[8]; Supp. Documents, Deleted SSEB Interim Report Slides, at 4.[9]

These responses were summarized in the Army's interim past performance evaluation rating, and the numerical scores were tallied in a chart in the SSEB interim report. Id. For the MTV Troop/Cargo contract, Navistar received an overall "excellent" rating. Navistar AR, Tab 30, Navistar Interim Past Performance Evaluation, at 6. For the MRAP contract, Navistar received an overall "adequate" rating, based on the following element ratings: adequate for technical performance, adequate for delivery, and good for customer relations. Id.

In its report on the protest, the Army provided only two questionnaires concerning Navistar, one for its MTV contract, and one from the acquisition CO (ACO) for its MRAP contract. The MRAP ACO's questionnaire response contained three positive, and no negative ratings and comments. The deleted SSEB briefing slides show, however, that the agency received questionnaire responses for Navistar's MRAP contract from four individuals, providing 32 responses. Navistar AR, Supp. Documents, Deleted SSEB Interim Report Slides, at 4. The agency explains its inability to produce the missing evaluation documents as follows:

The Army admits that it cannot produce the Past Performance Questionnaire (PPQ) response containing the adverse comments/ ratings from the USMC Program Manager . . . for the MRAP . . . vehicles. Unfortunately the response to the questionnaire was not saved electronically (i.e. scanned) and the paper copy, along with copies of all of the PPQ responses were shredded at the conclusion of the SSEB. This was an inadvertent error and unintentional.

Navistar SAR at 3. Thus, of the 32 questionnaire answers received from the four individuals identified above, the documents produced by the agency--the MRAP ACO's questionnaire response and the agency's IFDs and interim past performance evaluation--support only three of the positive ratings, and the agency's summary of seven of the negative ratings. There is no record of the narrative comments for the 22 other ratings received--almost all of which appear, based on the numerical summaries, to be positive--or the agency's evaluation of these ratings.

Navistar challenges its "adequate" rating for the MRAP contract, arguing that the agency did not reasonably evaluate its responses to discussions questions. Additionally, the protester also contends that the agency's evaluation cannot support its rating of adequate for the MRAP contract, and hence its past performance rating overall, because the agency cannot provide the documentation for the evaluation.

Notwithstanding its inability to produce the relevant questionnaires concerning the MRAP contract, the Army argues that the record is adequate for our review because the agency allowed Navistar to address negative ratings and comments provided by the MRAP PM during discussions. Although Navistar responded to each negative rating and comment, the Army's evaluation states that the questionnaire respondent found no basis to change the negative rating. AR, Tab 30, Navistar Interim Past Performance Evaluation, at 6-7.

While, as a general matter, the evaluation of an offeror's past performance is a matter within the discretion of the contracting agency, we will question an agency's evaluation of past performance where it is unreasonable or undocumented. Clean Harbors Envtl. Servs., Inc., B-296176.2, Dec. 9, 2005, 2005 CPD para. 222 at 3. Although an agency is not required to retain every document generated during its evaluation of proposals, the agency's evaluation must be sufficiently documented to allow our Office to review the merits of a protest. Apptis, Inc., B-299457 et al., May 23, 2007, 2008 CPD para. 49 at 10. Where an agency fails to document or retain evaluation materials, it bears the risk that there may not be adequate supporting rationale in the record for us to conclude that the agency had a reasonable basis for its source selection decision. Id.

In sum, we think that the record here is inadequate for our Office to determine whether the agency's evaluation of Navistar's past performance was reasonable. The agency's evaluation cited the existence of positive and negative comments; however, only certain of the negative comments were described, leaving our Office with no insight as to the nature of almost all of the positive comments, and no insight as to how the agency viewed any of them.[10] Because we cannot review the documents upon which the agency relied, we cannot determine whether the conclusions that the agency drew from them with regard to both the positive and negative comments were supported by these documents. We sustain the protest.

Remaining Issues

The issues discussed above reflect the areas where we agreed with the protesters, and concluded that they were prejudiced by the errors in the agency's evaluation. The remainder of this decision discusses issues where we disagree with the protesters. These areas involve: (1) BAE's argument that the agency should have found Oshkosh's cab design unacceptable due to its use of materials different from those specified in the TDP (or alternatively, that the agency should have assessed greater risk in the proposal's approach to cab design); (2) the agency's evaluation of price; and (3) Navistar's argument that the agency's affirmative determination of responsibility with respect to Oshkosh is flawed.[11]

Cab Design Element Evaluation

BAE argues that the Army unreasonably evaluated Oshkosh's proposal as excellent/low risk under the cab design element of the technical subfactor. BAE raises two primary arguments: (1) Oshkosh's proposed use of materials that are different from those specified in the TDP rendered its proposal technically unacceptable, and (2) the agency failed to reasonably evaluate the risk raised by Oshkosh's problems developing a prototype version of the cab. For the reasons discussed below, we disagree with BAE.

The cab design element under the technical subfactor considered the proposed designs for building the cab to be used on all variants of these trucks. As discussed above, the government owns the TDP for the FMTV overall, while BAE owns the proprietary designs for the cab portion of the trucks. The RFP provided BAE's drawings of the cab, but not the detailed design and production specifications.

In its initial proposal, Oshkosh stated that it would create drawings and manufacturing processes for the cab based on the proprietary BAE designs provided in the RFP. Navistar AR, Supp. Documents, Oshkosh Initial Proposal, Vol. 2, at 2-9. Oshkosh also stated that it was building a prototype cab prior to award to "eliminate[] production schedule risks associated with the cab TDP and tooling." Id.

In its final proposal, Oshkosh stated that it would use non-galvanized steel on the parts of the cab that support the armor welding because, during the development of its prototype cab, the company had experienced difficulties with "off-gassing," a problem that can occur when zinc coating on galvanized steel is welded. Navistar AR, Tab 00H, Oshkosh Final Proposal, Vol. 2, at 2-11o. While Oshkosh acknowledged that non-galvanized steel does not have the same corrosion resistance as galvanized steel, it anticipated that by e-coating all components of the cab it could meet the RFP's corrosion-resistance requirements. The proposal also stated that the company would conduct additional tests to address risks concerning this issue. Id. Moreover, the proposal also acknowledged that its first prototype cab demonstrated that the "available tooling did not support the roof structure adequately which resulted in unacceptable dimensional variations in a few areas." Id. at 2-9c. Oshkosh stated that it would build a second prototype cab to address these issues. Id.

The Army's evaluation of Oshkosh's final proposal noted the change to non-galvanized steel, and stated that the "Government has reviewed these materials and does not foresee any potentially negative impacts," and that "these materials are deemed to be acceptable in the construction of the cab spaceframe." Navistar AR, Tab 00P, Navistar Final Cab Evaluation, at 1.

BAE first argues that Oshkosh's cab design was unacceptable because it did not comply with the TDP, which includes drawings that specify the use of galvanized steel. The Army acknowledges that galvanized steel is a requirement for one of the cab drawings, but argues the RFP permitted offerors to propose alternate designs for the cab.

We think that the RFP permitted offerors to submit an alternative cab design that took exception to the TDP. See RFP sect. C.1.2. The RFP states--albeit in a somewhat circular manner--that contractors are "required to build the vehicles as described on the drawings in the TDP with the exception that the Government will consider alternate designs to the base LTAS cab w/door assembly P/N 12505151." Id. (emphasis added). The provision continued: "Alternate cab designs shall comply with the base LTAS cab as described on drawing 12505151." Id. (emphasis added). Section M of the solicitation states that the agency will evaluate offerors' proposed "designs to the base LTAS cab w/door assembly P/N 12505151," and that the design information would be "evaluated against the cab interface control drawing (ICD), the vehicle TDP, and performance specification requirements." RFP sect. M.9.2.1.

At best, the RFP contained a patent ambiguity concerning whether alternate proposals had to comply with the TDP; given this ambiguity, we think both BAE's and the Army's interpretations of the solicitation are reasonable. Thus, BAE's argument that only its interpretation of the cab design requirement is reasonable is untimely as it was not raised prior to the solicitation's closing time. Bid Protest Regulations, 4 C.F.R. sect. 21.2(a)(1) (2009); See Poly-Pacific Techs., Inc., B-293925.3, May 16, 2005, 2005 CPD para. 100 at 3. Moreover, as discussed below, we think the agency reasonably evaluated Oshkosh's proposed use of an alternative metal.

Despite BAE's arguments to the contrary, we also think that the agency reasonably viewed as a strength Oshkosh's proposal to develop a prototype cab.[12] The RFP did not require offerors to produce prototypes; instead, Oshkosh voluntarily undertook this effort to identify and address risks in advance of contract award. While BAE argues that the agency should have been concerned that Oshkosh needed to build a second prototype to resolve difficulties experienced with its first, the agency viewed the approach of building a prototype as a strength because it allowed Oshkosh to identify risks prior to performance. On this record, we think the agency was within its discretion to view Oshkosh's work on its prototype as a strength, rather than a risk.

Finally, we note that during the hearing, the Army cab design evaluator testified that he was aware of and understood the risks posed by Oshkosh's attempt to address the difficulties arising from welding galvanized steel. See Tr. at 36:20-37:11, 112:9-14, 117:12-118:2, 118:10-14, 120:11-14; see also Navistar AR, Tab 00P, Navistar Final Cab Evaluation, at 1. The evaluator also testified that he was aware of and understood the difficulties identified in Oshkosh's proposal concerning "unacceptable dimensional variations" in the first prototype. Id. at 133:11-134:9. Despite these risks, the evaluator testified that he viewed these problems as relatively minor, that problems like these could be expected when a new contractor was developing designs from BAE's drawings, and that Oshkosh's identification of potential problems through its use of prototypes gave him confidence that Oshkosh would solve these problems. See id. at 118:10-119:5. With regard to the selection decision, the SSA testified that she was generally aware of these issues, but relied on the judgments of the technical evaluators that Oshkosh's proposal merited a low risk rating. See Tr. at 696:6-697:2, 697:12-698:5.

In conclusion, we think the record shows that the agency was aware of the potential risks raised by Oshkosh's proposal in the area of cab design, and reasonably determined that these risks did not pose a significant risk of unsuccessful contract performance. In this regard, we think the agency acted within its discretion when it concluded that the risks were relatively minor, and that Oshkosh's proposed approach reasonably mitigated those risks. We also think the SSA reasonably concluded that there was no meaningful difference under this evaluation element between BAE's proposal, which relied on its current production of the cab, and Oshkosh's proposal, because the agency had confidence that Oshkosh's pre-award efforts served to mitigate the risk that Oshkosh would be unable to perform the contract requirements. We think these judgments were well within the discretion afforded evaluators and the SSA, and on this record, we find no basis to sustain the protest.

Price Realism Evaluation

Navistar and BAE next argue that the Army failed to reasonably evaluate the realism of Oshkosh's proposed price.[13] As discussed above, the RFP stated that the Army would evaluate offeror's proposed prices for realism and reasonableness. RFP sect. M.8.1. The protesters primarily contend that the agency failed to reasonably evaluate the profit, material, and general and administrative (G&A) portions of Oshkosh's fixed price. We disagree.

Agencies are only required to perform a cost realism analysis when the solicitation anticipates the award of a cost-reimbursement contract. In contrast, where an RFP contemplates the award of a fixed-price contract, or a fixed-price portion of a contract, an agency may, as here, provide in the solicitation for the use of a price realism analysis for the limited purpose of measuring an offeror's understanding of the requirements or to assess the risk inherent in an offeror's proposal. Puglia Eng'g of California, Inc., B-297413 et al., Jan. 20, 2006, 2006 CPD para. 33 at 6. Although the FAR does not use the term "price realism," it states that cost realism analysis may be used to evaluate fixed-price proposals for purposes of assessing proposal risk, but not for the purpose of adjusting an offeror's evaluated price. FAR sect. 15.404-1(d)(3).

As our Office has repeatedly held, the depth of an agency's price realism is a matter within the sound exercise of the agency's discretion. Citywide Managing Servs. of Port Washington, Inc., B-281287.12, B-281287.13, Nov. 15, 2000, 2001 CPD para. 6 at 4-5. In reviewing protests challenging price realism evaluations, our focus is whether the agency's review was reasonable and consistent with the terms of the solicitation. Grove Resource Solutions, Inc., B-296228, B-296228.2, July 1, 2005, 2005 CPD para. 133 at 4-5. As a general matter, it is unobjectionable for an offeror to submit a below-cost proposal for a fixed-price contract, since fixed-price contracts generally are not subject to adjustment during performance and the contractor, not the agency, bears the financial risk of cost overruns. Crown Title Corp., B-298426, Sept. 21, 2006, 2006 CPD para. 145 at 5-6.

With regard to Navistar's and BAE's specific challenges, the protesters first argue that the agency did not reasonably evaluate Oshkosh's proposed material costs, in part because Oshkosh's material costs were lower than theirs.

The Army concluded that Oshkosh's material costs were realistic, even though its proposal only provided details regarding approximately 66 percent of its material costs; these costs were supported by vendor quotes, or estimates based on similar parts or historical prices. Navistar AR, Tab 16, Interim Oshkosh Price Evaluation, at 14. The record shows, however, that the price evaluator understood that the limited amount of information was a result of the RFP's requirement, discussed above, that offerors identify in their bill of materials only those direct costs with a unit price in excess of $500 per vehicle; the agency assumed that the remaining material costs were excluded because Oshkosh planned to produce many of its vehicle components in-house. See Tr. at 569:22-570:12; Navistar AR, Supp. Documents, Retrospective Price Realism Report, at 4. Further, the agency assumed that Oshkosh's proposed material costs were lower than those proposed by BAE and Navistar because Oshkosh planned to produce more of its components in‑house, and rely less on subcontractors. Id. We see nothing unreasonable about the agency's evaluation.

Next, BAE argues that Oshkosh's proposal did not provide a breakdown of the material and labor resources required to produce the cab. This level of detail, however, was not required under the solicitation, and the price evaluator testified that the agency understood the cab costs to be included within the general material and labor cost categories in Oshkosh's price proposal. Tr. at 564:9-19, 569:22-570:12; 573:1-17. Again, we see nothing unreasonable about this conclusion.

Finally, the protesters contend that the agency failed to reasonably consider the risks posed by Oshkosh's decision to offer a [deleted] percent G&A rate, particularly in light of the fact that it offered a [deleted] profit rate of [deleted] percent.

The record here shows that the agency addressed this issue during discussions, and asked Oshkosh to explain its decision to [deleted]. Navistar AR, Tab 001, IFD_OCP_001 at 1. In its response, Oshkosh both explained its rationale, and confirmed its pricing. Id. at 2. In addition, Oshkosh and the Army agreed on a modified contract clause [deleted]. Navistar AR, Tab 00L, Oshkosh IFD_OCP_020; Tr. at 533:9-534:6. The agency explained that it relied on these exchanges, as well as its understanding of Oshkosh's current and anticipated business volumes, to conclude that [deleted] G&A costs for the FMTV contract did not render Oshkosh's proposal unrealistic. As discussed above, since the contractor, and not the government, bears the financial risk of any cost overrun, it is unobjectionable for an offeror to submit a below-cost proposal for a fixed-price contract, Crown Title Corp., supra. Accordingly, we find no basis in this record to conclude that the agency's evaluation of this issue was unreasonable.

The protesters argue, and we agree, that the Army's interim and final price evaluation provided limited detail concerning its price realism evaluation. Nonetheless, the agency prepared what it called a "Retrospective Report" concerning the realism of Oshkosh's price proposal during the course of this protest. The agency explained that the retrospective report was intended to summarize and provide additional detail concerning conclusions that were reached during the procurement, and to provide more detail concerning the summary conclusions that Oshkosh's proposed price and the constituent cost elements were realistic. Navistar SAR at 4. We think that the agency's retrospective report and the testimony by the price evaluator are consistent with the contemporaneous record, and that they show that the agency conducted a reasonable price realism analysis. See NWT, Inc.; PharmChem Labs., Inc., supra.

The record here shows that the agency evaluated the offerors' proposed profit and material, subcontracting, warranty, indirect, G&A, labor, and federal tax costs. See, e.g., Navistar AR, Tab 16, Interim Oshkosh Price Evaluation, at 14-15; see also Supp. Documents, Retrospective Price Realism Report, at 2-9. During the hearing, the price evaluator testified that he consulted with the technical evaluators as to whether they considered offerors' proposed costs realistic for the work to performed, and that no concerns were identified. See, e.g., Tr. at 508:4-509:4, 544:20-545:6, 571:17-572:7, 577:17-578:8, 581:15-19.

The SSEB final report noted the differences between the offerors' proposed price for each FMTV vehicle and trailer variant, as compared with the historical cost for those items. Navistar AR, Tab 13, SSEB Final Report, at 142. The agency concluded that there were three "Major Reasons for Price Differences" between the offerors' proposals: (1) Oshkosh's lower costs for its in-house production, and its use of "little subcontracting," compared with the other offerors; (2) Oshkosh's lower profit rate of [deleted] percent, as compared to Navistar's and BAE's higher profit rates of [deleted] percent and [deleted] percent, respectively; and (3) Oshkosh's [deleted] lower G&A rate of [deleted] percent, as compared with Navistar's and BAE's higher G&A of [deleted] to [deleted] percent, and [deleted] percent, respectively. Id. at 92. The SSA concluded that, based on the information she reviewed, the offerors' proposed prices were reasonable and realistic. Navistar AR, Tab 14, SSD, at 6. Based on our review of the contemporaneous record, the retrospective document, and the testimony provided by the Army price evaluator, we think that the agency conducted a reasonable, and properly documented, price evaluation.

Oshkosh's Financial Capability

Finally, Navistar argues that the Army failed to reasonably evaluate Oshkosh's financial capability, as required by the solicitation. In this regard, Navistar argues that the CO's affirmative determination of responsibility for Oshkosh was flawed for two reasons: (1) the CO did not consider two reports prepared by the Defense Contract Audit Agency (DCAA) concerning Oshkosh, and (2) the CO improperly relied on a pre-award survey prepared by the Defense Contract Management Agency (DCMA) and did not adequately scrutinize or question the survey's conclusions. We disagree.

Contracts may only be awarded to responsible prospective contractors. FAR sect. 9.103(a). In making a responsibility determination, the CO must determine, among other things, that the contractor has "adequate financial resources to perform the contract, or the ability to obtain them." FAR sect. 9.104-1(a). We will consider a challenge to a CO's affirmative determination of responsibility only where it is alleged that definitive responsibility criteria in the solicitation were not met, or where the protester identifies evidence raising serious concerns that, in reaching the responsibility determination, the CO unreasonably failed to consider available relevant information or otherwise violated statute or regulation. 4 C.F.R. sect. 21.5(c); T.F. Boyle Transp., Inc., B-310708, B-310708.2, Jan. 29, 2008, 2008 CPD para. 52 at 5.

The RFP stated that, as part of the CO's determination of responsibility, "the Government may assess the offerors financial and management capabilities to meet the solicitation requirements." RFP sect. M.5. The Army states that the CO determined that Oshkosh was a responsible offeror, and as part of that determination concluded that the awardee had the financial capability to perform the contract. Navistar AR at 19; Navistar SAR at 9. The CO states that he relied on a pre-award survey that was prepared at the Army's request by the DCMA, which concluded that, based on its financial condition, Oshkosh was a "MODERATE RISK contractor," but that contract award was nonetheless recommended. Navistar AR at 19; Navistar AR, Tab 43, DCMA Pre-Award Survey, at 5 of 11. The CO did not separately document his determination, but instead, consistent with the FAR, indicated that he viewed Oshkosh as a responsible offeror by signing the contract. See FAR sect. 9.105-2(a)(1) ("The contracting officer's signing of a contract constitutes a determination that the prospective contractor is responsible with respect to that contract.")

Navistar first argues that the CO should have also considered two DCAA reports concerning Oshkosh that were prepared in connection with other procurements. The record shows that these audit reports were provided to the SSEB chair, but were not provided to the CO, and were not considered by him in his responsibility determination.

We note first that neither of these DCAA reports identified any specific concerns with Oshkosh's financial capability. Instead, in both reports DCAA specifically declined to take a position on Oshkosh's financial capability, and recommended a future review. See Navistar AR, Supp. Documents, DCAA Report, June 6, 2008, at 2; DCAA Report, February 5, 2009, at 1. The Army elected to request a review of Oshkosh's financial capability by DMCA, and the record shows that the resulting DCMA report was reviewed and relied upon by the CO in his affirmative determination of responsibility. Navistar AR at 19; Navistar SAR at 9; Tr. at 646:18-647:4. On this record, we find no basis to conclude that the CO unreasonably failed to consider available relevant information.

Navistar also argues that the CO did not adequately scrutinize the DMCA audit report. During the hearing, the CO acknowledged that he did not understand certain conclusions in the DCMA report, and thus did not make a completely independent judgment as to whether DCMA's conclusions about Oshkosh's financial status were accurate. See, e.g., Tr. at 674:3-14. Despite this testimony, we think the CO did not abuse his discretion by accepting the conclusions of the pre-award survey that recommended award, notwithstanding Oshkosh's moderate risk rating, given the DCMA's detailed analysis and specific conclusions. We find no basis here to conclude that any of the exceptions to our jurisdictional bar concerning challenges to a CO's affirmative determination of responsibility applies, and therefore dismiss this aspect of Navistar's argument. [14]

Conclusion and Prejudice

As discussed above, the protesters raised multiple challenges to the Army's evaluation of offerors' proposals under all of the evaluation factors. With the exception of two areas, we find that the agency's evaluation was reasonable. We conclude, however, that the agency's errors with regard to the key tooling and equipment element of the production capability subfactor and the past performance subfactor warrant sustaining the protest because we think, as described below, that the outcome of the competition could have been affected and that the protesters were thereby prejudiced by those errors. See McDonald-Bradley, B-270126, Feb. 8, 1996, 96-1 CPD para. 54 at 3; see also, Statistica, Inc. v. Christopher, 102 F.3d 1577, 1681 (Fed. Cir. 1996).

In essence, the record in this procurement shows that the agency did not conduct a tradeoff analysis because the SSA viewed the proposals as equal with regard to the non-price evaluation factors. Specifically, the SSA found no differences between the offerors' proposals under the capability or the past performance/small business evaluator factors, and concluded that she "must therefore rely on the Total Evaluated Price as the determining Factor in my decision." Navistar AR, Tab 14, SSD, at 6. With regard to the elements of the production capability subfactor, the SSA stated as follows: "While all three offerors have subtle differences in specific details regarding these seven elements, they are all rated Excellent with a Very Low Risk, and therefore, I find no meaningful differences between them as evaluated." Id. at 4.

With regard to the key tooling and equipment element, we think that the solicitation explicitly identified the manner in which the Army would evaluate risk. It is a fundamental principle of government procurement that competitions must be conducted on an equal basis, that is, offerors must be treated equally and be provided with a common basis for the preparation of their proposals. Continental RPVs, B-292768.2, B-292768.3, Dec. 11, 2003, 2004 CPD para. 56 at 8. Contracting officials may not announce in the solicitation that they will use one evaluation scheme and then follow another without informing offerors of the changed plan and providing them an opportunity to submit proposals on that basis. The S.M. Stoller Corp., B-400937 et al., Mar. 25, 2009, 2009 CPD para. 193 at 5-6. Our Office will sustain a protest that an agency improperly waived or relaxed its requirements for the awardee where the protester establishes a reasonable possibility that it was prejudiced by the agency's actions. Datastream Sys., Inc., B-291653, Jan. 24, 2003, 2003 CPD para. 30 at 6.

Key tooling and equipment was one of the three most important elements under production capability, which was the most important subfactor under the capability evaluation factor. Thus, if the agency had properly evaluated Oshkosh's proposal under this element, following the evaluation scheme set forth in the solicitation, the awardee's proposal could have received a lower rating--or, to be more specific, a higher risk rating--and the agency would have been required to conduct a tradeoff between the price and non-price evaluation factors. We therefore conclude that both protesters, Navistar and BAE, were prejudiced by the Army's evaluation.[15]

With regard to the evaluation of Navistar's past performance, we conclude, as discussed above, that there is not enough documentation in the record for our Office to assess the reasonableness of the Army's evaluation. Navistar received one excellent and one adequate rating for its past performance references. Because the record does not provide support for the "adequate" rating for one of the two past performance references, there is a possibility that Navistar's past performance rating could improve from good to excellent. Moreover, because the past performance subfactor was "significantly more important" than the small business subfactor, there is a possibility that Navistar's past performance rating could improve in a way that would increase its rating above the "good" rating received by all three offerors under the past performance/small business participation evaluation factor. Such an improvement, would, again, require the agency to conduct a tradeoff between the price and non-price evaluation factors. We therefore conclude that Navistar was independently prejudiced by the agency's past performance evaluation.

RECOMMENDATION

We recommend that the Army reevaluate these proposals under the key tooling and equipment element in a manner consistent with the terms of the solicitation. The agency should also consider whether its evaluation under the time phased critical path element was affected by its conclusions regarding the key tooling and equipment evaluation.[16] We also recommend that the agency conduct a new evaluation of Navistar's past performance that adequately documents the agency's judgments. Following its reevaluation of offerors' proposals, the agency should make a new selection decision. If Oshkosh is not found to offer the best value to the government, the agency should terminate Oshkosh's contract for the convenience of the government.

Finally, we recommend that Navistar and BAE be reimbursed the costs of filing and pursuing the successful grounds of their protests related to their challenge of technical and past performance evaluation issues, including reasonable attorney fees. 4 C.F.R. sect. 21.8(d)(1). Navistar and BAE should submit their certified claims for costs, detailing the time expended and cost incurred, directly to the contracting agency within 60 days after receipt of this decision. 4 C.F.R. sect. 21.8(f)(1).

The protests are sustained.

Lynn H. Gibson
Acting General Counsel