Matter of: Cessna Aircraft Company File: B-261953.5 Date: February 5, 1996

Matter of: Cessna Aircraft Company File: B-261953.5 Date: February 5, 1996

Protester's contention that agency improperly evaluated proposals is denied where the record shows that the agency evaluated in accordance with the criteria announced in the solicitation, and the record reasonably supports the evaluators' conclusions. Allegation that the agency's source selection advisory council (SSAC) improperly changed the source selection evaluation board's (SSEB) rating of protester's proposal for trainer aircraft under the "system safety" evaluation factor from blue (exceptional) to green (acceptable) is denied, where the record shows that the SSEB's rating was primarily based on the protester's twin-engine design, and, despite a specific request from the SSAC, the SSEB could not convince the SSAC that a twin-engine design resulted in a "safer" aircraft. Contracting agency reasonably evaluated protester's performance risk based upon unfavorable information in "contractor performance assessment reports" and in responses to questionnaires reflecting the protester's recent poor performance on other contracts. Contracting agency's life-cycle cost analysis involves the exercise of informed judgment and our Office will not question such an analysis unless it clearly lacks a reasonable basis. Protester's contention that the agency's cost analysis was flawed is denied where the protester has not shown that the agency's approach was unreasonable. Where solicitation and agency's letter requesting best and final offers (BAFO) clearly required offerors to substantiate any BAFO price reductions, so that the agency could adequately assess and develop most probable life-cycle costs, agency's decision to rely on protester's pre- BAFO pricing was reasonable, where record shows that protester failed to substantiate significant price reductions in its BAFO.

Attorneys

DECISION

The Cessna Aircraft Company protests the proposed award of a contract to the Beech Aircraft Corporation under request for proposals (RFP) No. F33657-94-R-0006, issued by the Department of the Air Force for a joint primary aircraft training system (JPATS). Cessna contends that the Air Force improperly evaluated technical proposals and that the agency's life-cycle cost evaluation was flawed.

We deny the protest.

BACKGROUND

The objective of the JPATS procurement is to replace the Air Force's T-37B and the Navy's T-34C aircraft and associated ground-based training systems (GBTS). The primary mission of the JPATS aircraft and its GBTS is to train entry-level Air Force and Navy student pilots in primary flying and to prepare them to transition into advanced training tracks leading to qualification as a military pilot. The JPATS aircraft and GBTS will also provide entry-level officers with a basic understanding of airmanship prior to their designation as Naval Flight Officers or Air Force Navigators, as well as provide support and training for pilot instructors.

The RFP characterized the acquisition as a "fly-before-buy" procurement. That is, the RFP stated that as part of the evaluation process, the government would fly each offeror's proposed aircraft to assess its performance and flying qualities. Thus, in addition to written proposals, offerors were required to provide an evaluation aircraft with flying qualities which duplicated those of the proposed production aircraft.

The RFP stated that the flight evaluation would assess the capability of each aircraft to train an entry-level student pilot with no prior flying experience to the proficiency level required by the primary pilot training syllabus. Air Force and Navy test pilots and customer pilots would evaluate each aircraft. In addition, the Air Force's Operational Test and Evaluation Center was to perform an early operational assessment of each aircraft. Offerors were also required to provide a full-scale mock-up of the cockpit of the proposed production aircraft, which was to be evaluated for crew member accommodation characteristics, including anthropometric range capability, lighting checks, and other engineering components.

The RFP, issued on May 18, 1994, contemplated the award of two contracts to one offeror--a fixed-price incentive contract with award fee and economic price adjustments for the manufacturing development of a primary training aircraft, with options for varying quantities of production aircraft; and a fixed-price contract for a base period with up to nine 1-year options for logistics support.

Section M of the RFP stated that in evaluating proposals, the agency would consider three types of criteria: (1) areas (related to important program characteristics); (2) assessment criteria (related to an offeror's proposal and ability to perform); and (3) cost/price and schedule. The RFP listed the following evaluation areas in descending order of importance (factors within each area are shown in parentheses): operational utility/technical (operational capability; crew accommodations; structural integrity; certification/qualification; aircraft missionization; and system safety); manufacturing and quality assurance (manufacturing; production control; and quality assurance); cost/price; logistics support (acquisition logistics; partial contractor logistics support (CLS); and total CLS for the Air Force and Navy); [1] management (aircraft management; and GBTS support and management); and schedule. The RFP stated that the manufacturing and quality assurance area was slightly more important than the cost/price area. [2]

The RFP further stated that, except for the cost/price and schedule area, the agency would apply three "assessment criteria" of equal importance (soundness of approach, understanding the requirements, and compliance with requirements) to the evaluation areas. The RFP also identified "general considerations" (including, but not limited to, proposed contractual terms and conditions and the results of the early operational assessment of each offeror's evaluation aircraft). The agency would also assess proposal and performance risk. The RFP stated that the integrated assessment of the results of the evaluation would favor proposals that offered the best value to the government considering development risk and total system life-cycle cost. The contracts were to be awarded to the offeror whose proposal was considered to be most advantageous to the government.

Seven offerors responded to the RFP by the time set on July 18, for receipt of initial proposals. Air Force and Navy pilots conducted flight evaluations and a source selection evaluation board (SSEB) evaluated written proposals. A performance review assessment group evaluated the offerors' past performance and a separate panel evaluated cost/price. Based on the results of those evaluations, the agency eliminated one offeror's proposal from further consideration, and initiated discussions with the remaining six offerors. Following initial discussions, the agency amended the RFP; requested and evaluated revised proposals; held further discussions; and requested best and final offers (BAFO).

The SSEB evaluated BAFOs by assigning color/adjectival and risk ratings [3] to each factor announced in the RFP. With the exception of "crew accommodations" and "production control," where Beech's proposal was rated "yellow" and "blue" respectively, [4] Beech's proposal was rated "green" under all other evaluation factors, with low proposal and performance risk.

Cessna's proposal [5] received final ratings of "yellow" with "moderate" proposal risk and "low" performance risk under the "operational capability" factor, and "blue" with low proposal and performance risk under "crew accommodations." Under all other evaluation factors, Cessna's proposal was rated "green." With the exception of "structural integrity," "manufacturing," and "production control," where Cessna's offer received proposal risk ratings of either "moderate" or "low-moderate," Cessna's proposal received proposal risk ratings of "low" under all other evaluation factors. As for performance risk, except for a "moderate" rating under "aircraft missionization," Cessna's proposal received a rating of "low" risk under all other factors. The agency developed a most probable life-cycle cost (MPLCC) of $14.597 billion for Beech and [DELETED] billion for Cessna. [6] The evaluation results for Beech and Cessna were as follows: [DELETED]

A source selection advisory council (SSAC) reviewed the results of the SSEB's final evaluation and presented its findings to the source selection authority (SSA). [7] Based on the SSAC's report, the SSA selected Beech for award. This protest followed. [8]

PROTESTER'S CONTENTIONS

Cessna challenges several aspects of the evaluation of Beech's and Cessna's technical proposals, including proposal and performance and risk ratings. Specifically, Cessna challenges the evaluation of proposals under the "system safety" and "operational capability" factors. [9] Cessna also challenges the evaluation of Beech's proposal under the "crew accommodations" factor and maintains that the agency erred in assessing Beech's and Cessna's performance risk. With respect to evaluated cost, Cessna challenges virtually every aspect of the agency's life-cycle cost evaluation.

EVALUATION OF PROPOSALS

System Safety

The SSEB rated Cessna's proposal "blue" with low proposal and performance risk under this factor, based primarily on Cessna's twin-engine design. During 2 days of briefings, however, the SSAC members discussed at length whether Cessna's proposal warranted that rating, especially since it was based primarily on the proposed twin-engine design. In view of recent technological and engineering advances, the SSAC members were particularly concerned with the relevance of certain information, such as historical accident rate data of single versus twin-engine aircraft, which formed the basis of the SSEB's rating. The SSAC's concerns reflect what apparently is a long-standing debate in the industry surrounding the issue of whether an aircraft is inherently "safer" (and therefore deserving of a higher rating under the "system safety" factor) by virtue of having more than one engine.

Following those initial discussions, the SSAC requested that the SSEB submit additional information in support of Cessna's "blue" rating under this factor. In response, the SSEB presented the SSAC a chart comparing "mishaps rates" between different single-engine and twin-engine aircraft. Based on its review of that additional information, and following further discussions, the SSAC concluded that the SSEB had not shown that two engines necessarily result in a safer aircraft. Not convinced by the SSEB's presentation, the SSAC assigned Cessna a "green" rating under "system safety."

Cessna contends that the SSAC's decision to change the SSEB's rating under this factor was unreasonable. In support of its position, Cessna points to the "mishaps rates" data shown to the SSAC comparing twin-versus single-engine aircraft. According to Cessna, the data conclusively show that a twin-engine design is "safer" than a single-engine aircraft, and thus its proposal deserved the "blue" rating initially assigned under this factor.

Source selection officials in negotiated procurements are not bound by the recommendations or evaluation judgments of evaluators. Grey Advertising, Inc., 55 Comp.Gen. 1111 (1976), 76-1 CPD para. 325. Thus, source selection officials have broad discretion in determining the manner and extent to which they will make use of the technical and cost evaluation results, and their technical judgments are governed only by the tests of rationality and consistency with the stated evaluation criteria. Id.; Oklahoma Aerotronics, Inc.--Recon., B-237705.2, Mar. 28, 1990, 90-1 CPD para. 337. Source selection officials may properly question the conclusions reached by a technical evaluation panel, and, where those conclusions are not convincing or lack support, we see nothing improper with selection authorities requesting additional information from the evaluators in support of their ratings. See, e.g., Wyle Labs., Inc.; Latecoere Int'l., Inc., 69 Comp.Gen. 648 (1990), 90-2 CPD para. 107; Calspan Corp., B-255268, Feb. 22, 1994, 94-1 CPD para. 136; Latecoere Int'l., Inc.--Advisory Opinion, B-239113.3, Jan. 15, 1992, 92-1 CPD para. 70. In reviewing a protest challenging an agency's technical evaluation, we examine the record to ensure that the agency's evaluation was reasonable and consistent with the stated evaluation criteria. See Abt Assocs., Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD para. 223. Based on our review of the record, we find no basis to question the SSAC's decision to assign Cessna's proposal a "green" rating under "system safety."

The SSAC discussed this issue at length and concluded that although Cessna's aircraft offered several strengths as identified by the SSEB related to the twin-engine design, those strengths did not support a "blue" (exceptional) rating. In this connection, the agency [10] points out that there are numerous factors which have far greater effects on accident rates, and hence system safety, than the number of engines. For instance, the agency points out that a twin-engine aircraft that displays unsuitable flying characteristics to train entry-level student pilots with no previous flying experience may be less safe than a single-engine trainer aircraft without unsuitable characteristics. The Air Force explains that a twin-engine aircraft with objectionable [DELETED] such as Cessna's, for example, which makes it difficult or impossible for a student to [DELETED]. In another example cited by the Air Force, a twin-engine aircraft such as Cessna's, which is difficult to [DELETED].

The record shows that the SSAC debated the issue of the relative safety of twin-versus single-engine aircraft, and that the SSAC's discussions took into account the dynamics of various relevant factors and characteristics of Cessna's aircraft, which interact to affect system safety. The SSAC considered in its deliberations the strengths identified by the SSEB, based on Cessna's twin-engine design, and agreed with the SSEB's conclusion that some strengths in Cessna's proposal were attributable, in large part, to Cessna's twin-engine design. The SSAC was not convinced, however, that those strengths warranted rating Cessna's aircraft "exceptional" with respect to safety, and requested the SSEB to further support its recommendation.

Despite the specific request from the SSAC for support for the "blue" rating, the SSEB did not convincingly show the SSAC that Cessna's twin-engine design had any inherent safety advantages over a single-engine aircraft to warrant a rating of "exceptional" under this factor. The SSAC's reason for changing the SSEB's initial rating was based on its extensive discussions, consideration of the SSEB's additional information, and the technical expertise of its members and their familiarity with the issues surrounding this debate. Based on our review of the record, including the agency's explanation in response to the protester's allegation, we think that the SSAC reasonably could conclude that the protester's twin-engine design was not inherently safer than a single- engine aircraft, and that Cessna's proposal did not warrant a "blue" rating under this factor. The protester's contention, that the SSAC's decision to change the SSEB's initial rating under this factor was arbitrary, is not supported by the record. While Cessna disagrees with the SSAC's conclusions, that disagreement does not make the SSAC's final rating unreasonable. Calspan Corp., B-258441, Jan. 19, 1995, 95-1 CPD para. 28.

The protester also contends that the agency's rating of Beech's proposal under this factor was unreasonable because Beech's proposal failed to comply with an RFP provision requiring that the proposed aircraft be fail-safe, "such that no single point failure, or combination of failures with a failure rate greater than 1 X 10-7 per flight can cause critical or catastrophic mishap. . . ." In this connection, offerors were required to submit a preliminary analysis for all hazards with failure rate probabilities greater than 1 X 10-7. The agency states that the purpose of that analysis was to identify safety-critical areas, provide an initial evaluation of any hazards, and identify follow-up corrective actions to be taken to either eliminate, control, or otherwise mitigate the hazards to acceptable levels. In accordance with this RFP provision, Beech submitted a preliminary hazards analysis and proposed corrections.

The record shows that the agency evaluated Beech's analysis, including proposed corrective actions, and Beech's proposed system safety program to eliminate or mitigate any hazards in its aircraft. The evaluators concluded that the scope and depth of Beech's preliminary analysis adequately addressed all potential safety and environmental risks, and that Beech's proposed safety program would mitigate identified hazards to acceptable levels. Based on our review of the record, we have no basis to question the agency's conclusions in this regard. [11]

Operational Capability

A team of Air Force and Navy pilots and engineers evaluated Cessna's aircraft as part of the flight evaluation under this factor. [12] The flight evaluation team flew 13 "sorties" of the Cessna aircraft totaling almost 27 flight hours. The team identified several deficiencies with the aircraft, and issued clarification requests (CR) and deficiency reports (DR) to Cessna. The three most significant items that remained unresolved following discussions are briefly described below.

Two DRs and a CR concerned deficiencies with flying qualities of Cessna's aircraft. [DELETED] informed Cessna that the evaluators had found it difficult to [DELETED]. That DR also stated that the [DELETED].

In [DELETED], the agency informed Cessna that [DELETED]. The DR also informed Cessna of a problem resulting from [DELETED].

In its responses to these items, Cessna proposed various changes either to the aircraft's external structural configuration or flying speeds. According to Cessna, [DELETED].

After receipt of responses to the discussion items, the Air Force issued amendment No. 0002 to the RFP. As relevant to this aspect of the protest, the contracting officer's cover letter to the amendment stated in part:

"Amendment 0002 provides a number of changes to the solicitation. Offerors are reminded, however, that additional aircraft [f]light [e]valuations will not be conducted. Proposed changes by [o]fferors that may affect the flying qualities of their proposed production aircraft (e.g., responses to [DRs], responses to solicitation amendments, [BAFOs] etc.) will not change the results of the [g]overnment's [f]light [e]valuation. However, any proposed changes that may affect flying qualities may affect the [g]overnment's integrated evaluation. The [g]overnment will assess the potential of proposed changes to either enhance or degrade aircraft performance or flying qualities. . . ."

Subsequently, the agency issued amendment No. 0003. Although the protester submitted a revised proposal in response to these amendments, Cessna did not revise the modifications to the aircraft it had proposed earlier. The agency evaluated Cessna's responses and concluded with respect to [DELETED] that while the probability was high that Cessna's proposed change would correct the [DELETED] deficiency, the effects of the [DELETED] on other flying qualities of the aircraft could not be assessed without another flight evaluation. [13]

With respect to DR [DELETED] the evaluators concluded that Cessna's proposal to [DELETED]. With respect to the CR, the evaluators concluded that Cessna's limited [DELETED] did not allow for an adequate evaluation of the aircraft's flying qualities [DELETED] which is essential to maintaining safe primary student operations. Based on the evaluators' conclusions, Cessna's proposal received a final rating of "yellow" with "moderate" proposal risk under the "operational capability" factor. The protester argues that the Air Force improperly evaluated its proposed changes in response to these deficiencies. [14]

The record shows that the SSEB reviewed Cessna's proposed changes and concluded that the impact of the solutions either could not be fully assessed without another flight evaluation, or that the changes significantly affected the flying qualities of the aircraft. Regarding DR [DELETED], the agency explains that because the aircraft [DELETED]. The evaluators found that Cessna's solution [DELETED] significantly affected some of the flying qualities and aerodynamic performance of the aircraft demonstrated during the initial flight evaluation. The evaluators concluded that, while the proposed [DELETED] changes "may" correct the deficiency, the aircraft would have to be reflown to fully assess the effect of the [DELETED] on other flying qualities of the aircraft.

With respect to DR [DELETED] the evaluators found that while Cessna had committed to performing tasks after award that would improve the [DELETED], Cessna's proposed changes would affect some of the flying qualities and performance characteristics demonstrated and assessed during the flight evaluation. For example, as a solution to this deficiency, Cessna proposed [DELETED]. The evaluators noted, however, that the aircraft was flight-tested based on [DELETED], and that the evaluation pilots noted deficiencies due to the [DELETED]. In the evaluators' opinion, [DELETED], as Cessna proposed, could magnify those deficiencies. [15] Since Cessna's aircraft had not been evaluated at the proposed [DELETED], the evaluators concluded that the aircraft would have to be reflown to fully assess the proposed solutions and their impact on other flying qualities of the aircraft.

The evaluators noted that although Cessna may have flight-tested its proposed solutions to the deficiencies, Cessna subsequently stated during oral discussions that the data provided in its responses to the CR and DRs were based on qualitative or analytical predictions, not on data collected from an instrumented aircraft. [16] Cessna did not present any new quantitative data with its proposed solutions. According to the evaluators, even though the protester's proposed changes affect only a portion of the flight envelope, those changes were of such significance--[DELETED]--that in order to verify whether the proposed solutions corrected the deficiencies, the aircraft would have to be reflown. As explained in the contracting officer's cover letter to amendment No. 0002, quoted above, additional flight tests would not be conducted.

The protester argues that the Air Force failed to acknowledge that the proposed solutions to the DRs and the CR corrected the respective deficiencies, even without additional flight testing. This argument is without merit. The record shows that the evaluators carefully considered Cessna's proposed solutions and concluded that some of the problems identified during discussions remained unresolved. Specifically, the evaluators concluded that the problems related to the [DELETED] remained unresolved. The evaluators also concluded that given the nature of the proposed solutions, in order to verify whether the deficiencies were corrected, Cessna's aircraft would have to be reflown--an additional step in the evaluation process that offerors were warned was not contemplated by the RFP, and would not be undertaken to verify the effectiveness or impact of the proposed solutions. The fact that the initial rating of "yellow" assigned to Cessna's proposal under this factor remained unchanged following discussions does not compel a conclusion that the SSEB failed to recognize that Cessna's proposed solutions corrected the noted deficiencies. [17] Rather, the final rating under this factor reflects the evaluators' reasonable conclusions based on their assessment of Cessna's responses to the DRs and the CR that certain deficiencies remained "open" or unresolved.

Unequal Evaluation

The protester also argues that the Air Force evaluated Cessna's and Beech's corrective actions in response to the discussion items in a disparate manner. For instance, Cessna asserts that Beech's initial rating under the "operational capability" factor improved from "yellow" with "moderate" performance risk to "green" with "low" performance risk following discussions based on Beech's allegedly inadequate responses, while the Air Force did not raise Cessna's rating under this factor even though the protester provided adequate solutions.

Beech's initial proposal was downgraded under the "operational capability" factor primarily because the flight evaluation team was unable to assess the aircraft's [DELETED]. [DELETED]. The agency issued a DR to Beech identifying this deficiency.

In response to the [DELETED] DR, Beech submitted additional data which showed that its aircraft was [DELETED]. Specifically, Beech included a detailed table [DELETED]. The record shows that the agency analyzed Beech's response and concluded that the additional data satisfied the evaluators' concerns in this regard. The evaluators further concluded that Beech's response did not affect any of the previously rated flying qualities. In short, the record shows that the Air Force analyzed Beech's supplemental flight test data submissions addressing this particular deficiency and found that the data satisfied the evaluators' concerns. [18] Accordingly, we think that the improved rating assigned to Beech's proposal under the "operational capability" factor reasonably reflects the SSEB's conclusion that the additional data Beech had provided satisfied the evaluators' initial concerns.

The fact that Cessna's rating under the "operational capability" factor did not improve following discussions does not show that the agency treated offerors differently, or that the agency's evaluation was unreasonable. Rather, the final ratings reflect the fact that several of Cessna's responses did not reasonably convince the evaluators that the proposed fixes would not place at risk other flying qualities of the aircraft. [19]

Crew Accommodations

Section M of the RFP stated as follows:

"The evaluation of this factor will be based on an assessment of the offeror's [cockpit] mock-up. . . . The anthropometric range accommodated by each aircraft will be determined on the mock-up using multivariate anthropometric measurement techniques. The [g]overnment's goal is to open flying careers to the widest possible range of qualified applicants . . . . Therefore, the JPATS production aircraft shall accommodate not less than 80 [percent] of the population of eligible women (Cases [20] 1-6 contained in the "Systems Requirement Document"), and it is highly desired that the JPATS production aircraft accommodate equal percentages of the eligible population of men and women commensurate with appropriate flight control forces and safe ejection capabilities . . . . "

Cessna argues that the Air Force improperly evaluated Beech's proposal under this factor because [DELETED]. The protester also contends that Beech failed to provide a functional ejection seat.

Following the initial evaluation, the SSEB concluded that [DELETED]. As a result of these weaknesses, Beech's proposal received a rating of "yellow" (marginal) with moderate proposal risk under this factor. The agency issued a DR to Beech pointing out these deficiencies.

In its response to the DR, Beech explained that based on its own assessment, the Air Force's measurements were not accurate [DELETED].

The Air Force remeasured Beech's cockpit and concluded that all of the weaknesses and risks identified at the initial evaluation with respect to [DELETED] had been resolved. In its final factor summary, the SSEB listed each of the weaknesses and risks noted at the initial measurement of Beech's cockpit and explained how each was resolved to its satisfaction following discussions. For example, the SSEB noted that the problem with respect to [DELETED]. The evaluators specifically concluded that "[Beech's] cockpit will accommodate the desired population range (cases 1-7) to fly the aircraft (97 [percent] of eligible female pilot candidates which is essentially equal accommodation of both males and females) as measured in the cockpit mock-up."

Beech's final rating under this factor did not improve from the initial evaluation, however, primarily because of weaknesses and risks related to Beech's ejection seat. In this connection, the evaluators found that Beech's proposed seat design [DELETED]. Beech was made aware of these problems throughout discussions, and the SSEB concluded that the problems with the seat essentially were weaknesses that could be overcome with close government monitoring.

Cessna's argument that Beech did not provide a functional ejection seat is not supported by the record. What the record shows is that the evaluators considered Beech's ejection seat design to present some weaknesses and proposal risks, and downgraded Beech's proposal accordingly. As a result of these remaining weaknesses and risks, Beech's initial rating of "yelllow" (marginal) with "moderate" risk remained unchanged following remeasurement of the cockpit mock-up. [21] Although Cessna argues that Beech's proposal deserved the lower "red" (unacceptable) rating under this factor, given the SSEB's conclusions--that the noted problems could be easily resolved--a rating of unacceptable was not warranted. In our view, the SSEB's rating of Beech's proposal under this factor was reasonably based.

EARLY OPERATIONAL ASSESSMENT

A Department of Defense Acquisition Board conducted a separate review of the JPATS procurement concurrent with the JPATS source evaluation and selection process. The agency states that this separate review is the process by which the Secretary of Defense authorizes the military services to initiate and continue major acquisition programs such as the JPATS procurement. As part of this review process, the Air Force was required to submit an early operational assessment (EOA) to the Secretary of Defense. The EOA at issue here was conducted by the Air Force's Operational Test and Evaluation Center (AFOTEC).

In addition to submitting the results of the AFOTEC's EOA to the Secretary of Defense, the Air Force decided that the SSA should also consider the results of the EOA in her source selection decision. Accordingly, section M of the initial RFP stated as follows:

"The [g]overnment will conduct an [EOA]. This [EOA] is independent of the Operation[al] Utility/Technical evaluation and will be conducted concurrently with the Flight Evaluation. The results of the [EOA] of each aircraft will be presented to the [SSA] to be considered as part of the source selection decision."

Subsequently, the agency amended this provision as follows:

"The [g]overnment will conduct an independent [EOA]. In conducting this assessment all relevant proposal information concerning technical suitability and performance characteristics submitted by the offerors, generated during the source selection process, and received through discussions with offerors may be considered. The results of the [EOA] of each aircraft will be presented through the [SSAC] to the [SSA] to be considered as part of the source selection decision."

Cessna argues that the Air Force improperly compromised the role of the AFOTEC as an independent evaluation entity. The protester relies on Department of Defense Directive 5000.1 (Feb. 23, 1991) to argue that the AFOTEC was designed to be an independent evaluation activity, separate from the procuring agency, as established in the initial RFP. Cessna maintains that the Air Force improperly altered the role of the AFOTEC in this procurement; prevented the AFOTEC from presenting its findings directly to the SSA; and improperly required the AFOTEC to change its initial conclusions. As explained in greater detail below, we conclude that Cessna's allegations regarding the role of the AFOTEC in this procurement are untimely.

Our Bid Protest Regulations require that protests based upon alleged improprieties in a solicitation which are apparent prior to the closing time for receipt of proposals must be filed prior to the closing time. 4 C.F.R. Sec. 21.2(a)(1) (1995). This rule includes challenges to alleged improprieties which did not exist in the initial solicitation but which are subsequently incorporated into the solicitation. In such cases, the solicitation must be protested not later than the next closing time for receipt of proposals following the incorporation. NASCO Aircraft Brake, Inc., B-237860, Mar. 26, 1990, 90-1 CPD para. 330.

Here, the amendment clearly announced two important changes to the manner in which the AFOTEC would conduct the EOA, which are central to the protester's complaint. First, the amendment announced that the results would be presented through the SSAC, rather than directly to the SSA. The amendment also informed offerors that in conducting the EOA, the AFOTEC would consider all relevant information submitted by the offerors, including proposed corrective actions generated during discussions. Thus, since the RFP, as amended, announced that the AFOTEC could consider additional information including proposal modifications, and the results of discussions, Cessna either knew or should have known that the AFOTEC's final EOA could differ from its initial assessment, especially if the weaknesses and deficiencies initially noted were corrected following discussions. If Cessna believed that the agency's approach as announced in the amendment "compromised" the role of the AFOTEC or its independence, or that the AFOTEC should not have reevaluated proposals based on new information provided during discussions, for instance, Cessna was required to raise these concerns prior to the time on March 10 established by the amendment for receipt of revised proposals. Since Cessna did not do so, this aspect of the protest is untimely and will not be considered. [22] See Stanford Telecomm., Inc., B-258622, Feb. 7, 1995, 95-1 CPD para. 50.

PAST PERFORMANCE EVALUATIONS

Cessna argues that the Air Force erred in assessing Beech's and Cessna's performance risk. With respect to Beech, the protester argues that the agency failed to consider a recent guilty plea Beech entered in connection with disclosure violations related to a contract for the sale of aircraft to a foreign government. With respect to its own proposal, the protester argues that the agency failed to update Cessna's past performance record to take into account a recent decision by the Armed Services Board of Contract Appeals (ASBCA) favorable to Cessna.

We will review an evaluation of an offeror's performance risk to ensure that it was reasonable and consistent with the stated evaluation criteria, since the relative merit of competing proposals is primarily a matter of agency discretion. See Dragon Servs., Inc., B-255354, Feb. 25, 1994, 94-1 CPD para. 151. An agency's evaluation of past performance may be based on its reasonable perception of inadequate prior performance, even where the contractor disputes the agency's interpretation of the facts. See Pannesma Co. Ltd., B-251688, Apr. 19, 1993, 93-1 CPD para. 333. Here, we have reviewed the record in light of the protester's arguments and find that it reasonably supports the Air Force's assessment of the protester's and Beech's past performance.

Performance Evaluation: Beech

Cessna argues that the Air Force erred in assessing Beech's past performance because the agency ignored Beech's recent guilty plea arising from a foreign military sales (FMS) contract with the government of Egypt. That plea stemmed from Beech's failure to certify to the government a subcontractor's payment of a contingent fee to a foreign company. [23] The protester maintains that had the Air Force properly considered the plea agreement in its evaluation, Beech's performance risk scores would have been downgraded under several evaluation factors. Cessna also contends that Beech misled the Air Force in its certification.

The RFP contained the clause entitled "Certification Regarding Debarment, Suspension, Proposed Debarment, and other Responsibility Matters," Federal Acquisition Regulation (FAR) section 52.209-5. [24] In pertinent part, that clause states as follows:

"(a)(1) The Offeror certifies, to the best of its knowledge and belief, that--

"(i) The Offeror and/or its Principals--

"(B) Have [ ] have not [ ], within a [3] year period preceding this offer, been convicted of or had a civil judgment rendered against them for: commission of fraud or a criminal offense in connection with obtaining, attempting to obtain, or performing a public (Federal, state, or local) contract or subcontract; violation of Federal or state antitrust statutes relating to the submission of offers; or commission of embezzlement, theft, forgery, bribery, falsification or destruction of records, making false statements, or receiving stolen property. . . ."

The record shows that Beech properly completed the certification in the affirmative and that Beech also provided a brief explanation of the circumstances surrounding the plea agreement. The contracting officer referred the certification to the Air Force's Office of General Counsel, Contractor Responsibility Office (CRO). The CRO reviewed Beech's certification and concluded that "[t]here are no issues of contractor integrity or responsibility that would affect the JPATS procurement process." The record further shows that the SSA was fully briefed with respect to Beech's (and several other offerors') certification, and the conclusions reached by the CRO concerning Beech's eligibility for award.

Beech specifically disclosed the plea agreement in its proposal and briefly explained the circumstances leading to the plea; nothing on the face of the information Beech submitted to the agency called into question the correctness or sufficiency of the certification. Thus, the record shows that the SSA had before her sufficient information concerning Beech's certification to permit her to incorporate that information and the CRO's conclusions in her integrated assessment of the merit of proposals, and ultimately to consider that information in her selection decision.

To the extent that Cessna alleges that Beech lacks business integrity, that is a challenge to an affirmative determination of responsibility which our Office will not review in the absence of a showing of possible fraud or bad faith on the part of contracting officials or a failure to apply definitive responsibility criteria. John C. Holland Enters., B-216250, Sept. 24, 1984, 84-2 CPD para. 336. Neither exception applies here. Further, contrary to Cessna's suggestion, the information Beech submitted concerning the plea agreement did not render the firm ineligible for award. See, e.g., Pearl Properties; DNL Properties, Inc., B-253614.6; B-253614.7, May 23, 1994, 94-1 CPD para. 357; Universal Technologies, Inc.; Spacecraft, Inc., B-248808.2 et al., Sept. 28, 1992, 92-2 CPD para. 212.

Performance Evaluation: Cessna

The performance risk assessment group (PRAG) gave Cessna's proposal a performance risk rating of "moderate" under the "aircraft missionization" factor. That rating was based primarily on unfavorable performance reports obtained from the Navy showing that Cessna had experienced several problems during its recent performance of a Navy contract, referred to as the Undergraduate Naval Flight Officer (UNFO) services contract.

The agency states that although nominally a services contract, the UNFO contract required some engineering development of off-the-shelf air-to-air ground radar equipment. In its response to the Air Force's past performance questionnaire, the Navy reported that Cessna had experienced problems in meeting [DELETED] requirements and that contract specifications were changed so as to permit Cessna to meet those requirements. The Navy also indicated that Cessna had difficulties in acquiring appropriate [DELETED] and had experienced problems in the areas of [DELETED]. The information obtained from the Navy showed that Cessna had been rated [DELETED] in most areas under the UNFO contract. Based on its review of that information, the PRAG rated Cessna's performance risk under "aircraft missionization" as "moderate." Cessna contends that the agency erred in this regard in view of the fact that the ASBCA [25] found that the performance problems and cost overruns reported under the UNFO contract were the result of the Navy's actions.

The protester argues that the evaluators had a duty to use the most current information available, and, thus, should have been aware of the ASBCA decisions. We are aware of no requirement, however (and the protester does not cite to any) that a contracting agency search for information that contradicts or mitigates accurate, but unfavorable, past performance information. Further, there is nothing to suggest that the reports the Navy provided to the Air Force concerning Cessna's performance problems under the UNFO contract were inaccurate or incomplete. [26]

In addition, the record shows that in two separate CRs issued to Cessna, the Air Force specifically raised the performance problems the Navy reported on the UNFO contract. [27] In its response, Cessna informed the Air Force that "Cessna's claims to recover the Navy responsible costs [under the UNFO contract] are currently in litigation." The record shows, however, that the first ASBCA decision (No. 37726) on Cessna's UNFO contract claim had been issued 11 days prior to Cessna's response to the CRs; the second (No. 48118) was issued several weeks before oral discussions and nearly 3 months before BAFOs were due. We think that it was incumbent upon Cessna to update its responses to the CRs concerning the UNFO contract, since, as the protester contends, the ASBCA decisions were favorable to Cessna, and the ASBCA decisions had been issued during the course of the evaluation process. Although the protester was afforded several opportunities to do so, Cessna failed to inform the agency of the ASBCA decisions in its favor. Accordingly, we have no basis to disagree with the PRAG's assessment that based on the Navy's unfavorable reports, Cessna's proposal posed a "moderate" performance risk under "aircraft missionization."

MOST PROBABLE LIFE-CYCLE COST

Cessna takes issue with virtually every aspect of the Air Force's cost evaluation. For example, Cessna contends that the Air Force misevaluated the costs associated with the number of times aircraft would be "stripped and repainted" and the frequency of required engine maintenance and overhauls. Cessna also argues that the agency improperly disregarded its BAFO price reductions. The protester also contends that the agency failed to take into account certain costs (such as different attrition rates); improperly adjusted labor costs for Cessna; and improperly included various other costs (such as the costs of inventory) in the evaluation. In connection with these allegations, Cessna has presented its own analyses allegedly showing that either the assumptions underlying each of the agency's calculations are flawed, or that the agency applied incorrect figures. According to the protester, had the agency conducted a proper evaluation, it should have concluded that both of Cessna's proposals (i.e., the basic proposal and its "commercial" alternate) were lower in cost/price than Beech's proposal. Specifically, according to Cessna's calculations, the agency should have found that its basic proposal was approximately [DELETED] lower in cost than Beech's proposal. [28]

In conducting a life-cycle cost analysis, procurement officials must make informed judgments as to the extent to which proposed prices represent a reasonable estimation of future costs. Dynamic Energy Corp., B-235761, Oct. 6, 1989, 89-2 CPD para. 325. Such informed judgments are properly within the administrative discretion of the procuring agency, and its judgment is entitled to great weight since the agency is in the best position to assess the impact of various factors on future costs and must bear the impact of any difficulties or expenses that may result from a flawed cost analysis. See Dynamic Energy Corp., supra. Our review of the agency's cost analysis is limited to a determination of whether the evaluation was reasonable and consistent with the terms of the RFP. See Corporate Air Servs. Inc., B-215053, Oct. 18, 1984, 84-2 CPD para. 417.

In response to Cessna's numerous allegations, the agency has provided a detailed explanation of its calculations, including the assumptions underlying the cost panel's calculations in support of the MPLCC developed for Cessna, and the net impact of applying the protester's approach to the calculations. The agency explains that, for the most part, even if it were to accept the protester's figures and assumptions, the impact on Cessna's MPLCC would be insignificant given the disparity of more than [DELETED] billion between Cessna's and Beech's MPLCC, and would not have affected the source selection decision.

We have reviewed the record in light of each of the protester's allegations, and agree with the agency that the protester's contentions lack merit. To illustrate our conclusion, we briefly discuss two aspects of the agency's analysis with which the protester finds fault: (1) "stripping and repainting" the aircraft; and (2) adjustments to Cessna's proposed BAFO reductions.

With respect to "stripping and repainting," Cessna asserts that the agency should have used a different number of aircraft as a basis to calculate the costs of these events. The protester maintains that the Air Force based the estimated number of these events on the total anticipated fleet of aircraft (711), rather than the number of "primary aircraft authorized" (575). The agency explains, however, that stripping and repainting, which is necessary to prevent corrosion and maintain uniform physical appearance, is a cost associated with the entire fleet, and not just with the 575 primary aircraft. That is, the government intends to use all 711 aircraft contemplated under the contract in order to maintain a primary operational fleet of 575 aircraft. Thus, since all aircraft in the fleet would eventually undergo "stripping and repainting," we think that the agency reasonably used as a basis the total number of aircraft in the fleet.

With respect to adjustments to Cessna's BAFO, the protester alleges that the agency unreasonably determined that Cessna had failed to adequately support its BAFO cost reductions. Specifically, Cessna alleges that the agency arbitrarily determined that its reductions in [DELETED] for instance, were not adequately supported. According to the protester's calculations these errors resulted in overestimating its MPLCC by nearly [DELETED].

The RFP placed the burden of adequately documenting BAFO cost or price reductions on the offerors. In this connection, several RFP provisions reminded offerors of the requirement to present "adequate supporting documentation to ensure that the cost/price proposal" was understood by the cost panel. The RFP also reminded offerors to provide a description of the scope, limitations, and qualifications of the acquisition and CLS cost proposals, so as to allow the cost panel to adequately identify and analyze the offerors' costing methods. In particular, section L of the RFP stated that offerors' "budgetary estimates for the acquisition contract outyears were to include assumptions, ground rules, methodology, and substantiating data." In addition to these specific RFP instructions, the contracting officer's letter requesting BAFOs warned offerors that "[a]ny change incorporated in your BAFO must be fully explained, substantiated, and tracked to your previous proposal." That letter went on to say that unsupported BAFOs or cost/price changes that were not adequately explained or lacked traceability to technical proposals "may not be considered credible in the final evaluation."

The record shows that while Cessna reduced its BAFO prices by what the agency considered to be a significant amount [DELETED], the cost panel concluded that contrary to the RFP's specific instructions, Cessna had not adequately supported its reductions. Specifically, the evaluators concluded that rather than providing any explanatory rationale, Cessna had only provided cursory, conclusory statements for its BAFO reductions, which made it virtually impossible for the cost panel to trace the basis for the reductions.

For example, in its BAFO Cessna reduced its proposed [DELETED].

The cost panel concluded that while Cessna's statements might summarize the basis for its reductions, those statements, without more detailed documentation, were virtually meaningless in view of the magnitude of the reductions. Specifically, the panel concluded that BAFO reductions based only on Cessna's [DELETED] statements did not provide the level of detail necessary for the government to develop an accurate MPLCC for Cessna.

The protester's argument that the agency's actions in this regard were unreasonable is without merit. The record shows that in developing Cessna's MPLCC, the cost panel thoroughly reviewed Cessna's BAFO reductions and concluded that, given the lack of substantiation, Cessna's BAFO prices could not form the basis for predicting outyear life-cycle costs. Rather than relying on Cessna's BAFO, the panel relied on what it had considered to be adequately substantiated, accurate data--Cessna's pre-BAFO prices.

As already explained, the RFP contained specific instructions for offerors to provide detailed explanations with their BAFO to allow the cost panel to fully assess the basis of any price reductions. The agency's letter requesting BAFOs also warned offerors of the consequences of their failure to adequately substantiate BAFO price reductions. We have reviewed the information Cessna submitted with its BAFO and agree with the agency's position that in several respects, Cessna simply failed to provide sufficiently detailed explanations to allow the agency's cost panel to "trace" Cessna's BAFO reductions, or accurately forecast life-cycle costs for Cessna. Accordingly, we think that the cost panel's decision to rely on Cessna's pre-BAFO pricing information in developing certain aspects of the protester's MPLCC was reasonable. See AmerInd, Inc., B-248324, Aug. 6, 1992, 92-2 CPD para. 85.

The protest is denied.

Comptroller General of the United States

1. Offerors were required to submit proposals for supporting two approaches for CLS--(1) a combination of partial CLS for the Air Force and total CLS for the Navy; and (2) total CLS for the Air Force and the Navy. The RFP stated that although the Air Force would evaluate both approaches, award would be based on only one approach. The proposed award to Beech was based on the partial CLS approach.

2. Within the "operational utility/technical" area, the "operational capability" and "crew accommodations" factors were of equal importance, with each one of those two factors being more important than the remaining factors within this area. The factors within each of the other areas were of equal importance.

3. The color/adjectival ratings were blue (exceptional); green (acceptable); yellow (marginal) and red (unacceptable). Risk ratings were high, moderate, or low.

4. Beech's proposal received a proposal risk rating of "moderate" with "low" performance risk under the "crew accommodations" factor.

5. Cessna submitted two proposals, identified in the record as alternates "1" (the basic proposal) and "2," both of which were evaluated by the SSEB. Cessna's basic proposal retained all of the model contract clauses included in the RFP; in its alternate "2" (which the protester refers to as its "commercial" proposal), Cessna proposed changes to several model contract clauses. Cessna's "commercial" proposal received "moderate" proposal risk ratings under "aircraft missionization" and "aircraft management." Both proposals received identical ratings in every other respect.

6. The MPLCC developed for Cessna's "commercial" proposal was [DELETED] billion, slightly less than the MPLCC developed for Cessna's basic proposal.

7. The Secretary of the Air Force was the SSA for this procurement.

8. The Rockwell International Corporation, another unsuccessful offeror, also filed protests in our Office challenging the award to Beech. In our decision, Rockwell Int'l Corp., B-261953.2; B-261953.6, Nov. 22, 1995, 96-1 CPD para., we denied Rockwell's protests.

9. In its initial protest, Cessna also challenged the evaluation of its proposal under the "certification/qualification," "aircraft missionization," and "manufacturing" factors. In its comments on the agency report, Cessna expressly stated that it was not pursuing these specific issues.

10. Beech also asserts that Cessna's theory that twin-engine aircraft are inherently safer than single-engine aircraft trivializes a very complex issue involving several factors, of which the number of engines is but one. Some of those factors include operational environment, adequacy of training and supervision, reliability and worthiness of aircraft structure and engines, flying characteristics, and other design features which interact to determine the relative safety of any aircraft. The record indicates that there is considerable debate in the industry concerning this issue.

11. Cessna also contends that the Air Force improperly "coached" Beech throughout discussions and held unequal discussions with Beech and Cessna. We have reviewed the voluminous record in these protests, particularly regarding written and oral discussions, and conclude that there is no evidence that the Air Force "coached" or otherwise treated offerors unequally during the discussions. Cessna's arguments in this regard are simply without any support in the record.

12. Section M of the RFP described the purpose of the evaluation under this factor as follows:

"Operational capability. Evaluates the performance, flying qualities, and training mission accomplishment of the proposed production aircraft. The evaluation will integrate the results of the [f]light [e]valuation with the results of the evaluation of the written proposal for the production aircraft. The operational capability assessment will be a quantitative and qualitative evaluation conducted by [g]overnment personnel to assess the offeror's aircraft's suitability to perform a 65-sortie, 89-flying hour primary training syllabus given an entry level student pilot with no previous flying experience."

13. The evaluators noted that the change to performance or flying qualities was [DELETED].

14. Cessna argues that the Air Force used an unstated evaluation criterion in its evaluation of proposals under the "operational capability" factor, and that in so doing, the Air Force deprived Cessna of a meaningful opportunity to correct deficiencies. Cessna also contends that the Air Force improperly "compressed" all technical color/adjectival scores into the "green" (acceptable) range, thus making it virtually impossible to distinguish between proposals of different technical merit. Rockwell also raised these issues in its protests, and they were addressed and disposed of in our decision in Rockwell Int'l Corp., supra.

15. The evaluators also noted that [DELETED].

16. Cessna had apparently removed flight test instrumentation and recorders from its evaluation aircraft after the flight evaluation.

17. In fact, the record shows that based on the SSEB's initial evaluation, the agency identified several weaknesses and risks associated with the aircraft's [DELETED]. The record shows that the agency identified these problems during discussions with Cessna, and the protester's solutions resolved these problems to the SSEB's satisfaction. The protester's argument that the agency failed to consider or recognize that its proposed solutions corrected deficiencies in its proposal, is without merit.

18. In addition [DELETED].

19. Cessna also challenges the agency's evaluation of its "commercial" or alternate "2" proposal. In this connection, Cessna argues that the Air Force improperly downgraded the risk ratings the agency assigned to its alternate proposal and failed to consider the cost advantages associated with that proposal. Cessna's arguments notwithstanding, the RFP did not encourage or favor "commercial" proposals. Rather, the RFP merely permitted offerors to propose exceptions and deviations to the terms and conditions of the JPATS RFP and required offerors to provide a detailed rationale in support of the alternate proposal. We have reviewed the record, including Cessna's alternate "2" proposal, the agency's evaluation materials, and the protester's submissions in this regard. The record shows that the evaluators were not convinced by Cessna's rationale for the changes Cessna made in its alternate proposal. Based on our review, we find no basis to question the agency's downgrading of Cessna's alternate "2" proposal under two risk factors. Cessna's mere disagreement with the agency evaluators does not render the evaluation of its alternate proposal unreasonable. Allied-Signal Aerospace Co., B-250822; B-250822.2, Feb. 19, 1993, 93-1 CPD para. 201.

20. Each of these "cases" is a group of nine body dimensions used to describe the threshold population required to be accommodated. The agency states that accommodation of a seventh case was a desired goal, but not an RFP requirement.

21. The remeasurement of the cockpit mock-up also allowed Cessna to correct several weaknesses in its cockpit, resulting in an improvement of the rating assigned Cessna's proposal under this factor from [DELETED].

22. To the extent that Cessna argues that the AFOTEC's evaluation was unreasonable, we have reviewed the record in this regard and find no basis to question the AFOTEC's final EOA report. The record shows that each instance in which the AFOTEC changed Cessna's or Beech's initial ratings was the result of the AFOTEC's assessment of the offerors' responses to CRs and DRs. Specifically, with respect to the protester's proposal, the record shows that the AFOTEC found that Cessna's proposed changes to its aircraft required additional testing, and reasonable downgraded the proposal accordingly. Cessna's mere disagreement with the AFOTEC's judgment do not render the final EOA unreasonable. See Mentor Technologies, Inc., B-258009, Nov. 17, 1994, 94-2 CPD para. 195.

23. The contract in question was for the sale of six Beech aircraft to Egypt. Beech was required under the FMS program to file certifications with the Defense Security Assistance Agency (DSSA) identifying any foreign components or services used in fulfilling the contract as well as any contingent fees paid in connection with the foreign sale. Motorola, a subcontractor to Beech on the FMS contract, paid a contingent fee to an Egyptian firm in connection with the contract. In its certifications to DSSA, however, Beech failed to identify that services had been procured from the Egyptian firm or that a contingent fee had been paid to that company in connection with the contract. Beech was charged with furnishing a false certificate to DSSA. The record shows that in May 1995, Beech entered a plea of guilty to one misdemeanor count resulting in fines and sanctions totaling more than $1 million.

24. The purpose of this certificate is to assist the contracting officer in determining an offeror's responsibility. FAR section 9.408. Even if that Cessna were correct that Beech's certification was deficient, the failure to properly complete the certification would not require the rejection of its proposal. See Universal Technologies, Inc.; Spacecraft, Inc., B-248808.2 et al., Sept. 28, 1992, 92-2 CPD para. 212; see also Intermountain Elec., Inc., B-236953.2, Jan. 31, 1990, 90-1 CPD para. 143.

25. The decisions, Cessna Aircraft Co., ASBCA No. 37726, Dec. 12, 1994, 95-1 BCA para. 27,373, and Cessna Aircraft Co., ASBCA No. 48118, Mar. 6, 1995, 95-1 BCA para. 27,560, involved claims for equitable adjustments based upon constructive changes to contract specifications. The agency points out that none of the disputes in decision No. 48118 involved technical issues that would bear on Cessna's performance risk assessment. The agency also states that the Navy has appealed decision No. 48118 to the United States Court of Appeals for the Federal Circuit.

26. Further, the agency challenges Cessna's position that Cessna was not at fault, arguing that the Board left open the question of the degree of fault between the Navy and Cessna. The agency states that the allocation of missionization risk/fault to Cessna alone could reasonably support a "moderate" performance risk assessment even if the PRAG had been made aware of the ASBCA decision.

27. Cessna also argues that the Air Force failed to conduct meaningful discussions with the firm because the agency did not give Cessna any notice that it believed that Cessna had failed to meet original [DELETED] with respect to the UNFO contract. Contrary to the protester's assertions, the record shows that the Air Force informed Cessna during discussions of the performance problems the Navy reported under the UNFO contract.

28. Cessna also states that under a proper evaluation, its "commercial" alternate proposal would have been approximately [DELETED] lower in cost/price than Beech's proposal.