Matter of: CTA Incorporated File: B-253654 Date: October 12, 1993

B-253654: Oct 12, 1993

Additional Materials:

Contact:

Shirley Jones
(202) 512-8156
jonessa@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

PROCUREMENT Bid Protests GAO procedures Interested parties Direct interest standards Where both protester's and awardee's proposals were technically acceptable but protester's proposal presented high performance risk while awardee's proposal was rated low risk in all areas. Protester's price was less than 1 percent lower than awardee's. General Accounting Office will not review allegation that agency misevaluated protester's technical proposal risk. PROCUREMENT Competitive Negotiation Contract awards Administrative discretion Cost/technical tradeoffs Technical superiority Protest that agency's decision to award to higher-priced offeror lacked an adequate basis is denied where record clearly supports agency's conclusion that awardee's lower-risk proposal was worth the price premium of less than 1 percent.

Matter of: CTA Incorporated File: B-253654 Date: October 12, 1993

PROCUREMENT Competitive Negotiation Offers Risks Evaluation Technical acceptability Agency reasonably concluded that protester's proposal presented a high performance risk where protester received poor performance evaluations under three similar contracts and a favorable evaluation under only one similar contract. PROCUREMENT Bid Protests GAO procedures Interested parties Direct interest standards Where both protester's and awardee's proposals were technically acceptable but protester's proposal presented high performance risk while awardee's proposal was rated low risk in all areas, and protester's price was less than 1 percent lower than awardee's, General Accounting Office will not review allegation that agency misevaluated protester's technical proposal risk, since any improvement in protester's proposal risk rating would not place it in line for award. PROCUREMENT Competitive Negotiation Contract awards Administrative discretion Cost/technical tradeoffs Technical superiority Protest that agency's decision to award to higher-priced offeror lacked an adequate basis is denied where record clearly supports agency's conclusion that awardee's lower-risk proposal was worth the price premium of less than 1 percent.

Attorneys

DECISION CTA Incorporated protests the award of a contract to Hughes Training, Inc. under request for proposals (RFP) No. F33657-92-R-0003, issued by the Department of the Air Force for unit training devices (UTD) for F-15 and F-16 aircraft. CTA alleges that the Air Force improperly downgraded its proposal based on technical and past performance considerations.

We deny the protest.

The solicitation contemplated the award of a firm, fixed-price contract for two F-15 and three F-16 UTDs, including associated support services. Each UTD consists of a pilot station providing realistic flying qualities, an instructor operator station, and computer/processor equipment and interfaces to operate the trainer and generate visual displays. The RFP required that existing hardware and software be used to the maximum possible extent.

Award was to be based on the Air Force's integrated assessment of proposals in four areas: technical, management, supportability, and most probable life cycle cost (MPLCC). Technical factors were considered the most important; management, supportability and MPLCC were progressively less important. In each area except cost, the proposals were to receive a color-coded rating (i.e., blue for exceptional, green for acceptable, yellow for marginal, or red for unacceptable), a proposal risk rating, and a performance risk rating. Within each area, each of the three ratings (color, proposal risk and performance risk) was to be given equal weight in the integrated assessment of proposals. Finally, the evaluation was to include "general considerations," including an operational demonstration of the proposed UTD. The demonstration was to be used in assessing "the risk of obtaining the proposed specification capability within the proposed schedule, manpower resources, and cost."

Five offerors submitted proposals in response to the RFP; all five were considered to be in the competitive range. Following discussions and submission of best and final offers (BAFO), the source selection evaluation team (SSET) rated the offerors as follows:

Color Proposal Performance MPLCC Offeror Rating Risk Risk (millions)

Hughes Green Low Low 48.0 Offeror B Green Low Low 51.1 Offeror C Green Medium Low 73.7 Offeror D Green Medium Low 109.7 CTA Green High High 47.6

Based upon the evaluation results, the source selection authority (SSA) determined that an award to Hughes would be most advantageous to the government. In justifying the award to Hughes at a MPLCC slightly (less than 1 percent) higher than CTA's low MPLCC, the SSA listed a number of perceived strengths of Hughes's proposed system that "more than offset" CTA's cost advantage. Upon learning of the award to Hughes, CTA filed this protest.

CTA alleges that the Air Force misevaluated its proposal with respect to both proposal risk and performance risk, and that it failed to adequately justify award to a higher-priced offeror. As discussed below, we find CTA's allegations without merit and conclude that the award to Hughes was proper.

TECHNICAL PERFORMANCE RISK

The RFP provided for the assignment of a performance risk rating, reflecting the probability that the offeror will successfully accomplish the proposed effort, to each of three technical evaluation areas: systems engineering (item T.1), UTD system design and performance (item T.2), and simulation software modeling (item T.3). Each of the three areas was to receive equal consideration in the source selection decision.

The RFP provided that the performance risk assessment would be based on the offeror's present and past performance on government contracts; accordingly, it required each offeror to identify the contracts, in order of relevance to the UTD program, that would best reflect its ability to perform the UTD effort as proposed. The risk assessment was performed by a past performance risk assessment group (PPRAG). For each contract identified by an offeror, the PPRAG researched available past performance information, including Air Force contract performance assessment reports (CPAR), and sent questionnaire forms to the contracting activity. Based on this information, the PPRAG arrived at a risk rating for each technical area of low, medium or high. Where an offeror received a rating of other than low risk, the offeror was notified of the concern in writing and was asked to address it. The offeror's explanation was then evaluated to determine whether the risk rating should be changed.

CTA identified four recent contracts for the PPRAG's consideration: the Air National Guard F-15/F-16 part task trainer (PTT), the T-45 undergraduate navigator trainer system (UNTS), the pilot candidate selection method (PCSM), and the table-top navigational rendezvous trainer (TTNRT). In addition to those four contracts, the Air Force identified the U.S. Army AH-15 helicopter trainer program (AH-15) as a relevant CTA contract. Based on the contracting agencies' responses to the PPRAG questionnaire, the PPRAG assessed CTA's performance as acceptable for the PTT and PCSM contracts and unacceptable for the remaining three contracts. CTA then was given the opportunity to address the perceived performance problems on those contracts; none of CTA's responses resulted in any change in the firm's risk assessment. The PPRAG concluded that CTA's performance risk was moderate for item T.1 and high for items T.2 and T.3; these ratings were combined into a summary rating of high for all three areas.

CTA challenges the agency's conclusion that its proposal reflected a high level of performance risk, arguing that the agency improperly gave too much weight to its poor performance records on three contracts that were not similar to the UTD effort, and not enough weight to its favorable performance evaluation on the PTT contract, the one most similar to the UTD effort. In support of its argument, CTA points to a memorandum from the PPRAG chairman to the SSET chairman expressing the PPRAG's inability to determine whether a contractor's satisfactory performance on one contract very similar to the UTD program should outweigh an overall poor performance record. CTA asserts that this memorandum was intended to refer specifically to CTA's performance risk evaluation. The memorandum essentially concludes that the SSA should be responsible for considering whether the offeror's improved performance on a recent, similar contract indicates that the contractor has "climbed sufficiently on the learning curve to avoid repeating [its] past mistakes," thereby establishing that the risk of poor performance on the UTD program is minimal. CTA argues that the SSA improperly failed to make such a determination with respect to CTA's past performance.

In reviewing an evaluation of an offeror's performance risk, we will examine it to ensure that it was reasonable and consistent with the stated evaluation criteria, since the relative merit of competing proposals is primarily a matter of agency discretion. See Instrument Control Serv., Inc., B-247286, Apr. 30, 1992, 92-1 CPD Para. 407. We find that the record here supports the Air Force's conclusion that CTA's proposal presented a high performance risk, and that the SSA properly took this into account in his selection decision.

As noted above, the RFP informed offerors that performance risk would be assessed based on the agency's review of the offeror's performance history under certain identified contracts. Accordingly, the PPRAG reviewed the performance history of five relevant contracts in determining the performance risk that an award to CTA would pose; the record shows that each contract was thoroughly considered in the risk assessment. In this regard, the record does not show that CTA's PTT contract is more relevant in assessing performance risk for the UTD program than any of the other contracts the PPRAG reviewed. While the PTT program apparently is similar to the UTD program in more respects than the three contracts for which CTA received unfavorable performance reviews, the record shows that those three contracts were very similar to UTD in one way or another, and the similar aspects were specifically considered in the risk assessment.

For example, in considering CTA's poor performance history under the TTNRT contract, the PPRAG noted that the software development and testing processes under that contract were very similar to those required under the UTD contract. Thus, the PPRAG concluded that CTA's marginal management and poor technical and cost performance under the TTNRT contract were relevant to the risk assessment for the UTD contract. Likewise, the PPRAG determined that several tasks under the UNTS contract, including development of software and updating of student and operator stations, were related to UTD tasks; CTA's software development problems and consequent poor ratings under technical, management and cost performance were therefore taken into account in the UTD risk assessment. Finally, the PPRAG found that the AH-15 contract had a software rewrite requirement that was similar to a UTD requirement. The Army attributed CTA's poor performance under that contract to an inadequate software management process; the PPRAG concluded that this represented a risk to the firm's performance under the UTD contract.

Although the PPRAG apparently was concerned about the possibility that an offeror might be a better risk than its overall past performance would indicate, there was nothing inherently unreasonable in the PPRAG's conclusion that CTA's favorable performance record on one contract did not out-weigh the firm's consistently poor performance record on the three other recent contracts. See Instrument Control Serv., supra. Since we find reasonable the PPRAG's conclusion that CTA presented a high performance risk for the UTD effort, we have no basis to question the SSA's incorporation of that risk rating into his selection decision.

PROPOSAL RISK

The RFP provided for the assignment of a proposal risk rating, reflecting the risk of the offeror's proposed approach, to each of three technical evaluation areas: systems engineering (item T.1), UTD system design and performance (item T.2), and simulation software modeling (item T.3). Each of the three areas was to receive equal consideration in the source selection decision. The Air Force evaluators rated CTA's proposal risk moderate for item T.1, high for item T.2, and low for item T.3. The high risk rating for item T.2 was based on a perceived weakness in CTA's flight control software. The weakness involved CTA's use of its PTT software as a model for the more complex UTD software; the SSET found that CTA had not provided enough detail about the flight test data that it planned to incorporate into the PTT software modifications to assure the Air Force that the more stringent UTD requirements would be met. The SSET chairman determined that since the fidelity of the flight control software is of critical importance to the UTD program, the high risk associated with item T.2 warranted an overall risk rating of high for the technical evaluation factor.

CTA contends that the Air Force's assessment of its proposal risk was inconsistent with the RFP because it failed to give equal weight to each of the three areas. If the agency had given equal consideration to each area in the evaluation, CTA asserts, it would have averaged the three area risk ratings of medium, high and low into a summary rating of medium. If the agency believed that the T.2 area was so critical, CTA argues, it would have given the area more weight in the RFP's evaluation scheme. CTA maintains that the Air Force did not in fact attach critical importance to this area during the evaluation, as illustrated by the agency's treatment of the perceived weakness in CTA's flight control software. In this regard, CTA notes that the agency issued it a clarification request (CR) as opposed to a deficiency report (DR) for this item. According to Air Force Regulation 70-30, under which this procurement was conducted, a CR is used "for the purpose of eliminating minor irregularities, informalities, or apparent clerical mistakes." A DR, on the other hand, is used to address an "unacceptable risk" or "omission of data which makes it impossible to assess compliance" with the solicitation requirements. CTA concludes that the agency either misled CTA about the seriousness of its concerns regarding the firm's approach to the flight control software, or is now grossly exaggerating the seriousness of the risk in a post-hoc rationalization of its award decision.

The Air Force responds that CTA's argument is misplaced, as it concentrates on the mathematical average of the firm's risk ratings rather than the proposal weaknesses that led to those ratings; the agency argues that the particular weaknesses that warranted a high risk rating under item T.2, in combination with the weaknesses in the other two areas, justified a high risk rating overall. While the record appears to support the agency's position, we need not address the merits of this issue because it is clear that CTA was not prejudiced even if the agency erred in this regard. Even if, as CTA alleges, the firm should have received a moderate proposal risk rating for the technical factor, it still had a high performance risk rating. In addition, CTA had a high performance risk rating under the cost factor, and moderate risk under the management factor; CTA has not challenged these ratings. Since Hughes had low risk ratings for both performance and proposal risk under all evaluation factors, a change in CTA's proposal risk rating under the technical factor from high to moderate--or even to low, for that matter--would not have affected the firm's competitive standing given its minimal price advantage. It therefore would serve no useful purpose for us to question the agency's assessment. See Comarco, Inc., B-249697.2, Jan. 26, 1993, 93-1 CPD Para. 65.

OPERATIONAL DEMONSTRATION

CTA contends that the Air Force failed to give its operational demonstration proper consideration in the award decision. Noting that section L of the RFP provided for consideration of the demonstration in assessing "the risk of obtaining the proposed specification capability within the proposed schedule, manpower resources, and cost," CTA argues that its favorable performance in the demonstration (which the Air Force does not dispute) should have been used to reduce its overall risk rating under the technical factor.

We need not decide whether the Air Force properly evaluated CTA's operational demonstration because CTA would not have been prejudiced by any misevaluation. Even if CTA's overall risk rating were reduced from high to low (although the record does not indicate that such a substantial adjustment was warranted), this would not improve the firm's competitive position, since Hughes had a superior technical proposal, lower risk ratings under the management and cost evaluation factors, and a price premium of less than 1 percent. Accordingly, we have no basis to consider this protest ground further. See Tektronix, Inc., B-244958; B-244958.2, Dec. 5, 1991, 91-2 CPD Para. 516.

TECHNICAL/COST TRADEOFF

Finally, CTA asserts that the SSA's source selection decision was not adequately documented because, while it discussed the strengths associated with Hughes' proposal, and concluded that these strengths outweighed CTA's cost advantage, it did not discuss the weaknesses and risks associated with CTA's proposal. While the decision itself did not enumerate the weaknesses and risks of the CTA proposal, these are well-documented in the record, as we have discussed, and were reasonably taken into account in the decision that the relative strength of Hughes's proposal was worth its slightly higher price. We therefore have no basis to question the SSA's failure to specifically discuss them in the decision document.

The protest is denied.

Sep 16, 2019

Sep 13, 2019

Sep 12, 2019

  • TCG, Inc.
    We dismiss the protest in part and deny the protest in part.
    B-417610,B-417610.2

Sep 11, 2019

Sep 10, 2019

Sep 9, 2019

Looking for more? Browse all our products here