Matter of: ECC International Corporation File: B-277422, B-277422.2 Date: October 14, 1997 * REDACTED DECISION

Matter of: ECC International Corporation File: B-277422, B-277422.2 Date: October 14, 1997 * REDACTED DECISION

DIGEST

Attorneys

DECISION

ECC International Corporation protests the award of a contract to AAI Corporation under request for proposals (RFP) No. F33657-97-R-2009, issued by the Department of the Air Force for concurrency upgrades and logistics support of the agency's C-17 aircraft maintenance training devices. ECC maintains that the Air Force misevaluated proposals in the area of past performance and made an irrational cost/technical tradeoff in awarding the contract to AAI. ECC also maintains that the evaluation reflects bias on the part of the agency against ECC.

We deny the protest.

The RFP contemplated the award of a combination cost reimbursement/firm, fixed-price contract on a best value basis to perform modifications to two suites of C-17 aircraft maintenance training devices (MTD), as well as ongoing logistics support and the conduct of a maintenance training analysis study for the C-17 MTD program. The modifications portion of the contract, the largest dollar item being procured, is to be performed on a cost reimbursement basis, while the logistics support and maintenance training analysis study are to be performed on a firm, fixed-price basis. The MTDs are used to train agency personnel in various aspects of C-17 aircraft maintenance. The modifications are necessary in order to make the configuration of the C-17 MTDs "concurrent" with, or identical to, the configuration of presently fielded aircraft; design and engineering changes to the aircraft since the time the C-17 MTDs were originally built have necessitated the current requirement.

The RFP advised offerors that proposals would be evaluated for cost realism, reasonableness, and completeness, as well as for technical adequacy under three primary technical criteria (none of which is at issue in this protest), and that a proposal risk rating would also be assigned in each of the technical evaluation areas. In addition, the RFP advised that firms would be evaluated by a performance risk assessment group (PRAG) for past performance based on information supplied with the proposals, as well as information obtained independently by the agency.

In response to the solicitation, the Air Force received five initial offers. After evaluating the offers and conducting a performance risk assessment, the agency determined that it was in the best interest of the government to make award based on initial offers. ECC's and AAI's proposals received comparable technical and proposal risk scores, and ECC's evaluated cost was low. The Air Force nonetheless made award to AAI, the firm offering the second lowest evaluated cost, based largely on the conclusion of the PRAG that AAI presented a lower performance risk than ECC. The agency's performance risk evaluation is the focus of ECC's protest.

The PRAG conducted its evaluation by reviewing each offeror's performance under the three technical evaluation areas: Concurrency Approach/Implementation, Maintenance Training Analysis, and Contractor Logistics Support. Risk ratings also were assigned for Cost/Price. Ratings of low, moderate, or high risk were assigned under each of the four areas unless the agency concluded that there was insufficient information to evaluate a firm in a given area, in which case the PRAG assigned a rating of not applicable. In conducting its assessment, the PRAG reviewed the information submitted in the past performance volumes of the proposals, Contractor Performance Assessment Reports (CPAR), [1] and responses to questionnaires prepared by contracting activities that had administered contracts performed by the firm being reviewed. The ratings assigned to ECC and AAI for performance risk were as follows:

CRITERIA AAI ECC

Concurrency/Implementation Moderate High Maintenance Training Analysis Not Applicable Not Applicable Contractor Logistics Support Low Low Cost Moderate Not Applicable

CONCURRENCY APPROACH/IMPLEMENTATION

ECC takes issue for several reasons with the agency's assignment of a high risk rating to its proposal under the Concurrency Approach/Implementation [2] criterion, as compared with the assignment of a moderate risk rating to AAI's proposal. ECC maintains that the risk evaluation improperly failed to consider its positive performance under several contracts and at the same time ignored AAI's allegedly negative performance under several of that firm's contracts. With respect to those CPARs that were considered, ECC maintains that the agency essentially ignored criticisms of AAI's performance and also ignored ECC's comments submitted in response to the CPARs reviewed. In this latter regard, ECC maintains that the evaluation record fails to show that the PRAG considered ECC's comments in response to the CPARs. [3]

In reviewing an agency's performance risk evaluation, our Office considers only whether the evaluation was reasonable and consistent with the solicitation's stated evaluation scheme. H.F. Henderson Indus., B-275017, Jan. 17, 1997, 97-1 CPD Para. 27 at 3. In performing these evaluations, agencies properly may discriminate between past performance references based on the relevance of the prior contract to the requirement being solicited, and may attach greater weight to those contracts reasonably found to be more relevant. Israel Aircraft Indus., Ltd., MATA Helicopters Div., B-274389 et al., Dec. 6, 1996, 97-1 CPD Para. 41 at 9.

ECC's Performance Risk Rating for Concurrency Approach/Implementation

This aspect of the performance risk assessment was reasonable. In evaluating ECC's past performance, the agency relied heavily on the CPARs prepared in connection with the firm's performance of the predecessor production contract for the C-17 MTDs. (ECC was the contractor for the fabrication of the C-17 MTDs, and the current contract is essentially a follow-on upgrade contract to that effort.) The record shows that ECC experienced significant performance difficulties throughout the course of the predecessor C-17 MTD contract. In particular, [DELETED]. We find nothing unreasonable in the agency's assigning ECC a high performance risk rating based on this information.

There also is no basis for finding that the agency unfairly ignored ECC's comments submitted during the preparation of the CPARs. During preparation of the CPARs for the predecessor contract, ECC commented extensively regarding the negative performance evaluations, and the record shows that these comments were considered, but ultimately discounted, by the agency's reviewing official. In responding to ECC's comments on the final CPAR, the reviewing official explains that, after examining the criticisms of ECC's performance, discussing the matter with the cognizant program officials, and reviewing the firm's comments in response thereto, he found that the CPAR amounted to a fair but critical evaluation of ECC's performance. The reviewing official goes on to cite numerous examples where ECC's comments were not borne out by the facts and concludes:

In addition to the above discrepancies in the contractor's response, the program team answered all the contractor's points of dispute in telephone conversations, and during a video teleconference held solely to confirm the contractor's shortcomings in meeting contractual requirements. I am confident that the contractor's performance has been fairly reported.

The record thus shows that the protester's comments were reviewed in detail at the time they were submitted, and that the agency's conclusion at the time--that they were at variance with the facts--was reflected in the CPARs. The PRAG could reasonably have relied on that conclusion in essentially discounting ECC's comments, and the fact that they did not specifically state this in their evaluation report is immaterial in light of the agency's earlier detailed review.

As for the remainder of material relating to ECC's performance risk, the record shows that there were questionnaire responses for ECC relating to three other prior contracts. One of these was a "build to print" requirement, which the agency did not view as especially relevant to the current requirement, since it did not involve any design or engineering on the part of the protester. In addition, the record shows that this questionnaire included negative performance information about ECC. The agency explains that, as with all offerors, it did not consider any negative performance information on which the firm had not had a previous opportunity to comment, since award was to be made without discussions. As for the remaining two contracts, the record shows that, while they were relevant and contained favorable information relating to the protester's performance, the agency concluded that the information was insufficient to overcome the negative information included in the CPARs for ECC's prior C- 17 MTD contract. Given the high degree of relevance of ECC's performance on the C-17 MTD contract ([DELETED]), we think the agency reasonably accorded substantially less weight to the more favorable information in these two questionnaires, and reasonably assigned ECC a high risk rating. See Israel Aircraft Indus., Ltd., MATA Helicopters Div., supra. [4]

AAI's Performance Risk Rating for Concurrency Approach/Implementation

ECC maintains that the agency essentially ignored negative criticism of AAI's past performance in assigning AAI a moderate risk rating in the Concurrency Approach/Implementation area. The record shows that the agency assigned a moderate risk rating in this area based on [DELETED] AAI experienced on one of its predecessor contracts; ECC maintains that AAI also experienced [DELETED]. According to the protester, AAI should have received a high risk rating based on these facts.

The record shows that AAI's [DELETED] problems were less severe than ECC's. In this respect, [DELETED]. ECC also maintains that the agency ignored deficiencies in AAI's performance [DELETED] under the same predecessor contract. The record shows, [DELETED]. We conclude that the record reflects a qualitative difference between the difficulties experienced by ECC and AAI, and that this difference provides a reasonable basis for the agency's different risk ratings for the two firms.

Finally, ECC maintains that the Air Force improperly ignored several questionnaire responses that contained criticism of AAI's performance under a number of other contracts. This argument is without merit. For one of these contracts ([DELETED]. Since the record shows, however, that the agency was not requiring AAI to make concurrency upgrades in connection with that contract, we think the agency could properly view the rating as having only marginal relevance for purposes of assessing AAI's potential for success under the present RFP. As for the remaining contract questionnaire (which the agency concedes it did not consider), the record shows that, although AAI received some mild criticism in connection with its performance, it nonetheless received an "adequate" rating in the area identified by ECC in its protest.

In sum, the record shows that there was a reasonable basis for distinguishing between AAI's and ECC's past performance records. There is nothing in the record to show that the agency engaged in disparate treatment in evaluating performance risk in the Concurrency Approach/Implementation area for the two firms.

COST RISK ASSESSMENT

ECC maintains that the agency misevaluated cost in connection with the performance risk assessment. In this respect, the RFP called for the PRAG to assess risk on the basis of the offerors' performance under past contracts. The protester maintains that the agency improperly ignored its positive performance under a prior cost reimbursement contract, failed to consider information submitted by the offerors relating to their performance of fixed-price contracts, and improperly assigned a moderate, rather than a high, cost risk rating to AAI. [DELETED].

The record shows that ECC submitted information relating to only one cost reimbursement contract. The agency found that this contract was of limited value for purposes of predicting the potential cost risk associated with ECC's performance because the contract had not been competitively awarded and was for a substantially lower dollar value than the solicited requirement ([DELETED]). Accordingly, the PRAG assigned ECC a neutral ("not applicable") cost risk rating. Since, in our view, the agency acted reasonably in distinguishing this prior contract from the present requirement on the basis of its much smaller value, we have no basis to question this aspect of the evaluation. Similarly, we see nothing unreasonable in the agency's conclusion that fixed-price contracts would be of limited predictive value, since they would not provide insight into the offerors' performance in a cost reimbursement setting. [5]

Finally, with respect to ECC's assertion that the agency should have assigned a high, rather than a moderate, risk rating to AAI in the cost risk area, the protester's position amounts to little more than disagreement with the agency's evaluation conclusions. The record shows that, [DELETED]. We conclude that the PRAG reasonably assigned AAI's proposal a moderate risk rating based on AAI's improved performance over the life of the contract. [6]

MAINTENANCE TRAINING ANALYSIS RISK ASSESSMENT

ECC contends that the agency erred in assigning it a neutral ("not applicable") risk rating in the Maintenance Training Analysis area. According to the protester, it should have received a positive rating in this area because it proposed a subcontractor with an outstanding performance history. The agency explains that it did not evaluate ECC's subcontractor's performance risk because the subcontractor was proposed to perform less than 10 percent of the contract effort; the agency states that it treated all offerors alike in this regard. The record further shows that ECC did not include any information in the performance risk assessment portion of its proposal relating to prior contracts that had been performed by the subcontractor. We think the agency's actions in not assigning a maintenance and training analysis risk rating to ECC was reasonable, given the limited participation of its subcontractor, the lack of past performance information for the firm in ECC's proposal, and the fact that all offerors were treated alike in assessing the performance risk of their subcontractors.

COST/TECHNICAL TRADEOFF & SOURCE SELECTION

ECC contends that the agency failed to conduct a rational cost/technical tradeoff. To the extent that this argument depends on the challenges to the Air Force's evaluation of performance risk, it fails, since, as discussed above, we find no basis to question the reasonableness of the agency's evaluation. To the extent that the protester believes that the agency was required to quantify the dollar value of AAI's superior performance risk rating, the protest is without merit. There is no requirement that an agency quantify the value of technical superiority vis-a-vis low cost/price to determine the best value to the government. Picker Int'l, Inc., B-249699.3, Mar. 30, 1993, 93-1 CPD Para. 275 at 12. On the record here, we find that the agency has adequately justified its cost/technical tradeoff and award decision.

The protest is denied.

Comptroller General of the United States

1. The CPARs are part of the Air Force's ongoing performance review process. These annual reports review a firm's performance for the preceding year and, after being prepared, are submitted to the contractor for comment. After the contractor's comments are submitted, the CPARs are reviewed by a senior-level agency official who evaluates the relative merits of the CPARs and contractor comments and prepares a written opinion regarding the entire package. This package then is entered into a central Air Force data base which is available to contracting activities for purposes of conducting past performance reviews on subsequent procurements.

2. Under this criterion, technical proposals were evaluated under several subfactors, including the firm's ability to systematically and timely evaluate data to ensure concurrence between the training services and actual C-17 aircraft configuration, the firm's ability to integrate testing and evaluation requirements throughout the life of the contract so as to avoid resource-intensive, end-of-contract verification of requirements, and the offeror's schedule and management approach to ensure initial and continued concurrence between the training devices and the actual aircraft. In evaluating performance risk, the agency assigned ratings only to the "area" level (i.e., the agency did not assign risk ratings for each subfactor), but it is clear from the record that the same considerations were at issue. In essence, the offerors' past performance was reviewed to predict how well the firm might perform the central objectives of the contract as embodied in the evaluation criteria.

3. ECC also maintains that the contemporaneous record from the agency's source selection activities shows that important information was not considered. While it is true that there are several matters not explicitly addressed in the contemporaneous record, the record as a whole, including sworn statements submitted by the agency's cognizant source selection personnel, indicates that the matters were discussed, even if no contemporaneous record of that discussion was kept; we therefore conclude that the protester's contention in this respect is without merit.

4. In a related contention, ECC maintains that the agency's evaluation and source selection officials were biased against it because of the firm's performance on the predecessor C-17 MTD contract. As discussed, we find that the agency's performance risk assessment of ECC was reasonable. Thus, even if we were to find evidence of bias (and we have found none here), it would provide no basis for our Office to sustain ECC's protest since, in order to demonstrate bias, a protester must also show that the bias translated into action; where the evaluation is otherwise reasonable, there is no basis for making this latter finding. Hagler Bailey Consulting, Inc., B-265708, Dec. 19, 1995, 95-2 CPD Para. 276 at 3.

5. The assessment of the relevance of performance under fixed-price contracts is an area where the contemporaneous record does not reflect the agency's assertion that this matter was discussed at the time and that it was decided that those contracts were of limited predictive value. As noted above, we have accepted the agency's assertion that this matter was discussed, because of the sworn statement to that effect submitted in response to the protest, the lack of countervailing argument or evidence, and the reasonableness of the position itself. We note as well that ECC has failed to demonstrate prejudice in connection with the agency's alleged failure to consider fixed-price contract information. [DELETED].

6. ECC also maintains that the agency improperly did not consider AAI's performance on two other contracts where, according to the protester, AAI's performance was unsatisfactory. The record shows that, as to one of these contracts, the requirement was developmental in nature, that the rating official stated with respect to cost that there were many "unknowns" at the beginning of performance and that, as a consequence, there was not a firm target cost against which to measure performance. The rating official ultimately assigned AAI an "acceptable" rating for "effective cost performance." The record shows that the other contract was only in the first year of contract performance and that costs still were within the government's evaluated cost estimate used for source selection purposes. This rating official also ultimately rated AAI's effective cost performance as "acceptable."

* DOCUMENT FOR PUBLIC RELEASE

The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.