B-240647.4, Jul 23, 1991, 91-2 CPD ***

B-240647.4, Jul 23, 1991, 91-2 CPD ***

PROCUREMENT - Competitive Negotiation - Offers - Evaluation errors - Evaluation criteria - Application DIGEST: 1. Protest that agency did not follow stated evaluation criteria is denied where record indicates that evaluation was reasonable and consistent with the requirements set forth in the solicitation. 2. Contention that agency evaluation of protester's technical proposal was "erroneous, false, misleading, and unjustified" is denied where there is no evidence of agency bias in the record; the evaluation has a reasonable basis and is consistent with the stated evaluation criteria; and the agency's revisions to its evaluation report based on prior General Accounting Office decision provide sufficient detail to support the agency's evaluation findings.

Attorneys

Amtec Corporation

Amtec Corporation protests the award of a contract to Delta Research, Inc. under request for proposals (RFP) No. DASG60-90-R-0016, issued by the Department of the Army, Strategic Defense Command, for detailed technical studies of specific defensive missile systems and certain analyses associated with the flight test program for those systems. Amtec argues that the Army's evaluation of its proposal does not follow the evaluation factors stated in the RFP and applies minimum requirements not stated in the evaluation factors and subfactors. Amtec also argues that the evaluation was "erroneous, false, misleading, and unjustified," and therefore lacked a reasonable basis.

We deny the protest.

The Army selected Delta for award after reconsidering its evaluation of Amtec's proposal in light of our prior decision in Amtec Corp., B-240647, Dec. 12, 1990, 90-2 CPD Para. 482, recon. denied, B-240647.2, Feb. 26, 1991, 91-1 CPD Para. 211. In that decision, we denied Amtec's challenges to the Army's review of Amtec's proposed costs, and to the adequacy of discussions, but we sustained Amtec's challenge to the evaluation of its proposal as marginal under the technical approach subfactor because that conclusion was not adequately supported by the record. Since the Army's award decision was based on its finding that Delta's superior technical proposal justified its higher cost, we recommended that the Army review its marginal rating of Amtec and reconsider its selection decision.

As a result of our decision, the Army's Proposal Evaluation Team revised its report to provide greater details explaining its assessment of Amtec's proposal. Specifically, the evaluation team retained the marginal rating for Amtec for the technical approach subfactor of the technical factor, and recommended an overall rating of marginal for the proposal. addition, the Source Selection Authority reconsidered his selection decision and again selected Delta for award based on the higher overall rating given to Delta's proposal. After awarding the contract to Delta on February 27, 1991, the Army provided Amtec with copies of the revised evaluation report and source selection decision on March 19. This protest followed.

BACKGROUND

On January 26, 1990, the Army issued this solicitation as a small business set-aside, anticipating the award of a cost-plus-fixed-fee, level -of-effort contract. Proposals were to be evaluated on the basis of three factors: technical, management, and cost. For both the technical and management factors, the RFP listed two subfactors each and provided that each subfactor would be accorded equal weight within its factor. For the technical factor, the subfactors were technical approach and qualifications of personnel; for the management factor, the subfactors were organization structure and applicable experience. The RFP also stated that the technical factor was significantly more important than the management factor, and that the technical and management factors were significantly more important than evaluated probable cost. In evaluating the proposals, members of the evaluation team scored each factor and subfactor using one of five adjectival ratings: exceptional; good; acceptable; marginal; or unacceptable.

Only Amtec and Delta submitted proposals in response to the RFP. After extensive written and oral discussions, the Army awarded the contract to Delta based on its overall rating of exceptional compared to Amtec's overall rating of marginal, even though Amtec's proposal had the lower evaluated cost. This initial selection decision, and the underlying evaluation of Amtec's proposal as marginal, was the subject of Amtec's first protest to our Office.

As explained above, Amtec's initial protest was denied on several counts, but was sustained on its claim that the Army had failed to support adequately its determination that the Amtec proposal was marginal in the technical approach subfactor. Specifically, we stated:

"We have reviewed Amtec's submissions under the RFP, and given the detail and apparent organization of the technical discussion in them, and the corresponding lack of any detailed explanation from the Army, either in the evaluation documents or in the report on the protest, of its conclusion that Amtec's technical proposal was poorly organized and lacking in technical detail, we find that the record does not support the Army's rating of Amtec as marginal in the technical area." (Footnote omitted.)

As a result of these findings, the Army revised the reports prepared by its Proposal Evaluation Team and its Source Selection Authority to reflect the basis for the prior decision.

DISCUSSION

Amtec's protest cites three challenges to the Army's evaluation of its proposal based on the revised evaluation report and selection memorandum: (1) that the Army did not evaluate Amtec's proposal in accordance with the evaluation factors stated in the RFP; (2) that the Army failed to inform Amtec of minimum requirements for certain evaluation factors and subfactors; and (3) that the Army's evaluation was "erroneous, false, misleading, and unjustified," and thus lacked a reasonable basis.

Amtec's contentions raise essentially the same issue: whether the Army's evaluation of Amtec's proposal followed the evaluation factors set forth in the RFP, or whether the Army improperly abandoned the stated evaluation factors to impose certain unstated requirements, and thus denied Amtec the opportunity to submit a proposal fully responsive to the agency's needs.

As Amtec correctly argues, evaluations and award decisions in negotiated procurements must be in accordance with the terms of the RFP. Environmental Technologies Group, Inc., B-235623, Aug. 31, 1989, 89-2 CPD Para. 202. In reviewing protests against allegedly improper evaluations, we examine the record to determine whether the agency's judgment was reasonable and consistent with stated evaluation criteria and applicable statutes and regulations. ESCO, Inc., 66 Comp.Gen. 404 (1987), 87-1 CPD Para. 450. Such judgments by their nature are often subjective; nonetheless, the exercise of these judgments in the evaluation of proposals must be reasonable and must bear a rational relationship to the announced criteria upon which competing offers are to be selected. See Bunker Ramo Corp., 56 Comp.Gen. 712 (1977), 77-1 CPD Para. 427.

Based on our review of Amtec's proposal, the original evaluation materials, the revised decisional materials, Amtec's detailed challenges to those materials, and the Army's response to each of Amtec's arguments, we find that the evaluation was reasonable and did not deviate from the stated evaluation criteria. Amtec offers a multitude of arguments in its attempt to show that the Army deviated from the evaluation criteria; we will discuss a few representative examples in the following analysis. /1/

Simulation Software

Amtec complains that the Army unreasonably downgraded its proposal for overemphasizing the company's experience with certain simulation software expressly required by the statement of work. In this regard, the RFP requires offerors to use simulation software called The Endoatmospheric Nonnuclear Kill Simulation (ENDOSIM) to develop a model of an experimental missile. The RFP alerts offerors that this effort will require the use "of standard modules within ENDOSIM and the development of new modules." After noting Amtec's strength in the area of simulations, and in using ENDOSIM in particular, the evaluation report states that Amtec's proposal overstates the value of its familiarity with the simulation system while failing to demonstrate a strong technical capability in other required areas. Amtec asserts that this observation is incorrect and improper since the ENDOSIM system is required by the RFP.

In its proposal Amtec states that its subcontractor will provide the specific expertise in missile or target characteristics while Amtec will provide the expertise with simulation software. This proposed division of duties-- together with Amtec's failure to provide as much information to establish its familiarity with the underlying missile and target expertise as it provides about its familiarity with the required simulation tool-- led the Army to conclude that Amtec's proposal overemphasized the simulation process while underemphasizing the underlying missile expertise. Amtec argues that the criticism is unfair, but fails to squarely address the stated concern: that Amtec's proposal shows more familiarity with the tools of the analysis than the substance. Without addressing the concern, and in expressing only disagreement with the conclusion, Amtec has not established that the agency's view of the proposal in this regard is unreasonable. See ESCO, Inc., 66 Comp.Gen. 404, supra.

Lack of Detail

Related to its charge that the Army did not follow the evaluation criteria, Amtec also complains that the evaluators impermissibly applied a requirement for some minimum level of detail in evaluating the proposal. According to Amtec, whenever the evaluators stated that the proposal lacked detail or depth, they were creating an evaluation requirement not set forth in the RFP.

Amtec correctly asserts that agencies must give sufficient detail in solicitations to allow offerors to intelligently prepare their proposals and compete on an equal basis. See Quantum Research, Inc., B-242020, Mar. 21, 1991, 91-1 CPD Para. 310. Under the rules applicable to this procurement, however, agencies are not required to list all subfactors which may be used for evaluation purposes so long as those subfactors are reasonably related to the RFP's stated evaluation criteria. Id. In our view, certain assessments-- in this case, whether an offeror has provided sufficient detail in its technical proposal-- are an inherent part of the evaluation of technical proposals, regardless of whether the RFP specifically enunciates the item as an evaluation factor or subfactor. See Advanced Sys. Technology, Inc.; Eng'g and Professional Servs., Inc., B-241530; B-241530.2, Feb. 12, 1991, 91-1 CPD Para. 153 (consideration of risk involved in an offeror's approach is inherent in the evaluation of technical proposals even where the RFP does not specifically mention risk as a consideration).

Although the evaluation factors in the RFP did not include a stated minimum level of detail, the RFP, at paragraph L.19, required offerors to "... outline how to perform the actual work proposed as specifically as possible." Given this requirement-- stated explicitly here, but inherent in any RFP-- Amtec cannot successfully argue that the agency acted improperly in considering the depth and level of detail in its technical proposal.

In addition to challenging whether the Army could reasonably consider the level of detail in its proposal, Amtec also disagrees, in several instances, with the substance of the Army's criticism regarding lack of detail. For example, the statement of work, at paragraph 3.2.2.1, requires the offeror to analyze the intended infrared detector system selected for use here: a focal plane array using platinum silicide as the photon sensitive material. In analyzing the infrared detector, the offeror was required to:

"... analyze the seeker performance degradation due to the aero optic effect of the window cooling gas, based on flowfield and refraction calculations performed by the prime and other contractors. The contractor shall identify technical issues and recommend possible design changes to reduce the risks associated with the seeker."

The evaluation report concluded that Amtec's response was simply a general recitation of its subcontractor's experience in analyzing infrared seekers in two other programs, and lacked details of the "actual analysis performed, problem areas identified, and critical issues resolved." Amtec complains that the evaluation criticism is exaggerated and misleading since the proposal, in fact, contained more than two pages of technical discussion in the area of infrared seekers.

Although Amtec is correct when it argues that its technical proposal and its response to the second round of discussion questions included material on infrared seekers, it misses the point of the evaluation report's criticism: that the claimed relevant past experience was discussed in a cursory way without discussion of the actual analyses, problem areas identified, and critical issues resolved. Instead of discussing the specifics of actual past analyses in a way that might have provided the agency with greater insight into Amtec's ability to handle analysis problems encountered during contract performance, Amtec presented a general discussion of parameters that influence image quality in an infrared seeker. The agency's observations about Amtec's proposal, supported by our review of both the technical proposal and the response to the evaluation questions, /2/ caused the evaluators to reasonably conclude that although Amtec's technical discussions "indicate familiarity with the subject matter the discussions are lacking technical depth." See ESCO, Inc., 66 Comp.Gen. 404, supra. Inadequate Review by Evaluator

Amtec contends that its evaluation was flawed because, according to Amtec, the evaluator that rated the proposal unacceptable in its technical approach failed to read all of the proposal. Amtec claims that the evaluator's failure to read the proposal can be deduced by the evaluator's criticisms of the proposal listed on that evaluator's worksheets-- criticisms that Amtec claims were answered in the proposal past the point where the evaluator allegedly stopped reading. One example cited by Amtec as evidence of the evaluator's failure to read the proposal is the comment that the proposal "did not discuss development of an error budget." /3/Although Amtec promised to prepare an error budget and discussed in some detail the need for an error budget-- a need the agency already saw fit to include within the statement of work at paragraph 3.1.1-- Amtec did not discuss how such a budget should be developed. As a result, we do not agree that the evaluator's critical comment lacks a reasonable basis, or is evidence that the evaluator did not read the proposal.

Amtec suggests that its proposal was unfairly evaluated because of evaluator or agency bias, or that the agency demonstrated bias in presenting the results of the evaluation. Our review of the evaluation records and the revised decisional documents indicates that the evaluation had a reasonable basis with no indication of bias on the part of the agency or its evaluators.

Adequacy of Reevaluation of Amtec's Proposal

Amtec argues that the Army's reevaluation of its proposal continues to lack the level of detail necessary to support a conclusion that the proposal deserves a rating of marginal. As we concluded in our prior decision on this procurement, agency evaluations must contain a detailed statement of the basis for the conclusions regarding the evaluation. Implicit in the foregoing is that these judgments must be documented in sufficient detail to show that they are not arbitrary. Waddell Eng'g Corp., 60 Comp.Gen. 11 (1980), 80-2 CPD Para. 269. Where there is not adequate supporting rationale in the record for the source selection decision, we will not conclude that the agency had a reasonable basis for the decision. Amtec Corp., B-240647, supra.

The Army, in revising its evaluation report based on the findings in our initial decision, now references several instances in Amtec's proposal where it concludes the proposal lacks detail and provides the reasons for that conclusion. Examples include: the lack of detail presented about Amtec's capabilities to perform a preferred intercept range determination based on missile/target trajectory; the lack of detail presented about actual past analyses performed regarding the infrared seeker, and the lack of detail presented about Amtec's technical expertise in radar technology with specific consideration of millimeter wave seeker performance under real world clutter. Amtec's disagreement with the evaluators' conclusions about such lack of detail does not render those conclusions unreasonable. ESCO, Inc., 66 Comp.Gen. 404, supra. In our view, the revised agency record now presents an adequate basis for the conclusion that the Amtec technical proposal lacked detail. Waddell Eng'g Corp., 60 Comp.Gen. 11, supra.

The protest is denied.

/1/ Included within Amtec's challenges are assertions already addressed and denied by our prior decision. An example is Amtec's allegation that the Army's cost realism adjustment to Amtec's proposed travel costs was improperly based on considerations not set forth in the solicitation. Our prior decision specifically denied this contention and Amtec's attempt to raise it again is essentially a request for reconsideration lodged nearly 4 months after our prior decision. As a result, this contention is now untimely. 56 Fed.Reg. 3,759 (1991) (to be codified at 4 C.F.R. Sec. 21.12(b)).

/2/ We note that Amtec argues in its protest that it provided the Army with the analysis it proposed for reviewing the infrared seeker in its May 31, 1990, response to the second round of written technical discussions. Although Amtec does, in fact, provide an eight-block flowchart diagramming its analytic approach ("A flowchart describing the analysis we propose is presented in Figure 2-5."), the evaluation report did not state that the analysis was not there, only that it was skimpy.

/3/ Amtec's challenge misquotes the evaluator. Instead of the quotation provided above, Amtec claimed the evaluator stated that the proposal "did not discuss an error budget." Such an observation, had it been made, would have been at odds with the discussion in Amtec's technical proposal.