B-238420, B-238420.2, May 24, 1990, 90-1 CPD 497

B-238420, B-238420.2, May 24, 1990, 90-1 CPD 497

PROCUREMENT - Competitive Negotiation - Contract awards - Administrative discretion - Cost/technical tradeoffs - Technical superiority DIGEST: 1. Award to higher-priced offeror is unobjectionable where technical considerations were more important than cost under the solicitation and agency reasonably concluded that technical superiority of awardee's proposal was worth the additional cost. PROCUREMENT - Competitive Negotiation - Offers - Risks - Evaluation Technical acceptability 2. Consideration of technical risk in evaluating proposals is unobjectionable since, although not specified as an evaluation criterion in the solicitation, technical risk is reasonably related to the specified technical evaluation criteria.

Attorneys

GP Taurio Inc.:

GP Taurio, Inc. protests the Department of the Air Force's award of a contract to Allen Communication, Inc., under request for proposals (RFP) F44650-89-R-0017, for development of maintenance continuation training courses for F-15 and F-16 aircraft. Taurio challenges the evaluation of proposals and questions the determination to make award at a price significantly higher than Taurio's.

We deny the protests.

The solicitation requested proposals for the delivery of interactive maintenance continuation training courses for either or both the F-15 and F-16 aircraft; it specified 10 basic and 6 optional courses for each aircraft and stated that award of the optional courses would depend upon the availability of funds. Under the first phase of the contemplated contract performance, the contractor is required to perform an in-depth "front-end analysis" of existing courseware, establish minimum core tasks that each skill level must be capable of performing, and then ascertain the unmet training requirements to be satisfied under the contract. Part of the front end analysis involves analysis of F-15/F-16 maintenance tasks using a relatively new methodology called cognitive task analysis, which emphasizes the development of troubleshooting skills. The specification requires the contractor then to develop a knowledge assessment test (KAT) capable of measuring a technician's knowledge of the required core tasks so as to allow tailoring of course presentation to address each student's deficiencies. Finally, the specification calls for actual development of the interactive courseware, during which phase the front end analysis results are to be applied to the development of actual computer- and video -disc-based training courses.

The RFP provided for award to be made to the responsible offeror whose proposal was determined to be the most advantageous to the government, price and other factors considered; it specified that technical and management factors would be more important than price, and provided for prices to be evaluated by adding the total cost for the basic and optional courses. Under the technical/management factor, the solicitation listed in descending order of importance five evaluation criteria, including capabilities demonstration (with an undisclosed weight of 30 percent), program management (25 percent), compliance with requirements (20 percent), corporate background and experience (15 percent), and personnel qualifications (10 percent).

Five of thirteen proposals received in response to the solicitation were determined to be technically acceptable and were included in the competitive range; of these, Allen's received the highest technical score of 8.5 out of 10 possible points, and Taurio's received a score of 5.1. After conducting written discussions, the agency requested the submission of best and final offers (BAFOs). Allen's BAFO technical/management score of 8.7 remained the highest; Taurio's score of 5.3 was the lowest. Although Allen's offered price ($6,171,160 for 32 courses) was substantially higher than Taurio's ($3,776,134), the contracting officer determined that Allen's substantially higher-rated proposal offered the best value and least risk to the government, and made award to Allen for the 20 basic courses plus one optional course on January 16, 1990.

In its protest, Taurio essentially alleges that the technical and cost evaluations of its proposal were erroneous, and that in any event, Allen's price was so much higher than Taurio's that Allen should not have received the award regardless of technical superiority.

The determination of the relative merits of proposals is primarily a matter of agency discretion which our Office will not disturb unless it is shown to be unreasonable. Systems & Processes Eng'g Corp., B-234142, May 10, 1989, 89-1 CPD Para. 441. A protester's mere disagreement with the agency's judgment does not render that judgment unreasonable. Id.

The Air Force identified a number of significant weaknesses in Taurio's proposal. For example, in the most heavily weighted area, the live capabilities demonstration, while the agency rated Allen's demonstration as "very outstanding," Taurio's demonstration exhibited a number of weaknesses. Specifically, while Taurio's demonstration showed a general ability to meet the requirements of the performance work statement and a basic understanding of the required design techniques, its computer generated graphics presentations were found to be inconsistent in format and difficult to read, its demonstration of basic design techniques was considered marginal, and it failed to describe how the design techniques would be used to develop interactive courseware at the required levels of analysis. We note that Taurio has not refuted these conclusions.

Another serious concern of evaluators was Taurio's "fragmented" management approach. Taurio proposed to conduct its development effort from two locations: task analysis at Norfolk, Virginia, close to the contracting activity and Taurio's subcontractor, and knowledge assessment test and courseware development at Pensacola, Florida, close to three Air Force bases where F-15s and F-16s are located. While Taurio stated that this approach would allow it to devote less time and expense to travel and more to product development, the technical evaluation board (TEB) during discussions expressed concern that the task analysis and the knowledge assessment test and courseware development efforts would be performed by different personnel. Specifically, the agency viewed Taurio's approach as hindering the necessary continuity and consistency between various phases of the project. The agency ultimately found, reasonably in our view, that Taurio's subsequent proposal of high-level coordination did not fully address this concern and that therefore Taurio's approach posed a high degree of risk to the success of the overall effort.

Cognitive task analysis includes development of paradigms-- patterns of behavior-- by subject matter experts. While Allen proposed to rely on at least three subject matter experts for the development of a knowledge assessment test for each course, Taurio proposed one subject matter expert to assist in developing paradigms for the test. Taurio, when questioned about its approach during discussions, maintained that a single expert performer would be sufficient, but held out the possibility that during performance, Taurio would consider additional experts if requested by the agency. Agency evaluators determined that more than one subject matter expert was necessary to assure a complete analysis. Taurio now claims that it proposed more than a single subject matter expert for each course. Nothing in Taurio's proposal indicates that it would rely upon more than one expert, and we find no basis to question the agency's conclusion that Taurio's approach was significantly less advantageous in this regard.

Taurio's proposal was also rated deficient concerning its understanding of the in-progress review process, by which the Air Force would review courseware during development. The performance work statement outlined the sequence that the IPR would follow, which included incorporating changes recommended by the agency and supplying the agency with the final version. Taurio indicated in its proposal that the revised version would be submitted to the agency for a final review, and that development would not proceed until approval was given. The TEB found that Taurio's extra review step and the delay while awaiting notice to proceed would adversely affect the development schedule and increase the risk to the government, and asked Taurio to reconsider its approach in that light. Taurio responded by justifying its original approach to the requirement, thus confirming to the Air Force that it did not understand the requirement of the performance work statement. While Taurio now suggests that it proposed merely to furnish the government a copy of the revised version, we find reasonable the agency's conclusion that Taurio's proposed approach entailed a delay pending government approval and thereby was likely to adversely affect the project schedule.

Although Taurio asserts that the agency's consideration of the technical risk of its proposal was improper because technical risk was not a stated evaluation factor, this argument is without merit. We generally will not object to the use of evaluation factors not specifically stated in the RFP where they are reasonably related to the specified criteria. Consolidated Group, B-220050, Jan. 9, 1986, 86-1 CPD Para. 21. An offeror's proposed approach in any technical area involves a certain level of risk, and the degree of risk is a factor reasonably related to the technical evaluation criteria, and therefore properly considered here by the agency. Id.

Based upon the reasonably perceived weaknesses in Taurio's proposal, examples of which are discussed above, we find no basis to question the Air Force's determination that Allen offered a technically superior approach to contract performance.

Furthermore, we disagree with Taurio's view that the evaluation gave insufficient weight to price, and instead improperly relied on technical scores. Agency officials have broad discretion in determining the manner and extent to which they will make use of the technical and cost evaluation results; cost/technical tradeoffs may be made subject only to the test of rationality and consistency with the established evaluation factors. Litton Indus., Inc., B-236720, Dec. 26, 1989, 89-2 CPD Para. 595.

Here, the solicitation stated that technical and management considerations would be more important than cost. As previously explained, the TEB found (reasonably, we have concluded) that Allen's proposal was far superior in all technical areas to the other proposals in the competitive range and carried the least amount of risk. The cost evaluation panel found that Allen's higher price was supported by its technical approach, while Taurio's lower price was commensurate with its inferior technical approach and higher degree of risk. In these circumstances, we find that the contracting officer reasonably determined pursuant to the stated evaluation criteria that Allen offered the best overall value to the government.

We recognize that it does appear that the Air Force may not have considered the option prices in evaluating cost proposals, contrary to the precise evaluation scheme in the RFP. However, even if this was the case, Taurio clearly was not prejudiced as a result. There is a 62.2 percent difference between Taurio's price and Allen's price for the 20 basic courses; the difference between the two firms' prices for all 32 basic plus option courses is 63.4 percent.

Since evaluation based on either the price for the basic courses or the price for all courses would have yielded virtually the same result, we have no reason to disturb the award on this basis. See Science Applications Int'l Corp., B-232548, Jan. 23, 1989, 89-1 CPD Para. 52.

The protests are denied.