Matter of: DUAL, Inc.[*] File: B-252593.3 Date: August 31, 1993 93-2 CPD 190

Matter of: DUAL, Inc.[*] File: B-252593.3 Date: August 31, 1993 93-2 CPD 190

PROCUREMENT Competitive Negotiation Contract awards Administrative discretion Cost/technical tradeoffs Technical superiority Protest against award to higher priced, higher technically rated offeror is denied where the solicitation evaluation scheme gave greater weight to technical merit than to price; agency reasonably determined that awardee's proposal was technically superior to protester's; and agency reasonably concluded that protester's lower price did not outweigh technical advantages of awardee's proposal. REDACTED VERSION

Attorneys

DECISION DUAL, Inc. protests the award of a contract to Industrial Data Link Corporation (IDL) under request for proposals (RFP) No. N61339-92-R- 0002, issued by the Naval Training Systems Center, Department of the Navy, for the design and production of a simulation trainer for United States Army National Guard and Reserve M1 tank crews. DUAL challenges the evaluation of technical and cost proposals and argues that the agency failed to perform a proper price/technical tradeoff in order to support award based on IDL's higher rated, higher priced proposal.

We deny the protest in part and dismiss it in part.

BACKGROUND

The trainer to be produced under this procurement is the Guard Unit Armory Device Full Crew Interactive Simulation Trainer for Armor, known as the GUARDFIST I System, for which a prototype system had been developed under a previous contract. It will enable crew members to assume their actual stations in the tank and be presented with realistic training scenarios designed to simulate true combat situations.

The RFP requested firm, fixed prices for 6 trainers, with options for up to 261 additional units, along with associated data and support. Award was to be made to the offeror submitting the conforming proposal determined to be most advantageous to the government based upon consideration of evaluation factors for technical, price, and past performance; technical was more important than the other factors combined and price was more important than past performance. The technical factor was divided into subfactors for system design, integrated logistics support (ILS), and management, with system design being significantly more important than the other two subfactors combined, and ILS slightly more important than management. The system design subfactor, the evaluation of which is at issue here, included three sub-subfactors: computer system design, visual system, and hardware design, with the first area equal in importance to the remaining two combined, and the remaining two areas of equal importance.

The solicitation provided for the technical elements to be color rated as blue/exceptional, green/acceptable, yellow/marginally acceptable, or red/unacceptable. In addition to technical merit, proposals were to be assessed for proposal risk, encompassing technical and schedule risk for each technical element; proposal risk was less important than technical merit. Under past performance, proposals were to be assessed for performance risk under the ratings of superior/blue, good/green, adequate/yellow, and inadequate/red. Finally, the RFP provided a detailed method for the evaluation of price.

The Navy received proposals from seven offerors, including DUAL and IDL. After the initial technical evaluation, discussions (which included in- plant demonstrations of the visual systems), and two rounds of best and final offers (BAFO), the evaluation panel individually evaluated each proposal. The evaluators then met and agreed to a consensus evaluation rating for each proposal. IDL's technical proposal was determined exceptional overall, while DUAL's was rated acceptable. IDL's overall exceptional rating was based on its exceptional rating with respect to system design, which was the most important technical subfactor; DUAL was rated acceptable under this subfactor. The two proposals otherwise were evaluated relatively close: both proposals were rated acceptable under the remaining technical subfactors, ILS and management; IDL's proposal was rated the lowest risk of all proposals, and DUAL's was rated second lowest; and both proposals were rated adequate in the past performance risk assessment. Although IDL's price was evaluated at $28,095,221, 13 percent higher than DUAL's $24,413,774 evaluated price, the Navy concluded that IDL's technical superiority made its proposal the most advantageous to the government and on this basis made award to IDL. (Subsequent to the award, the agency corrected an error in the price evaluation that increased the difference between IDL's evaluated price ($28,073,986) and DUAL's ($23,822,882) to 18 percent. However, in a supplemental source selection statement, the source selection official concluded that IDL's proposal remained most advantageous to the government.)

DUAL challenges numerous aspects of the evaluation, but primarily questions the evaluation of its proposal under the computer system design and visual system sub-subfactors (under the system design subfactor), and IDL's proposal under the hardware design sub-subfactor. DUAL argues that its proposal should not have been rated below IDL's in these areas. Had DUAL received the same ratings as IDL under the system design subfactor, DUAL asserts, the two firms' overall technical ratings would have been equal and DUAL would have received the award based on its low price.

In reviewing a protest against the propriety of an evaluation, it is not our function to independently evaluate proposals and substitute our judgment for that of the agency. General Servs. Eng'g, Inc., B-245458, Jan. 9, 1992, 92-1 CPD Para. 44. Rather, we will review a technical evaluation only to assure that it was reasonable and consistent with the evaluation criteria in the RFP. Id.

We have reviewed all of DUAL's arguments and, based on the record, conclude that the evaluation was reasonable. We discuss several of the arguments below.

TECHNICAL EVALUATION

Computer System Design

The computer system design sub-subfactor consisted of the following five elements (of equal importance): (1) efficiency of the proposed software design/approach, (2) upgradeability for future weapon system changes, (3) maximization of diagnostic software and minimization of manual fault isolation for the user replaceable units, (4) reliability of the proposed software design, and (5) maturity of the software design. While IDL's proposal received exceptional ratings under all five elements, DUAL's was rated marginal under efficiency of software design and maturity of software design, and acceptable under the remaining three elements. DUAL maintains that because its proposal was evaluated to have certain strengths and offered some of the same features IDL's proposal offered, its own proposal merited the same exceptional rating IDL's received.

This argument is without merit. While DUAL's proposal was evaluated as having certain strengths and some of the same features as IDL's, this does not by itself establish that DUAL's proposal should have been rated as highly. In this regard, the record shows that DUAL's lower rating was based on evaluated weaknesses in the firm's proposal and a level of performance lower than that proposed by IDL, while IDL's higher rating was based on features unique to its proposal.

For example, under the maturity of software design sub-subfactor, DUAL's proposal was rated marginal and its software design was evaluated as immature, essentially because it utilized a new computer system that was not commercially released at the time of the offer and therefore had no proven track record. In this regard, DUAL's proposal indicated the need for new development of software code, estimated by the agency at 48,170 lines. As a result of the current unavailability of its proposed system, DUAL was unable to demonstrate (during the visual system demonstration) the system it intended to deliver, including software; it instead demonstrated a different system. Also, during discussions concerning software design, DUAL was unable to provide a requested validation certificate for its system configuration. In contrast, IDL's proposal was rated exceptional under the maturity of software design subsubfactor because its proposed software design was judged to be a mature design; over 95 percent of its software components were to be taken from the existing, proven prototype system, so that only an estimated 18,190 new lines of code would have to be developed.

DUAL does not deny that its software was new and had no proven performance at the time of its offer. DUAL instead primarily contends that its software design should have been considered mature because the firm proposed using advanced software development approaches (such as object oriented analysis). Although the record confirms that the Navy recognized DUAL's proposed "use of modern software engineering principles" as an advantage, the agency did not consider this to alleviate its concern with the lack of a mature design. We find no basis to question the Navy's conclusion in this regard. In our view, the agency could reasonably conclude that DUAL's strong approach to software development did not eliminate the uncertainties and risk inherent in its offer of unreleased, unproven software, and that Dual's offer of a less mature software design warranted a lower rating under the maturity of software design sub- subfactor than did IDL's proven software design.

Visual System

The visual system, known as the image generation and display system, was generally required to compute and display scenes consisting of surfaces, known as polygons, in true perspective. Specifically, it was to provide "real-time out-of-the window and through-the-sight visual display of simulated tank movement battle action and surrounding environment." The RFP's performance specification contained numerous requirements for the visual system, as well as preferred enhancements--preferred requirements-- for more realism, such as additional moving targets and polygons.

DUAL argues that its proposal, which was rated acceptable in this area, should have been rated equal to IDL's, which was rated exceptional, because, it maintains, there were no substantial differences between the proposals and it offered essentially the same enhancements as IDL under three of the five visual system elements--realism of the visual system to provide training (which was twice as important as the other four sub- subfactors), real-time processing and visual system major component characteristics. Alternatively, DUAL argues that the higher rating received by IDL in this area was for capabilities beyond those required, indicating that the Navy improperly evaluated proposals on the basis of unstated evaluation criteria. In this regard, DUAL further maintains that it was misled by the RFP warning to offerors that there was an economic breakpoint beyond which enhancements would be of marginal value.

We find that the record reasonably supports the lower rating of DUAL's proposal in the visual system area. Contrary to DUAL's assertion, the record indicates that there in fact were substantial differences between the two proposals. While DUAL's proposed visual system offered capabilities above the minimum requirements, the record indicates that DUAL proposed only a relatively minimal increase in performance level. In contrast, IDL's proposed system offered significant increases in performance, providing increased complexity and more realism to the training, thus meriting an exceptional rating. As noted by the agency, "IDL's major component characteristics are better able to depict a realistic training battlefield with more detail and in a more life-like environment."

Specifically, with respect to realism to provide training, the most important element under the visual system sub-subfactor (and twice as important as the other four elements), DUAL proposed a lesser increase in performance in the areas of moving targets and texture elements than did IDL. The RFP required 3 moving targets and 0.8 million texture elements (i.e., non-terrain and geo-typical image data) and stated a preference for more in both areas. While IDL offered 24 moving targets and 4.1 million texture elements, substantial increases from the requirements, DUAL offered only 5 moving targets and 1.5 million texture elements, minimal increases above the requirements. Likewise, while DUAL offered only two levels of illumination, day and night, the minimum required by the RFP, IDL offered continuously variable illumination. The Navy found IDL's approach superior because being able to simulate any time of day or night would "allow for greater flexibility and variety in the time of day a scenario can be presented." Given IDL's offer of substantially enhanced performance relative to the requirements, particularly in areas where the RFP expressed specific preference for increased performance, we believe the agency reasonably considered DUAL's offer of a minimal increase in performance relative to the requirements to be a less advantageous approach warranting a lower rating than IDL's.[1]

Further, the agency's evaluation of performance capabilities beyond the minimum required by the RFP was unobjectionable. Where detailed technical proposals are sought and technical evaluation criteria are used to enable the agency to make comparative judgments about the relative merits of competing proposals, as here, offerors are on notice that qualitative distinctions among technical proposals will be made under the various evaluation factors. Cybernated Automation Corp., B-242511.3, Sept. 26, 1991, 91-2 CPD Para. 293. Consequently, as offerors were on notice from the solicitation here of a comparative evaluation of the relative merits of proposals, IDL's higher rating for offering increased performance capabilities was entirely proper, particularly in light of the RFP's stated preference for increased capabilities in the areas in question, such as moving targets and texture elements. See Cardkey Sys., Inc., B-239433, Aug. 27, 1990, 90-2 CPD Para. 159.

Moreover, contrary to DUAL's further argument, we see no indication that the solicitation misled offerors as to the evaluation benefit of offering enhancements. In this regard, DUAL quotes section 3.1.1.3.1.1 of the specifications, which stated:

"The visual appendix to the specification represents the baseline performance necessary to provide the required training. Minor deviations up to 10 percent from the specified visual performance may be allowable where the requirements exceed the inherent capability of a proposed system or subsystem capability provided there are comparable compensating areas where the specification requirements are exceeded. . . . The intent of this provision is to permit designs to optimize technical performance which does not exceed an economic breakpoint where the additional performance is of minor value."

In our view, this provision, read as a whole, did not preclude or discourage the offer of superior performance, but instead permitted minor negative deviations from the requirements if they were offset by enhancements in other areas. Indeed, the solicitation stated a preference for enhanced performance in certain areas, such as moving targets and texture elements, at issue here, and DUAL itself offered somewhat enhanced performance in these areas. Consequently, we see nothing in the record which indicates that the evaluation was conducted on a basis other than the criteria stated in the RFP or that the RFP misled offerors.

Hardware Design

Under the hardware design sub-subfactor, IDL's proposal was rated exceptional, based on exceptional ratings in five of nine elements and acceptable ratings in the remaining four. DUAL argues that IDL's exceptional consensus ratings under the elements for minimization of components, design for maximization of operator replaceable units, and realism of hardware to provide training, were unsupported by the individual evaluator worksheets. DUAL maintains that IDL instead should have been rated as acceptable for these elements because three of the four individual evaluator worksheets documented only acceptable ratings; according to DUAL, the head evaluator "simply overruled the individual evaluators and imposed his own minority views" for "exceptional consensus" ratings.

We find no basis to question the evaluation of IDL's proposal under the hardware design sub-subfactor. It is proper for technical evaluators to discuss the relative strengths and weaknesses of proposals in order to reach a consensus rating, which often differs from the initial ratings given by individual evaluators. Syscon Servs., Inc., 68 Comp.Gen. 698 (1989), 89-2 CPD Para. 258; Schweizer Aircraft Corp., B-248640.2 et al., Sept. 14, 1992, 92-2 CPD Para. 200. The overriding concern in the evaluation process is that the final assigned scores/ratings reflect the actual merits of the proposals, not that they be mechanically traceable back to the scores/ratings initially given by the individual evaluators. JWK Int'l, Inc., B-251125, Mar. 4, 1993, 93-1 CPD Para. 205. Consequently, the fact that some of the evaluators initially rated IDL somewhat lower in this area does not by itself warrant questioning the final evaluation results. (Indeed, even if the final consensus ratings were not agreed to by each individual evaluator, this would not provide a sustainable basis of protest in the absence of a showing that the final ratings did not accurately reflect the merits of the proposal.) Since DUAL has not shown that the final assigned ratings did not reflect the actual merits of the proposals, we will not object to the evaluation in this regard.

In any case, even if IDL had been rated acceptable rather than exceptional under the hardware design sub-subfactor, the record shows IDL's higher rating in the system design area was reasonable. IDL's offer was rated exceptional under the computer system design sub-subfactor, which was significantly more important than the remaining sub-subfactors for visual system and hardware design, and also exceptional under the visual system sub-subfactor. In contrast, DUAL's offer was rated as only acceptable under each of the three system design sub-subfactors. Consequently, even if IDL's hardware design had been rated only acceptable, as DUAL argues it should have been, this would not have changed IDL's overall system design rating given the firm's exceptional ratings in the remaining two sub-subfactors, one of which was weighted significantly more important than the other two areas combined.[2]

PAST PERFORMANCE EVALUATION

DUAL argues that the Navy improperly evaluated IDL's past performance as indicating only a moderate performance risk; it asserts that the firm's proposal should have been rejected based on its unacceptable past performance.

We disagree. The RFP provided for an evaluation of past performance with respect to three areas of equal importance: (1) ability to meet technical requirements, (2) ability to perform on schedule, and (3) ability to propose and implement contract "changes". The initial assessment was made by the Performance Risk Analysis Group (PRAG) based on information obtained on the offeror's past performance on government contracts. IDL was rated as follows under this factor: (1) technical--good; (2) schedule- -inadequate; and (3) changes--adequate. These ratings were averaged for a final rating of adequate, the same rating received by DUAL.

We find no basis for questioning IDL's rating. The final advisory council evaluation report indicates that the PRAG chairman, noting that IDL had received positive, as well as negative, reviews with respect to schedule, specifically determined that the overall adequate rating under past performance accurately reflected the firm's performance risk. According to the report, the chairman determined that IDL's performance history indicated only "some," and not "significant," risk. DUAL has not shown this judgment to be unreasonable. In any event, nothing in the solicitation indicated that an inadequate rating in one of the areas would be grounds for rejection of the entire proposal rather than merely a downgrading in the comparative evaluation.

PRICE EVALUATION

DUAL argues that the agency failed to evaluate the total price of all option trainers to be purchased which, according to the protester, would have shown that the total cost of IDL's proposal represented a 50 percent premium over the cost of DUAL's proposal. The RFP requested prices for base quantities and additional option quantities of trainers, as well as associated data and support. It further advised offerors that the total price for evaluation purposes would be calculated essentially by adding the base line item prices to option prices for the mid-range quantities in the step-ladder option quantities set forth in the RFP. As this is precisely how the evaluation was conducted (and DUAL does not argue otherwise), the evaluation was proper. If DUAL believed that the entire option quantity should have been included in the price evaluation, it was required to protest on that ground prior to the initial closing time for receipt of proposals. See 4 C.F.R. Sec. 21.2(a) (1993).

PRICE/TECHNICAL TRADEOFF

DUAL challenges the price/technical tradeoff on the ground that it was based on an unreasonable technical evaluation. Agencies have the discretion, in making their source selection decisions, to trade cost savings for technical benefits, with that discretion constrained only by the requirement that the agency's determination be rational and consistent with the solicitation's evaluation criteria. Grey Advertising, Inc., 55 Comp.Gen. 1111 (1976), 76-1 CPD Para. 325. Award may be made to a higher rated, higher cost offeror where the decision is consistent with the evaluation factors and the agency reasonably determines that the technical superiority of the higher cost offer outweighs the cost difference. See General Servs. Eng'g, Inc., B-245458, Jan. 9, 1992, 92-1 CPD Para. 44.

We find no basis in the record to question the tradeoff here. IDL received an exceptional technical rating compared to DUAL's adequate rating, and the solicitation made the technical rating more important than the price and past performance factors combined. As discussed above, DUAL has not shown the overall determination of IDL's technical superiority to be unreasonable. In these circumstances, we conclude that the agency could reasonably determine that under the stated evaluation criteria the technical superiority of IDL's proposal was worth its approximately 18 percent higher price.[3]

The protest is denied in part and dismissed in part.

* The decision issued on August 31, 1993, contained proprietary information and was subject to the terms of a General Accounting Office protective order. It was released to the parties admitted to the protective order. The parties have agreed that this decision should be released in its entirety; the decision is now removed from the coverage of the protective order.

1. While DUAL offered a greater number of polygons than IDL, the source selection authority determined that there was doubt as to the extent to which the additional polygons could be utilized under the proposed system's scene content management scheme, which was only emulated during the demonstration. Also, the source selection authority determined that the use of the additional polygons could be hindered by the systems's minimum depth complexity--once the screen has been overwritten three times, it cannot draw any more pictures. Consequently, the effect on actual performance of DUAL's proposed increase in the number of polygons was uncertain. DUAL does not directly address this uncertainty.

2. DUAL further argues that the Navy improperly rated the firm's proposal under two of the nine equally weighted elements under the sub-subfactor for hardware design as acceptable when they instead should have been rated as exceptional. We need not discuss this allegation because it would not affect the outcome of the evaluation. Even if DUAL's offer had been rated exceptional in the two areas at issue, this would not have been reasonably adequate to change the firm's overall rating to exceptional (IDL's overall rating), since the sub-subfactor for hardware design was comprised of nine equally weighted elements, only two of which are at issue, and the firm's offer was rated as only acceptable or marginal under the remaining seven elements.

3. DUAL argues that the source selection decision was flawed because only after award did the agency take into account (in the revised source selection statement) the fact that the corrected price difference between the proposals was 18 percent, rather than the 13 percent on which the tradeoff initially was conducted. In addition, DUAL notes that the agency never quantified the dollar value to the agency of the evaluated superior features of IDL's system. However, where a source selection official provides an after-the-fact discussion of his price/technical tradeoff, either to supplement or explain the selection decision, we will not object to the tradeoff if it is consistent with the evaluation criteria in the solicitation and otherwise is supported by the record. Saco Defense, Inc., B-252066, May 20, 1993, 93-1 CPD Para. 395. As discussed above, there is no basis for questioning the tradeoff under the circumstances here. Further, the agency was not required to quantify the exact dollar value of the key features of IDL's proposal. See Picker Int'l, Inc., B-249699.3, Mar. 30, 1993, 93-1 CPD Para. 275.