Matter of: Cubic Defense Systems, Inc. and Metric Systems Corporation File: B-248649.2; B-248649.3 Date: September 25, 1992

Matter of: Cubic Defense Systems, Inc. and Metric Systems Corporation File: B-248649.2; B-248649.3 Date: September 25, 1992

PROCUREMENT Competitive Negotiation Contract awards Administrative discretion Cost/technical tradeoffs Technical superiority Protest against award to higher priced offeror, for upgrade of aircraft combat maneuvering instrumentation training range, is denied where: agency reasonably determined that awardee submitted the technically superior proposal, offering software improvements that would assure standardization and the documentation of software changes and facilitate future software maintenance and upgrades; the solicitation statement of evaluation criteria indicated that the technical criterion was more important than price; and the awardee's price was only approximately 2.9 percent higher than the low offer.

Attorneys

DECISION

Cubic Defense Systems, Inc. and Metric Systems Corporation protest the Department of the Air Force's award of a contract to Applied Data Technology, Inc. (ADTI), under request for proposals (RFP) No. F08626-92-R-0003, for the upgrade of the Air Combat Maneuvering Instrumentation (ACMI) training range at Tyndall Air Force Base in Florida. Metric and Cubic challenge the evaluation of technical proposals.

We deny the protests.

The solicitation requested primarily firm, fixed-price proposals to upgrade the Tyndall ACMI training range by: (1) replacing the 8-aircraft tracking system with a 36-aircraft tracking system, (2) replacing other existing equipment and adding new equipment, and (3) expanding the size of the training range to 2-1/2 times its present size by constructing 5 new ocean radio/microwave towers and 1 new land tower. The solicitation generally provided for award to be made to the offeror whose proposal indicated that it would accomplish the solicitation requirements "in a manner most advantageous to the Government, cost or price and other factors" considered. The RFP listed, in descending order of importance, the following "specific areas of evaluation": technical, cost/price, and management/logistics. In addition, the solicitation provided for proposals to be evaluated for "general considerations," including: (1) compliance with solicitation terms and conditions; (2) subcontract management; and (3) performance risk, based on the offeror's present and past performance as it relates to the probability of successfully accomplishing the proposed effort.

The Air Force received three proposals in response to the solicitation, including those submitted by ADTI, Cubic, and Metric; all three proposals were included in the competitive range. After conducting discussions with the offerors, the agency requested the submission of best and final offers (BAFO). The BAFOs were evaluated as follows:

ADTI CUBIC METRIC

Technical Exceptional Acceptable Acceptable Low Risk Moderate Risk Moderate Risk

Management/ Acceptable Acceptable Acceptable Logistics Low Risk Low Risk Low Risk

Past Low Risk Low Risk Low Risk Performance

Price/Cost $15,415,563 $15,215,583 $14,986,958

Based on these results, the source selection evaluation team (SSET) determined that ADTI had submitted the most advantageous proposal. Among the strengths which led agency evaluators to rate ADTI's proposal as exceptional/low risk under the determinative technical evaluation criterion was ADTI's proposal of automated tools for software development tracking, previously demonstrated by ADTI on another ACMI contract; according to the agency, these tools reduce human error, assure standardization, and ensure documentation of software changes.

In addition, the agency considered it a strength that ADTI had proposed software structural improvements developed for previous ACMI programs. In this regard, according to the agency, the proposed restructuring of the software code into a modular format would result in a more easily maintainable software package and simplify the adoption of future upgrades. The evaluators also considered ADTI's approach to the display and debriefing subsystem to be a strength. Although the solicitation provided for an advanced display and debriefing subsystem (ADDS) to be furnished to the awardee by the ADDS contractor (ADTI) as government furnished equipment (GFE), the solicitation included an option (contract line item number (CLIN) 0008) for procuring from the Tyndall contractor a less advanced display and debriefing subsystem (DDS). The evaluators viewed as a strength ADTI's commitment to furnishing the ADDS under CLIN 0008. In addition, the evaluators believed that even though the ADDS was to be provided as GFE, ADTI would possess an advantage in integrating the ADDS into the Tyndall ACMI as a result of its experience in developing the new subsystem. The overall SSET report also noted as strengths ADTI's detailed discussion of its approach to satisfying the specifications, its proposal to exceed the specifications in several significant respects-- including the required mean-time-between-failure--and what was believed to be ADTI's offer of all new hardware for the tracking instrumentation subsystem (TIS).

In contrast, while agency evaluators found several strengths in Metric's and Cubic's proposals under the technical evaluation criterion and considered the technical proposals to be acceptable, they also determined that the proposals offered a moderate degree of risk. Thus, while the evaluators viewed as a strength such aspects of Metric's approach as its proposal of new digital TIS elements and its estimate of only 30-45 days downtime for the Tyndall range, they also: (1) questioned the overall lack of detail concerning Metric's proposed approach; (2) found that in view of Metric's lack of experience in this regard, its proposal to build the digital TIS elements could cause a slip in schedule and a degradation of performance; and (3) concluded that Metric's determination to reuse certain existing TIS hardware, without providing for verification as to whether any needed to be refurbished or replaced, created a risk of increased maintenance cost and effort.

As for Cubic, the agency evaluators found its proposal risky and less desirable because it had failed to discuss in detail its approach to software development and configuration management, had proposed excessive downtime--15 weeks--for the Tyndall range, and did not plan to reverse engineer the current system, thereby leaving undetected any existing errors in the software documentation.

The source selection authority (SSA) concurred in the SSET's evaluation of proposals, finding that ADTI's proposal offered a lower risk and technically superior approach. He also noted that ADTI had recent, beneficial experience as a subcontractor for the development of ACMIs in Alaska and Wisconsin. The SSA concluded that these factors outweighed ADTI's slightly higher price/cost, which was approximately 2.9 percent more than the low offer. Upon learning of the resulting award to ADTI, Metric and Cubic filed these protests.

METRIC's PROTEST

Metric challenges various aspects of the evaluation of its and ADTI's proposals. Metric maintains that it, and not ADTI, submitted the technically superior proposal.

In reviewing protests against an agency's evaluation of proposals, it is not the function of our Office to independently evaluate those proposals. Signal Corp., B-241849 et al., Feb. 26, 1991, 91-1 CPD Para. 218. Rather, we will question the agency's technical evaluation only where the record shows that the evaluation lacks a reasonable basis or is inconsistent with the evaluation criteria listed in the RFP. Id.

We find no basis upon which to question the Air Force's overall determination that ADTI's proposal was technically superior to Metric's. In our view, the specific aspects of the evaluation claimed by Metric to be deficient either were reasonable exercises of the agency's judgment or were not significant enough to render the determination of ADTI's overall superiority arbitrary.

For example, Metric argues that the Air Force failed to take into consideration the higher reliability and ease of maintenance offered by its proposal of digital TIS elements to replace the current analog elements. As previously noted, however, the agency in fact viewed Metric's proposal of digital TIS elements as a strength. Nevertheless, it also determined that Metric's proposal of a system with which it lacked experience and for which the development and testing were incomplete posed a risk to the schedule and required performance. We find this conclusion reasonable, and see no basis upon which to question the agency's balanced approach to evaluating the advantages and disadvantages of Metric's proposed digital TIS elements.

Metric maintains that the Air Force did not take a similarly balanced approach to ADTI's proposed software modifications. According to the protester, the agency failed to take into consideration the technical risk arising from ADTI's proposal of extensive software modifications, amounting to the revision of approximately 30 percent of the existing lines of code, twice as much as proposed by Metric. As noted by the agency, however, ADTI had already implemented the proposed software structural improvements for the Alaska and Wisconsin ACMIs; the agency therefore concluded that there was little or no technical risk attendant upon ADTI's accomplishing the same changes for the Tyndall ACMI. In view of ADTI's prior experience in this regard and the resulting reduction in risk, we find reasonable the agency's conclusion that the significant advantages offered by the software changes outweighed any remaining risk.

Metric contends that the Air Force failed to properly evaluate offerors' approaches to reusing existing TIS hardware. Specifically, Metric points out that while the initial agency evaluation correctly noted that ADTI proposed the possible use of some TIS hardware, the SSET report described it as a strength of ADTI's proposal that it specified all new TIS hardware. Metric contrasts this unduly favorable treatment of ADTI's proposal with the agency's downgrading of its proposal for proposing to reuse TIS hardware.

Metric is correct, and the Air Force concedes, that ADTI was mistakenly given credit in the SSET report for proposing all new TIS hardware. Nevertheless, as noted by the agency, the two offerors in fact proposed different approaches in this regard that still support the conclusion that ADTI's approach was superior. While ADTI's proposal stated that existing "parabolic antennas and [radio frequency] cabling found to be serviceable may be retained and used" (emphasis added), Metric's proposal indicated that greater amounts of existing equipment--including ground-to-air, remote-to-master and remote-to-remote antennas, solar panels, batteries and battery boxes, and antenna cables--"will be retained." (Emphasis added.) Although Metric claims that its determination as to which existing equipment to retain was based on a recent site survey, the agency reports that it is unaware of any site survey in which the protester had the opportunity to closely inspect all of the existing equipment; the only authorized agency inspection was a cursory one confined to three of seven existing towers.

In any case, Metric neither mentioned in its proposal any inspection nor expressed a specific commitment to refurbish or replace worn out or inadequate equipment among the listed TIS hardware. (In this regard, we note that ADTI proposed to replace the existing solar arrays because they were inadequate to generate the power required for the new equipment.) Since ADTI's more cautious approach of considering the retention of less equipment and retaining that equipment only "if found to be serviceable," in fact offered greater assurance that adequate equipment would be in place at the conclusion of the contract, ADTI's approach reasonably could be viewed as more advantageous. That the SSET report somewhat mischaracterized ADTI's approach to existing TIS hardware does not warrant a different conclusion.

Metric also questions the Air Force's evaluation of the performance under the Alaska and Wisconsin ACMI contracts, for which Kollsman Instrument Company was the prime contractor and ADTI the subcontractor. Metric points out that nowhere do the evaluation documents indicate that the date of initial operating capability (IOC) for the Alaskan ACMI actually occurred 4 months after the date scheduled under the contract, and that the IOC date for the Wisconsin contract actually occurred 9 months late. The agency, however, reports that not only did ADTI perform its portion of the work on time and in accordance with the specifications, but further that, when Kollsman encountered problems with the integration and tuning of the TIS, ADTI assumed this responsibility and thereby prevented further program delays.

Further, in proposing Kollsman as the subcontractor to fabricate the Tyndall range TIS, ADTI explained that the effort required of Kollsman would be less demanding than that under the Alaska and Wisconsin contracts. Specifically, ADTI noted in its proposal that no redesign of the Wisconsin TIS would be required and that Kollsman would only be undertaking a "build-to-print" effort according to detailed specifications. In these circumstances, the agency could reasonably conclude not only that ADTI's experience under the Alaska and Wisconsin ACMI contracts was a strength but also that any risk resulting from the proposal of Kollsman as the TIS fabrication subcontractor had been minimized. (Even if Metric were correct that the proposal of Kollsman as a subcontractor should have given rise to greater concern for meeting the performance schedule, schedule was only one of five subcriteria under management/logistics, the least important evaluation criterion.)

Metric notes that the initial evaluation listed as a weakness ADTI's proposal of asymmetrical ocean radio/microwave towers, which the evaluators viewed as an inherently unstable design. In addition, the evaluators observed that since the antenna tower was not located directly above the support column, a substantially reinforced equipment deck would be required. Metric points out that the SSET report did not mention these weaknesses in ADTI's proposal; Metric claims that this failure is significant because ocean towers are subject to hurricane-force winds and waves generated by hurricane-force winds, which can cause the catastrophic failure of a tower.

According to the agency, and as noted by the agency evaluators in their evaluation documentation, these aspects of ADTI's proposal were considered only minor weaknesses; for this reason, they were not discussed in the SSET report. The agency maintains that an asymmetrical tower design, while seemingly unstable in the abstract, nevertheless may be as stable as a symmetrical design depending upon the nature of the ocean floor, as yet unknown, and the consequent required design of the subsurface structure of the tower. As for placement of the antenna away from the support column, the agency reports that while this will necessitate a reinforced equipment deck, a similar platform design has previously proved satisfactory.

We find no basis in the record for concluding that had the evaluators brought their concerns with the asymmetrical tower design to the attention of the SSA, it would or should have altered the source selection decision. On the contrary, notwithstanding any minor deficiencies in the source selection process, our review of the record supports the reasonableness of both the Air Force's determination that ADTI submitted the technically superior proposal, offering significant advantages from the proposed software approach, and the agency's consequent conclusion that this superiority under the most important evaluation criterion outweighed the minimal additional price of ADTI's proposal.

Metric's protest therefore is denied.

CUBIC PROTEST

Cubic contends that the Air Force solicited offers on the basis of one set of circumstances and then improperly failed to amend the solicitation when those circumstances changed. [1] Specifically, Cubic notes that the agency reports that it included CLIN 0008, the option for requiring the contractor to furnish a new DDS, as a contingency measure because it was uncertain whether the more advanced ADDS (which the specifications provided would be furnished as GFE) would be completed by ADTI, the ADDS contractor, in time to be furnished as GFE for the Tyndall contract. Cubic further notes, however, that according to the agency, after the receipt of proposals, agency evaluators discussed deleting CLIN 0008 because of their confidence that ADTI would in fact be able to provide the ADDS in time and that it therefore would not be necessary to use the DDS. Cubic questions the subsequent failure to amend the solicitation to delete CLIN 0008 when it appeared that it would not be necessary to exercise the option for the DDS; it argues that this failure resulted in a skewed evaluation and thus rendered the award to ADTI improper.

We disagree. The Air Force maintains (and the record does not show otherwise) that, notwithstanding any likelihood that the ADDS would be available on time, the need for the DDS option to cover the possibility that it would not be available in fact continued after the receipt of initial proposals. The agency reports that as late as May 28, 1992, 4 weeks after the award to ADTI, the ADDS was still in the early stages of production and it remained possible that the ADDS would not be available in time to be furnished as GFE for the Tyndall ACMI contract. Indeed, Cubic itself notes that, as of September 1, the date for the critical design review on the ADDS has been postponed twice; the protester suggests that the ADDS in fact may not be ready in time to meet the scheduled February 1993 delivery date. In our view, given the possibility then (and now) of a delay in the production of the ADDS, and the fact that CLIN 0008 was intended as a hedge against this eventuality, the agency's need for the protection afforded by CLIN 0008 continued and its determination to retain that option in the solicitation was reasonable.

Cubic further contends that "giving ADTI additional points for furnishing an item [the ADDS] that was to be furnished as GFE appears to be senseless" since the ADDS was to be furnished as GFE to the successful offeror no matter who it was. As noted by the Air Force, however, given ADTI's role in developing the ADDS, that firm was in the best position to ensure the successful integration of the ADDS into the overall system. In our view, the agency therefore could reasonably view ADTI's role as the developer of the ADDS as a strength of its proposal.

Cubic also argues that the Air Force misevaluated its proposal with respect to reusing existing TIS hardware. Cubic notes that the evaluator's item summary characterized as a weakness the fact that Cubic "proposed using existing hardware without explaining if any would be refurbished." Cubic maintains that, reasonably interpreted, its proposal in fact did not provide for the reuse of existing TIS hardware, but instead specified the use of new hardware.

Although the Air Force maintains that Cubic's proposal is, at best, ambiguous with respect to reuse of existing TIS hardware, we need not resolve this dispute over the interpretation of the proposal. The SSET report contained no mention of this alleged weakness and nothing in the record indicates that it affected the decision to make award to ADTI. In any event, given the reasonably perceived technical superiority of ADTI's proposal, offering significant advantages as a result of its proposed software approach, the stated weaknesses in Cubic's proposal, and the minimal evaluated price/cost difference between the proposals, with the cost/price of ADTI's proposal only 1.3 percent higher, it does not appear that any deficiency in the relative evaluation of ADTI's and Cubic's proposals resulted in competitive prejudice to Cubic.

The protests are denied.

1. After receiving the agency report responding to its initial objections to the evaluation, Cubic declined to rebut the agency's explanation of several of the challenged aspects of the evaluation. To the extent that it has failed to further address these areas, we consider them abandoned. See Eyring Corp., B-245549.7, Mar. 31, 1992, 92-1 CPD Para. 320.