B-231613, Sep 20, 1988, 88-2 CPD 267

B-231613, Sep 20, 1988, 88-2 CPD 267

PROCUREMENT - Competitive Negotiation - Contract awards - Administrative discretion - Cost/technical tradeoffs - Technical superiority DIGEST: 1. Where solicitation for test sets for the maintenance of aviation night vision devices provided that technical merit would be more important than price and emphasized the importance of simplicity of design, contracting agency did not act unreasonably in selecting for award a slightly higher- price proposal (lower-priced based on life cycle cost) offering a less complex design (with fewer parts of low or moderate reliability) and a superior ability to test for inadequate night vision devices. PROCUREMENT - Competitive Negotiation - Discussion - Adequacy - Criteria 2. Where perceived weakness in the protester's design, that it was unnecessarily complex and included too many parts of low or moderate reliability, was inherent in the design itself rather than in any failure to explain the design, and a significant improvement would require a complete redesign, than it does not appear that any lack of detail in the notice of the weakness provided during discussions deprived the protester of an opportunity significantly to improve its proposal.

Kollsman Instrument Company:

Kollsman Instrument Company protests the award of a contract to Hughes Optical Products, Inc., under request for proposals (RFP) No. DAAB07-87-R- F068, issued by the U.S. Army Communications-Electronics Command (CECOM), for special purpose test sets required for the maintenance of night vision systems. Kollsman challenges the evaluation of proposals and contends that the agency failed to conduct meaningful discussions concerning perceived weaknesses. We deny the protest.

The RFP solicited proposals for test sets used to determine whether aviation night vision systems have reached the end of their useful service life. The solicitation listed the primary evaluation criteria, in descending order of importance, as technical, cost/price, and production/management. Under the factor for technical approach, the most important technical subcriterion, the solicitation provided for consideration not only of whether the proposed technical approach offered a high level of assurance of successful performance, but also of the simplicity of the design and the extent to which it would facilitate operation, maintenance and training. The solicitation called for fixed prices, but also provided for the evaluation of estimated life cycle costs.

CECOM received four proposals and conducted written and oral discussions with all offerors. After requesting and evaluating best and final offers, the agency ranked Kollsman first under the criteria for cost/price and production/management. Kollsman's fixed price for the planned base production quantities of hardware ($4,293,678) exceeded the Hughes price ($3,978,278), but since Hughes' price for rights in technical data was higher than Kollsman's, Hughes' overall evaluated fixed price ($5,037,268) was 3 percent higher than Kollsman's ($4,888,678). /1/ Kollsman's proposal also was found to possess significant strengths with respect to production, including a more comprehensive quality control program and more extensive reliability testing than proposed by Hughes. CECOM determined, however, that Kollsman's slight fixed-price advantage and its relative strength under the least important evaluation criterion (production/management) were offset by the superiority of the Hughes proposal under the most important criterion, technical merit.

Hughes' technical advantage came primarily in two areas. First, although both Kollsman and Hughes redesigned the current generation of test sets to reduce errors from imperfections in mechanical components and otherwise improve performance by reducing transmission losses, CECOM found that the Kollsman design remained overly complex; Kollsman had proposed to use 12 times as many parts of low or moderate reliability as did Hughes, and more than 4 times as many parts in total. CECOM specifically questioned Kollsman's proposal of four separate optical channels, each using a separate light emitting diode (LED), concluding that this was twice the optimal number (and more than proposed by Hughes). CECOM considered this to be a significant weakness because LEDs are the least reliable parts in the test sets. The agency determined that the simpler, less complex Hughes design would improve performance and reliability, while reducing maintenance.

Secondly, CECOM concluded that the LED selected by Hughes, which would project the test pattern with a wavelength in excess of 800 nanometers (800 billionths of a meter), was superior to the Kollsman LED, operating on a wavelength of 700 nanometers. According to CECOM, since the degradation in performance of the night vision devices appears primarily in the wavelength between 700 and 880 nanometers, 780 nanometers is the optimum wavelength for detecting decreases in performance; the agency concluded that while the higher wavelength of the Hughes test set may result in rejection of some still useful night vision devices, the Kollsman test set would pass some unacceptable devices because the degradation in performance would not be detected by its lower wavelength test signal.

In view of its technical superiority, CECOM determined that Hughes' proposal offered the overall best value to the government and, accordingly, made award to Hughes on May 25. Shortly thereafter, Kollsman filed this protest with our Office challenging the award, particularly the evaluation, on several grounds. We discuss Kollsman's most significant arguments below.

TECHNICAL EVALUATION

The evaluation of technical proposals is primarily the responsibility of the contracting agency; the agency is responsible for defining its needs and the best method of accommodating them, and must bear the burden of any difficulties resulting from a defective evaluation. Accordingly, our Office will not make an independent determination of the merits of technical proposals; rather, we will examine the agency's evaluation to ensure that it was reasonable and consistent with stated evaluation criteria and applicable statutes and regulations. See GTE Government Systems Corp., B-222587, Sept. 9, 1986, 86-2 CPD Para. 276.

Complexity of Design

Kollsman first argues that since CECOM believes that complexity of design not only adversely affects reliability and maintenance, but also increases life cycle costs, design complexity should have been considered under the cost/price criterion rather than under the more important technical criterion. Moreover, Kollsman contends that the agency's determination that the Kollsman design is overly complex is inconsistent with the agency's assessment that the design offers improvements with respect to reducing mechanical error, and fails to take into consideration the fact that the Kollsman design incorporates features providing redundancy and thereby increasing the mean time between failures (MTBF). In particular, Kollsman alleges that a test set using four LEDS, each of which is activated only one-quarter of the time, will operate four times as long as one using only one LED.

We do not find the agency's findings in this area unreasonable. The solicitation specifically emphasized and provided for evaluation under the technical criterion of the simplicity of the design and the extent to which the design facilitated operation, maintenance and training. Accordingly, while complexity of design may also influence life cycle costs and therefore properly may have been a consideration under the cost/price criterion, it was also properly for consideration under the technical criterion, as that criterion was defined in the RFP. We see nothing inconsistent in the agency concluding both that the Kollsman design offered improvements over the current generation of test sets, and that the design nevertheless remained more complex than necessary and more complex than the Hughes design; a design can be improved and yet remain less than optimal and inferior to another design.

With respect to the redundancy argument, contrary to Kollsman's contention that its four LEDs will operate four times as long as a single LED, the agency reports that this is not the case, since all four of the LEDs must be operational for the test set to be used as intended; Kollsman has cited nothing in its proposal indicating that the set will perform satisfactorily with fewer than four LEDs. (Further, a test set using four LEDs, each of which is operating one quarter of the time, may not even be as reliable as a test set using one LED that is constantly on; the agency reports that benefits from a reduction in operation time for an LED likely will be more than offset by increased stress resulting from it being repeatedly turned on and off.) Accordingly, and in view of the solicitation emphasis upon simplicity of design, we find nothing unreasonable in the agency's conclusion that the more complex design of Kollsman's test set, with more parts of low or moderate reliability and more parts in total, represented a significant technical weakness in its proposal.

LED Wavelength

Kollsman concedes that, "all other things being equal, an 835 (nanometer) LED would give better performance ... than a 700 (nanometer) LED"; in particular, Kollsman concurs with the agency that using an LED operating on a wavelength of 700 nanometers would appear to result in the test set passing as acceptable some unacceptable night vision devices. Kollsman claims, however, that the tendency of a 700 nanometer LED to pass unacceptable tubes can be remedied by appropriate adjustments to the output of the LED, and that the superiority of one wavelength over another in fact depends upon the system in which it is used; in this regard, Kollsman notes that it proposed several improvements, including a closed loop control system that would constantly monitor and adjust the LED output, that made its design superior to a higher wavelength system without improvements.

In its evaluation, CECOM in fact took into consideration, and gave credit for, the improvements proposed by Kollsman. CECOM found, however, the Hughes' higher wavelength design also incorporated similar improvements, including the closed loop control system. Accordingly, and since there is no dispute that, all other things being equal, an LED operating on a wavelength in excess of 800 nanometers, as proposed by Hughes, is inherently more likely to detect unsatisfactory night vision devices, the agency did not unreasonably conclude that Kollsman's proposed test set was less likely to detect unsatisfactory devices than the set proposed by Hughes. This was a particularly important consideration in view of the intended use of some of the night vision devices: helicopter pilots rely on the devices to fly at night at extremely low altitudes and high speeds. We believe the resulting critical safety considerations warranted seeking the maximum possible assurance that the test sets will detect night vision devices whose performance is inadequate. See Litton Systems, Inc., et al., B-229921, et al., May 10, 1988, 88-1 CPD Para. 448.

We conclude that CECOM's preference for a test set more responsive to the solicitation emphasis upon simplicity, when considered in conjunction with the relative weakness of Kollsman's design with respect to detecting unsatisfactory night vision devices, provided a reasonable basis for the agency's selection of the Hughes proposal. As these factors were evaluated under the most important evaluation criterion, CECOM's determination that Kollsman's slightly lower initial acquisition cost and relative advantage under the least important evaluation criterion did not offset Hughes' technical superiority, was reasonable and consistent with the evaluation scheme.

DISCUSSIONS

Kollsman contends that CECOM failed adequately to apprise it during negotiations of the perceived deficiencies in its proposal. With respect to the agency finding that its design was overly complex, Kollsman argues that "while CECOM may have commented to Kollsman that its design seemed complex, Kollsman had no indication that this was considered a major weakness."

While discussions generally must be meaningful, this does not mean that offerors are entitled to all-encompassing discussions; agencies are only required to lead offerors into areas of their proposals considered deficient. Where a proposal is considered to be acceptable and in the competitive range, an agency is not obligated to discuss every aspect of the proposal that received less than the maximum possible rating. See Varian Associates, Inc., B-228545, Feb. 16, 1988, 88-1 CPD Para. 153.

In view of the clear solicitation preference for simplicity of design, we think advising Kollsman that its design appeared complex was sufficient to place the firm on notice that the agency considered this a weakness in the design. In any case, Kollsman does not claim that it could have changed its design, without a major redesign, to eliminate the complexity, and since it appears that the perceived weakness was inherent in the design itself (rather than in any failure to explain the design), there is no reason to believe more detailed discussions would have enabled Kollsman to improve its proposal significantly in this area. See Aydin Vector Division of Aydin Corp., B-229569, Mar. 11, 1988, 88-1 CPD Para. 253.

Kollsman also complains that, while it was requested to clarify how its LED design would satisfactorily test night vision device performance, it was not specifically advised that the wavelength for its LED was too low on the spectrum. Kollsman states that the cost of substituting another LED with a different wavelength would have been minimal, and claims that it would have done so had it known that the agency considered its LED to represent a weakness.

Kollsman's argument is unpersuasive. First, in explaining why it chose a 700 nanometer LED, Kollsman indicates that it was fully aware of the inherent limitations of that wavelength, but nevertheless chose to design around these limitations rather then select a higher wavelength better capable of detecting unsatisfactory devices. Moreover, it is clear that CECOM never intended to establish a certain wavelength as a requirement, opting instead to establish broad performance requirements and leave it to the ingenuity of the offerors to develop the overall optimal system meeting or surpassing the RFP requirements. Indeed, in response to a specific preproposal conference question, the agency refused to specify a required wavelength. Given this approach, while CECOM could have been more specific during discussions, it was not required to be. The protest is denied.

/1/ In a cost analysis conducted after award, CECOM concluded that as a result of the lower hardware price for the Hughes design and its likely greater reliability (see discussion below), the total life cycle cost for the Hughes test set ($11,900,168) would be substantially lower than the corresponding cost ($18,082,684) for the Kollsman set. Even assuming an equivalent reliability, the agency calculated that the life cycle cost for the Kollsman design ($18,082,684) would still exceed the cost for the Hughes design ($17,743,892).