Skip to main content

B-232175, Nov 7, 1988, 68 Comp.Gen. 48

B-232175 Nov 07, 1988
Jump To:
Skip to Highlights

Highlights

Absent a clear showing that an agency's evaluation was unreasonable. Exclusion of protester's proposal from the competitive range is warranted where agency evaluation finds the proposal unacceptable with major deficiencies that are considered to be the result of a poor and risky design and concludes that the proposal does not have a reasonable chance of being selected for award. The element of risk is clearly related to the evaluation of capability and approach. It is permissible to evaluate risk in a technical evaluation of a proposal for a firm fixed-price contract. Alleges that the reasons underlying its exclusion were not valid. The RFP was issued on June 23. The INVS units were to be developed in accordance with the government specifications included within Section C of the RFP.

View Decision

B-232175, Nov 7, 1988, 68 Comp.Gen. 48

Procurement - Competitive Negotiation Offers - Competitive ranges - Exclusion - Administrative discretion 1. Absent a clear showing that an agency's evaluation was unreasonable, or inconsistent with the stated evaluation criteria, exclusion of protester's proposal from the competitive range is warranted where agency evaluation finds the proposal unacceptable with major deficiencies that are considered to be the result of a poor and risky design and concludes that the proposal does not have a reasonable chance of being selected for award. Procurement - Competitive Negotiation - Offers - Risks - Evaluation - Technical acceptability 2. The element of risk is clearly related to the evaluation of capability and approach, and it is permissible to evaluate risk in a technical evaluation of a proposal for a firm fixed-price contract.

Kaiser Electronics:

Kaiser Electronics (KE) protests the exclusion of its proposal from the competitive range under request for proposals (RFP) No. N62269-87-R 0210, issued by the Naval Air Development Center (NADC) for a firm fixed-price contract for the design and development of an Integrated Night Vision System (INVS) to be used on tactical aircraft. /1/ The protester questions the agency's technical evaluation of its proposal, and alleges that the reasons underlying its exclusion were not valid.

We deny the protest.

The RFP was issued on June 23, 1987, with a revised closing date of September 4, 1987. The RFP called for the delivery of 20 full-scale development units, with accompanying data, and included four options-- a leader/follower program; limited production of 110 additional INVS units; and five more units as part of additional full-scale development. The INVS units were to be developed in accordance with the government specifications included within Section C of the RFP.

The RFP stated that the government would make a single award to the responsible offeror whose offer, conforming to the solicitation, was determined most advantageous to the government, price and other factors considered. According to the RFP, proposals were to be evaluated against three factors, listed in descending order of importance: technical approach, management approach, and logistics approach. Within the technical approach factor were four subfactors listed in descending order of importance: design approach, test and evaluation, pre-planned product improvement, and system effectiveness. The management approach factor had seven subfactors and the logistics approach factor had four subfactors, none of which are relevant to this protest. The RFP's emphasis under the design approach was placed on the extent to which the proposed INVS design meets the requirements of the RFP. The RFP further stated that evaluation of price proposals would be of secondary importance but that the degree of its importance would increase with the degree of equality of the proposals.

Several proposals, including KE's, were received and evaluated. The technical scores ranged from a low of 8.7 to a high of 76.5 out of a possible 100 technical points. KE's proposal received an initial technical score of 36.2 and was included in the competitive range. Oral and written discussions were then conducted with all firms in the competitive range. /2/ Proposals were then reevaluated and rescored by the technical evaluation committee. Although the technical score of KE's proposal increased to 48.3 out of the possible 100 points, the proposal was excluded from the competitive range at that time since it was determined technically unacceptable even after discussions.

KE's protest is essentially that the reasons the Navy presented for the technically poor rating of its proposal which led to its rejection were invalid. The protester contends that its proposal as amended is in total compliance with the requirements of the RFP and that the Navy evaluators misapplied and used unidentified evaluation criteria.

Initially, we note that a determination that an initial proposal is within the competitive range does not necessarily imply that the proposal would be technically acceptable for award, but merely denotes that the proposal has a real possibility of being made acceptable and that there is a reasonable chance it will be selected for award. See Federal Acquisition Regulation Sec. 15.609(a) (FAC 84-16); Space Communications Co., B-223326.2, B-223326.3, Oct. 2, 1986, 66 Comp.Gen. ***, 86-2 CPD Para. 377. Further, the evaluation of proposals and determination of whether an offeror is in the competitive range are matters within the discretion of the contracting agency, since it is responsible for defining its needs and must bear the burden of any difficulties resulting from a defective evaluation. The International Association of Fire Fighters, B-224324, Jan. 16, 1987, 87-1 CPD Para. 64. Consequently, we will not conduct a de novo technical review of the proposals; our review is limited to examining whether the evaluation was fair and reasonable and consistent with the RFP criteria. Maxima Corp., B-220072, Dec. 24, 1985, 85-2 CPD Para. 708. The fact that a protester may disagree with the agency's conclusion does not itself render the evaluation unreasonable. See TIW Systems, Inc., B-222585.8, Feb. 10, 1987, 87-1 CPD Para. 140. For the reasons stated below, we do not believe that the protester has shown that the agency's judgment as to the risks involved with the design approach proposed by KE, and which led to rejection of the proposal, was unreasonable or inconsistent with the evaluation criteria.

The Navy evaluators concluded that KE's optical design was extremely poor. Because of KE's inability to resolve certain major technical deficiencies, believed to be inherent in its design, KE was found to be unable to meet certain required specifications involving the following: (1) objective lens design; (2) relay and display optics assembly design; (3) image intensifier assembly; (4) physical adjustments; (5) unaided eye field of vision; and (6) schedule. For our purposes here, we will only discuss the technical problems involving the resolution (objective lens design) and gain/throughput (image intensifier assembly), since the Navy states, and we agree, that failure to comply with the RFP requirements involving either would have been cause to find the KE offer technically unacceptable after discussions.

With respect to lens design resolution, /3/ all parties agree that under the RFP, the minimum requisite resolution for an INVS unit with a 30 degree FOV is 1.0 cycle/milliradian (cy/mr), whereas an INVS unit with a 40 degree FOV need only have a resolution of .76 cy/mr. The KE design was evaluated by the Navy as offering a resolution of .80 cy/mr. The protester argues that its design offers a 40 degree FOV for which a resolution of .76 cy/mr or higher is acceptable. The Navy, however, contends that KE's design offers a 30 degree FOV, for which in accordance with the specifications a resolution of .80 is clearly unacceptable. Further, regardless of whether KE's design offers a 30 degree or 40 degree FOV, both parties agree that KE proposes to achieve the 40 degree FOV by canting /4/ the monoculars. It is the Navy's position that canting is specifically prohibited by the RFP. The Navy also states that canting introduces an unacceptable technical risk to the design. The protester, on the other hand, argues that the specifications do not prohibit canting and that technical risk was not a stated evaluation factor.

We have no reason to disagree with the agency's interpretation of the requirement. Paragraph 3.2.2.1.6.2 of the RFP clearly requires under binocular alignment that "the intensified image presented to each eye shall be coaxial with each other." By use of the term coaxial, the record shows that the agency intended the monoculars to be parallel, precluding canting (setting at an angle). The protester does not explain why the agency's interpretation is unreasonable. Moreover, KE was specifically advised in the February 11, 1988 discussions with the agency that canting the monoculars was a very poor design approach and presented unacceptable technical risks. Nevertheless, KE subsequently again insisted on its approach of "canting the optics inward," despite the Navy's advice that it presented unacceptable risks.

Although technical risk was not a stated evaluation factor, we believe that selection of a contractor which can best perform this contract involves a choice between design approach and the acceptance of a certain level of risk. While technical evaluations must be based on the stated evaluation criteria, the interpretation and application of such criteria often involve subjective judgments. Thus, we will not object to the use of evaluation factors not specifically stated in the RFP where it is reasonably related to the specified criteria and the correlation is sufficient to put offerors on notice of the additional criteria to be applied. See Consolidated Group, B-220050, Jan. 9, 1986, 86-1 CPD Para. 21 at 7, 8. We have condoned the evaluation of risk in a technical evaluation of a proposal for a firm fixed-price contract. Litton Systems, Inc., Electron Tube Division, 63 Comp.Gen. 585 (1984), 84-2 CPD Para. 317. Since the element of risk is clearly related to the evaluation of design approach, we find nothing improper in the Navy's use of risk as an element of the evaluation. The protester simply has not shown that it could provide an acceptable system with its proposed approach without canting, despite having been afforded the opportunity to do so after discussions.

The Navy also found that KE's image intensifier assembly did not meet the requirements of paragraph 3.2.2.4.4 for minimum photopic throughput /5/ of the light from the image intensifier of 62 percent. According to the Navy's evaluation, KE's throughput was only 11 percent. KE does not contend that the Navy's evaluation with respect to throughput and gain is incorrect or that its design did in fact meet the stated values in the RFP. Instead, the record shows that KE apparently made a conscious decision to make certain trade-offs when it chose to design for ultra-low system distortion and compatibility with corrective and protective eyewear using a configuration which it knew could not meet the 62 percent transmission. Once again, through discussions, the Navy informed KE that its approach was believed by the Navy to be of an "extremely high technical risk." However, KE never corrected this problem to the Navy's satisfaction.

According to the Navy, the KE proposal was vastly inferior technically (especially in design) to each of the other remaining offerors, and substantially higher in price than one of those offerors. Consequently, the Navy determined that KE did not have a reasonable chance for award and eliminated it from the competitive range.

Based on the record before us, we find that the evaluation of KE's proposal was reasonable. KE's proposal did not meet certain stated technical specifications in several respects and was considered to involve a high degree of risk. KE admits that it made certain technical tradeoffs with respect to meeting the specifications and merely disagrees with the agency's determination that its approach was technically inferior. Moreover, KE had the opportunity through discussions to improve or correct its proposal, but failed to do so.

KE also argues that the Navy failed to properly rank the evaluation criteria in the RFP and that it was misled by the RFP to improperly place more emphasis on developing an innovative approach than in meeting some of the stated RFP requirements. The Navy admits that there was an error in the ranking of evaluation "sub-subfactors" but believes that the protester was not prejudiced because even if the provisions were read as KE suggests, the firm's offer would remain technically unacceptable. The RFP listed in descending order of importance the three major technical evaluation factors and each major factor's subfactors. However, KE interpreted the RFP as also stating that the factors listed under each subfactor were also listed in descending order of importance. The RFP provided the following with respect to the evaluation of the subfactor, design approach:

"M.3.1.1 Design Approach

"The extent to which the proposed INVS design meets the requirements of the solicitation. This will include, but not be limited to, innovation, modularity, degree of and rationale for specification variances, Human Factors Engineering, demonstrated understanding of the tactical aircraft mission environment, demonstrated understanding of the system engineering requirements as well as system performance requirements, and demonstrated understanding of the scientific issues and engineering tradeoffs among critical design and integration parameters."

According to KE's interpretation, innovation is the most important factor under the subfactor, design approach. Consequently, KE argues that it concentrated its efforts in proposing an innovative design. However, even if the evaluation criteria are read as KE interprets then, the overriding factor under design approach would be the "extent to which the proposed INVS design meets the requirements" of the RFP, not innovation. Further, regardless of the "sub-subfactor" criteria under which KE's design approach was evaluated, KE's approach, as we have found, was reasonably viewed as being of extremely high technical risk, more costly, and inferior to the other proposals in the competitive range. While the RFP permitted an innovative design, the agency was not prohibited from evaluating the risks of the innovative design. We therefore find no merit to this protest ground. Finally, KE contends that the Navy failed to advise KE that there was a risk of it being eliminated from the competition on technical grounds. We find this allegation also to be without merit. The agency report contains a record of oral and written discussions conducted with KE. All areas of deficiencies were discussed with KE, including the Navy's perceived technical risk involved in KE's design approach. KE was notified of the agency's concerns and was given the opportunity to amend its proposal. We do not think that any more was required. We further find no merit to KE's contention that it was prejudiced by the agency's delay in notifying the firm that its proposal had been rejected, since we have determined that the Navy had a reasonable basis for determining KE's proposal to be outside the competitive range. The protest is denied.

/1/ The INVS is an image intensification device, mounted on a pilot's helmet, for use in fixed wing military airplanes as an aid to pilot vision during all phases of nighttime operation.

/2/ It is unclear whether the agency made a formal competitive range determination based on its preliminary technical evaluation. However, the agency did decide to keep KE in the competition until discussions were conducted with the firm.

/3/ Resolution (clarity) is a measurable characteristic of an optical system.

/4/ To set an angle.

/5/ The measure of the amount of visible light that is passed from the end of the image intensifier assembly that is visible to the eye.

GAO Contacts

Office of Public Affairs