Matter of: Wilcox Electric, Inc. File: B-270097 Date: January 11, 1996

Matter of: Wilcox Electric, Inc. File: B-270097 Date: January 11, 1996

Where solicitation discloses that proposed features will be evaluated under criteria concerning an item's valuable characteristics and its suitability/ease-of-use, agency's evaluation under both criteria does not constitute improper "double counting." Protest that agency improperly utilized an unstated evaluation criterion is denied where the alleged unstated criterion was simply an additional consideration to a rationale which, by itself, fully supported the contracting officer's determination that the higher-priced, higher technically rated proposal represented the best value to the government.

Attorneys

DECISION

Wilcox Electric, Inc. protests the award of a contract to NavAids, Inc. under request for proposals (RFP) No. DTFA01-95-R-27369, issued by the Federal Aviation Administration (FAA), Department of Transportation for supply of portable instrument landing system receivers (PIR). Wilcox contends that the evaluation and selection decision were flawed.

We deny the protest.

PIRs are used in the routine maintenance of instrument landing systems. The RFP, issued April 14, 1995, sought commercial/nondevelopmental PIR units rather than a receiver designed and built to FAA specifications. Offerors were required to submit two PIRs identical to those proposed under the RFP and two instruction manuals along with other technical documentation. The RFP contemplated award of a firm, fixed-price contract for a base quantity of 200 PIRs in the first year with four options for 100 units per year.

The RFP advised offerors that proposed PIRs would be evaluated in four areas, listed in descending order of importance: basic performance and design characteristics; additional "value enhancing" features; subjective evaluation of suitability and ease of use; and price. The RFP included a detailed list of the basic characteristics (Table 1) and advised offerors that proposed PIRs need not meet all listed characteristics in order to be rated "satisfactory." The RFP included a second list (Table 2) of features and characteristics known to be incorporated in modern digital PIRs and which the FAA believed were value enhancing. These features were rated low, moderate, or high for each of three sections based on value to the FAA. With regard to the third, ease-of-use factor, the RFP stated that the evaluation was to include, but not be limited to, eight specified areas, to be scored as "good," "fair," or "poor."

Award was to be made to the offeror whose proposal was most advantageous to the government considering the listed technical factors and proposed price. The RFP stated that award would not necessarily be made to the qualified offeror with the lowest price.

Four offerors, including Wilcox and NavAids, submitted PIRs and proposal packages to the FAA. The agency evaluated the PIRs in June 1995, and conducted written discussions with all offerors. The letters to NavAids and Wilcox informed them of those enhanced value features and ease-of-use considerations which their PIRs lacked or failed to fully implement. For example, Wilcox's letter advised that "overall, the visibility of the display was judged to be poor . . . [suffering] from extreme glare due to the reflection of sunlight . . . when the PIR was operated out in the open, and reflection of room lights when operated in the shelter." In response to the discussion letters, both offerors submitted best and final offers (BAFO) in which neither submitted any technical or price changes. The results of the final evaluation are as follows:

Factors (Points) NavAids Wilcox

Basic Features (100) 100 100

Valued Features (90) 74 52

Ease of Use (80) Good (80) Fair (50)

Total Score (270) 254 202

Total Price $4,782,800 $3,806,100

The contracting officer, as source selection official, reviewed the evaluations and evaluation criteria and determined that NavAids's PIR represented the best value to the FAA. On September 26, the FAA awarded NavAids the contract. After receiving a debriefing, Wilcox filed this protest.

Wilcox challenges the technical evaluation of the PIRs under the second and third factors. Where an evaluation is challenged, we will examine the evaluation to ensure that it was reasonable and consistent with the evaluation criteria and applicable statutes and regulations, since the relative merit of competing proposals is primarily a matter of administrative discretion. Information Sys. & Networks Corp., 69 Comp. Gen. 284 (1990), 90-1 CPD Para. 203. Mere disagreement with the agency's evaluation does not itself render the evaluation unreasonable. Litton Sys., Inc., B-237596.3, Aug. 8, 1990, 90-2 CPD Para. 115. Our review of the record here provides no basis for objecting to the agency's evaluation.

Wilcox first argues that the agency's evaluation of the proposed PIRs under the ease-of-use factor was inconsistent with the terms of the RFP. According to the RFP, a "good" rating was defined as "overall significant improvement over the old instruments" and "fair" was defined as "measurements can be made with no more difficulty and with no less confidence than using the old instruments." Wilcox contends that the evaluation did not include any comparison with the FAA's "old" PIRs. The allegation is mistaken.

The agency's technical evaluation plan identified the current analog PIR (Cardion 8766 or equivalent) to be used for comparison purposes. Each of the eight evaluation test report sheets contained a column for recording the reference PIR score and indicated when the reference item was a separate item or nonexistent. For example, under test four, the sheet notes that the "old" PIR does not measure modulation and advises that a modulation meter is used instead. For test six, the sheet identifies a vector volt meter as the standard reference test instrument. It is clear from the various specific references to the "old" PIR that the agency contemplated making the comparison identified in the evaluation scheme.

Further, the record shows that the evaluators made a comparison and specifically provided scores for the "old" PIR in two of the eight tests. It also shows that for test five, no comparison was made because there was no reference standard instrument. While the record is silent with regard to the other five tests, it is apparent that the evaluators were knowledgeable users of the FAA's current PIR. Thus, despite the absence of specific scores for the "old" PIR, the record indicates that the overall comparative scores on ease-of-use were based on the evaluators' experience with the reference PIR. To the extent that the evaluators did not make a specific comparison on every test, we find that the protester was not prejudiced. The evaluation sheets make plain that all proposed PIRs were tested in the same manner. In the clear absence of prejudice, we will not disturb a contract award. American Mutual Protective Bureau, Inc., B-229967, Jan. 22, 1988, 88-1 CPD Para. 65.

Wilcox next argues that its score of "fair" under the third factor was improper because the FAA "overinflated" the relative importance of two to three "relatively minor attributes" of its PIR. These attributes included glare on the display surface, use of menus for readings, and the need for a special battery pack.

The RFP listed eight considerations for the ease-of-use factor including the number of steps/operations for each measurement; need for additional adapters, cables, etc.; length of time needed for a reading to stabilize; ease of viewing the display under varied conditions; physical size, shape, weight relating to ease of carrying and using the item; ruggedness; inconsistencies in operation; and quality/completeness of instruction manual. In evaluating Wilcox's PIR, the evaluators found that Wilcox's menu-driven PIR required more operations to obtain the test measurements (33 operations in the 8 tests compared with 12 operations for the NavAids PIR). They also found that the display on Wilcox's PIR was more difficult to read under most lighting condition due to glare, while the NavAids PIR was easy to read in all light conditions. [1] For example, the bezel on the Wilcox unit sometimes obscured the top of the display making a "9" appear as a "4" at first glance. Wilcox's PIR also required the separate attachment of an antenna and used a hard-wired, special battery pack which made replacement more difficult. The Wilcox PIR required a separate carrying case for the battery charger, antennas, and accessory cables, while, apart from the support pole, all of NavAids's unit fit in a single small case. Out of the eight tests, Wilcox's PIR was rated "fair" in six tests, "good" in one test, and "poor" in one test. NavAids's PIR was rated "good" in all eight tests.

While the protester views these matters as relatively minor, they represent weaknesses in half of the listed considerations for this factor. Moreover, the PIR is designed primarily for outdoor use in making accurate measurements. Accordingly, we think Wilcox's PIR, which is harder to read, requires the attachment of an external antenna and more steps to make readings, and is more difficult to transport due to a separate accessory case, was reasonably evaluated as only "fair" under the ease-of-use factor. [2]

Wilcox also notes that its lower PIR score under the second factor was based on some of the same attributes for which it received a lower score under the third factor. Wilcox argues that this represented an improper double count of the importance of these attributes and thus violated the terms of the solicitation. Where an RFP lists a number of evaluation factors of stated importance, a single one cannot be accorded more than the weight prescribed in the RFP's evaluation methodology by the repeated consideration of the same factor in conjunction with the other major factors, i.e., it is improper to double or triple count the importance of a single listed factor. J.A. Jones Management Servs., Inc., B-254941.2, Mar. 16, 1994, 94-1 CPD Para. 244.

Here, however, the fact that the proposed PIRs were to be evaluated on the basis of the presence or absence of certain features and on the ease-of-use of the units in part based on these same features does not mean there was an improper, multiple consideration of the features. The RFP explicitly advised offerors that the ease-of-use evaluation "encompasses many of the specific items listed in the [first two factors], but viewed from a practical, rather than laboratory or inspection standpoint." Thus, the evaluation under the second factor was aimed at determining the existence and nature of various PIR features deemed of high, moderate, and lower value to the FAA. The evaluation under the third factor was aimed at assessing how well these features worked in actual operation of the units. It is not unreasonable to make a separate assessment of the existence and quality of a feature, and of how well it would work in its practical application; such assessments do not represent an improper multiple counting of the same factor. See Teledyne Brown Eng'g, B-258078; B-258078.2, Dec. 6, 1994, 94-2 CPD Para. 223. [3] Thus, there was nothing improper, for example, with the evaluators considering under the second factor the type of display offered while considering the ease-of-use of that display in a practical setting under the third factor.

Wilcox also contends that the source selection decision improperly relied on the monetary benefit of NavAids's longer PIR calibration interval, which it contends constituted use of an unannounced evaluation criterion. In this regard, Wilcox notes that prior to the submission of proposals, the FAA advised that it would not consider anything in the evaluation (specifically including calibration interval) beyond what was stated in section M of the RFP. Since the calibration interval in the RFP is stated in terms of a minimum 12-month period, Wilcox contends that consideration of a longer period is improper.

We need not consider whether consideration of a longer calibration period was improper; the source selection was not based on the potential cost savings of a longer calibration interval. It was only as an "additional consideration" that the contracting officer mentioned the life-cycle cost savings potential with the NavAids PIR. Before ever addressing this consideration, he found that NavAids's PIR's "high value" technical features (display readability, simplicity of operation, use of standard field replaceable batteries, and transportability) represented the best value to the government. "On this basis" he selected NavAids for the award.

In a justification for award document, the contracting officer provided a more detailed rationale for his determination that the technical superiority of the NavAids's PIR justified paying the $1,400 per unit price premium. He specifically noted that NavAids's PIR possessed all of the high and moderate value additional features, while Wilcox's PIR did not. He found significant the difficulty in transporting the Wilcox unit in its separate cases. He acknowledged that Wilcox's PIR possessed all of the lower value features, while NavAids's unit did not, and thus, could make some measurements which the NavAids unit could not. However, since these measurements were not frequently made, he found the advantage of the Wilcox unit to be of little significance. He concluded that the NavAids design and feature set was more "in tune" with the FAA's priorities. He also considered the superiority of the NavAids PIR as demonstrated in the ease-of-use tests. He noted that the fewer operations required to take measurements with the NavAids PIR, while a small time savings, represented an intangible saving in fewer errors or the need to consult an instruction manual. He also noted the problem with readability with the Wilcox display. This detailed record of the contracting officer's rationale makes plain that the potential cost savings was simply an additional consideration, which was not necessary to support his award determination.

The protest is denied.

Comptroller General of the United States

1. The FAA attached a videotape of the ease-of-use tests which we have reviewed, and which shows that the NavAids display is easier to read than that of Wilcox's PIR.

2. Wilcox also challenges the agency's translation of the ease-of-use scores into numbers. In the protester's view, it was unreasonable to make a "good" score 80 points and a "fair" score only 50 points. The use of numerical point scores are useful as guides in evaluating proposals. National Medical Seminars Tempharmacists, B-233452, Feb. 22, 1989, 89-1 CPD Para. 191. While the difference between fair and good is larger than the protester feels warranted, we find nothing objectionable in that difference. Further, even if this number score were eliminated from the evaluation, NavAids's unit would still have a significantly higher total point score and exceed Wilcox's adjectival score by one level. Thus, the use or absence of a point score for this factor would have no appreciable effect on the agency's evaluation.

3. In a related argument, Wilcox notes that it lost 24 points under the second factor because of its use of a menu display, its unique battery, and backlit display. The RFP specifically advised offerors of the relative value of these features and the agency advised the protester in discussions that these features had resulted in a lower score. To the extent that Wilcox is arguing that the weight of these features is too great, or that the evaluations were unfair, its protest is untimely. Solicitation improprieties must be protested prior to the closing time for receipt of proposals and other matters must be raised within 14 days of when the protester knew, or should have known, of its protest ground. Bid Protest Regulations, section 21.2(a), 60 Fed. Reg. 40,737, 40,740 (Aug. 10, 1995) (to be codified at 4 C.F.R. Sec. 21.2(a)).