[Protest of DOT Exclusion of Bid From Competitive Range for Computer Software]

Matter of: Rand McNally-TDM, Inc. File: B-248927 Date: October 7, 1992

PROCUREMENT Special Procurement Methods/Categories Computer software Sample evaluation Testing PROCUREMENT Specifications Minimum needs standards Competitive restrictions Performance specifications Justification The rejection of the protester's proposal for computer software was reasonable where solicitation warned that failure of an offeror's proposed software to meet any of the mandatory specifications may render the proposal unacceptable and could provide the basis for rejection of the proposal, and a functional test demonstration showed that protester's software did not meet two of the performance specifications.

Attorneys

DECISION Rand McNally-TDM, Inc. protests the exclusion of its proposal from the competitive range under request for proposals (RFP) No. DTRS-57- 91-R-00034, issued by the Department of Transportation for a perpetual site license for up to 400 copies of a software program to be used to calculate mileage for highway routes within the United States, Canada and part of Mexico. The agency rejected the protester's proposal because it found during a "Functional Test Demonstration" (FTD) conducted using the protester's software that it failed to comply with the RFP requirements that the software be capable of (1) generating routes which avoid toll roads and (2) printing a route list report. Rand McNally challenges the agency's conclusion that its software was unacceptable in these areas.

We deny the protest.

BACKGROUND

The software will be used with laptop computers by agency safety investigators to perform safety inspections and audits seeking to verify motor carriers' compliance with federal safety regulations which limit the number of hours a driver may drive without substantial off-duty time. The software will automate the investigators' tasks in making comparisons between drivers' records and the actual routes and distances by quickly generating accurate highway distance, travel time, and other relevant information.

The RFP set forth at section C software specifications which must be met. Two of those requirements are at issue here: (1) the generation of routes which avoid toll roads and (2) the ability of the software to print a route list report.

As far as the first of these is concerned, the RFP stated that "[t]he user shall have the ability to override any default options, such as routes which avoid toll roads." This requirement was the subject of two questions from prospective offerors. These two questions (and others not relevant here) and the agency's answers were printed and distributed to potential offerors as amendment No. 3 to the RFP.

Question and answer No. 3 appeared in amendment No. 3 as follows:

QUESTION

"[R]egarding the reference to override of any default options such as routes which avoid toll roads . . . [w]hat other types of routes, if any, does [the agency] want the ability to default to, or override?"

ANSWER

"The system may determine routes within a given context, such as most practical or shortest, with restrictions such as Class 8 vehicles or avoidance of toll roads, but the user to override all restrictions."

Question and answer No. 14 appeared in amendment No. 3 as follows:

QUESTION

"[P]lease define the minimum default options required to maintain compliance with this RFP. The example given is not currently available with our existing software, however all of the default options can be changed by the user. If avoiding toll roads is a necessary requirement, then we will need to adjust our price quotes accordingly."

ANSWER

"The system may determine routes within a given context, such as most practical or shortest, with restrictions such as Class 8 vehicles or avoidance of toll roads (which is a requirement), but the user must have the ability to override all restrictions."

Regarding the second of the two disputed requirements, the RFP provided at section C as follows:

"The software must be capable of both printing and displaying the calculated route list. The route list report(s) must include:

Highway name Distance on highway Travel time between route change points (time based on speed limit and other parameters included in the database) State and country border crossings Summary totals, i.e., total miles and time Notation for toll roads."

Regarding the FTD requirement, the RFP explained that the agency would test the proposed software submitted by each competitive range offeror after receipt of initial proposals to determine compliance with "all mandatory specifications as set forth in the RFP." The RFP warned that "[i]f after FTD testing, the software has failed to meet any of the mandatory specifications, the offeror's proposal may be declared technically unacceptable." The RFP stated that a finding of nonacceptability shall constitute a basis for rejection of the offer.

The agency received three proposals by the February 3, 1992, closing date. Each of the written proposals was reviewed by the agency and all three were included in the "preliminary" competitive range for purposes of participation in the FTD.

The agency conducted the FTD from February 19 through 21. The agency tested Rand McNally's software and found that it did not meet the two requirements described above. The agency evaluators concluded that Rand McNally's software could not eliminate toll roads. They found that while the software had "the ability to `deemphasize' toll roads by a percentage controlled by the user, even "100 percent deemphasis did not remove toll roads from the selected route."

The evaluators also found that the software did not have the requisite report generating capability. The evaluators found that the Rand McNally product would only print by use of the "print screen" function of the laptop computer and that this was unacceptable since "on-screen reports typically extend over several screens, requiring many `print screen' dumps." In further support of their conclusions, the evaluators pointed out that these "screen dumps" included menu key listings and other information not related to the report.

By letter dated March 31, the agency advised Rand McNally that it had concluded that its proposed software did not satisfy the requirements and that its proposal was therefore unacceptable and no longer eligible for award. The agency advised one of the other offerors that its proposal was considered unacceptable because its software also failed the FTD, but for different reasons.

Rand McNally filed a protest with the contracting officer challenging the agency's determination that its software was unacceptable. In the course of that protest, in addition to arguing that the RFP did not require that the software totally eliminate toll roads, Rand McNally argued that its software had the capability of producing routes which completely eliminate toll roads. The protester explained that this "alternate" method, which was not demonstrated during the FTD, requires that the user of the software manually enter intermediate points on toll-free roads.

In responding to Rand McNally's agency-level protest, the contracting officer and agency technical evaluators considered the protester's alternative method of generating toll-free routes. They concluded that this method was unacceptable since it required that the user manually determine the toll-free route. According to the evaluators, this task was to be performed automatically by the software. Moreover, the contracting officer stated that since the software was to be used protester's proposed alternative method would "defeat the purpose of the procurement." The agency denied the protest, and Rand McNally filed its protest with our Office on June 3.

Rand McNally argues that its software complies with all the RFP requirements and that the agency's conclusions to the contrary are based on unreasonable interpretations of the RFP. The protester contends that the exclusion of its proposal from the competitive range is particularly egregious given the fact that only one offer remains.

EVALUATION

We regard benchmarks, or, by analogy, demonstrations of the type required here, as extensions of the technical evaluation of proposals, the principal purpose of which is to provide a demonstration of the capability of offered hardware/software to perform the required functions. NCR Corp., B-209671, Sept. 16, 1983, 83-2 CPD Para. 335. Consistent with this view, we have been critical of strict pass/fail benchmarks, which lead to the automatic exclusion of otherwise potentially acceptable offers, and have held instead that such tests provide "strong evidence" of system capabilities which must be considered in determining technical acceptability. See CompuChem Labs., Inc., B-242889, June 17, 1991, 91-1 CPD Para. 572; NBI, Inc., B-201853.3, Aug. 9, 1982, 82-2 CPD Para. 114. As far as the agency's actual determination of technical acceptability is concerned, we will not make an independent determination of the merits of an offeror's proposal, or in the case of a demonstration, the performance of the offeror's product; rather, we will review the evaluation record, including the results of any test demonstration, to ensure that the agency's technical judgment is based upon the requisite "strong evidence," has a rational basis and is consistent with the stated evaluation criteria. See S-Cubed, A Div. of Maxwell Labs., Inc., B-242871, June 17, 1991, 91-1 CPD Para. 571; NCR Corp., supra.

TOLL ROAD AVOIDANCE REQUIREMENT

Rand McNally argues first that the agency's conclusion that its software's toll road avoidance feature was unacceptable was based on an unreasonable interpretation of the RFP requirement. Specifically, the protester argues that it understood the RFP's use of the term "avoid" to mean only a "relative tendency to refrain from." Thus, according to the protester, offerors were only required to propose software which had the capability to generate routes which avoid toll roads to some extent. Since its software has the capability to generate routes concludes that its software was compliant. We do not agree.

According to Webster's Ninth New Collegiate Dictionary 120 (1988), "avoid" means to "keep away from: shun" or "to refrain from." In this regard, "avoid" should be given its common usage definition. In our view, "avoid" does not, as the protester argues, denote a relative tendency to refrain from. We see no reason, nor has Rand McNally provided one, to conclude that the agency intended to qualify the term in the manner now suggested by the protester. The protester's assertion that it understands "avoid" to mean a relative tendency to avoid does not make it so.

In addition, the record shows that another offeror specifically modified its software to produce toll-free routes in accordance with the RFP and that none of the other offerors had problems with this requirement during the FTD. We therefore think that the only reasonable interpretation of the requirement was that toll roads must be completely excluded and that the specification did not contemplate merely a "relative avoidance" of them.

Rand McNally concedes that its product, as demonstrated at the FTD, would not produce a 100-percent toll-free route. Nevertheless, the protester argues that its software was capable of producing such routes had it been given an opportunity to demonstrate its ability do so at the FTD. The events surrounding this aspect of the FTD are in dispute. The protester and the agency have submitted conflicting versions of whether the agency personnel at the FTD gave the protester an opportunity to demonstrate its software's alternate capability to completely avoid toll roads. We, however, need not resolve the dispute because the agency has considered Rand McNally's alternate approach and determined that it is unacceptable. We will instead review the agency's conclusion concerning the protester's alternate approach.

As explained by Rand McNally, its alternate method, called the "via points" feature, permits the software user to replicate actual routes taken, including toll-free routes, by selecting "intermediate points" in addition to the origin and destination.

The agency's position that protester's alternate method of avoiding toll roads was unacceptable rests on its view that the solicitation required that the software, not the user, be capable of producing the route. Rand McNally disagrees that the requirement set forth in the RFP precludes its approach.

Where a dispute exists as to the meaning of solicitation language, we resolve the matter by reading the solicitation as a whole and in a manner that gives effect to all of its provisions. See Lithos Restoration, Ltd., 71 Comp.Gen. 367 (1992), 92-1 CPD Para. 379. To be reasonable, an interpretation must be consistent with the solicitation when read as a whole and in a reasonable manner. Id. Applying that standard here, we find, for the reasons set forth below, that the only reasonable reading of the toll road avoidance requirement as clarified in RFP amendment No. 3 is that the toll-free route required must be produced by the software, without the benefit of manually inserted, user selected intermediate points.

As we understand the basic operation of the mileage software program which is the subject of the RFP, it is to automatically generate optimal routes--usually in terms of the time or distance or a combination of the two--between designated travel points and to report certain specified information concerning the route generated such as distance covered and travel time. As stated earlier, this information will be used in most cases by agency safety investigators in their audits and inspections.

In addition, the RFP states in section C that:

"[T]he user must be able to specify a route composed of at least 20 intermediate way points (stop off points) and have the ability to insert or delete points from the route list."

This feature, which allows the selection of a particular route using intermediate points inserted by the user, is, as we understand it, designed to overcome the characteristics in the software's programming to calculate and plot the optimal route between two points. This is needed so that during an audit of a driver's record, the safety investigator can tailor the route to match a particular one which may not be optimal but which is, in fact, the one designated in the driver's log. So, in essence, the software must be able to calculate and plot an optimal route between two points automatically and permit the inspector to create his or her own route between the same points by manually inserting intermediate points. In both instances, the software must provide the same information concerning distance, time, etc.

Further, as we have concluded earlier, there is an RFP specification which mandates the generation of routes comprised only of toll-free roads. Rand McNally's "via points" method can produce a toll-free route along with the required information only if the user overrides the software's inherent tendency to produce an optimal route by manually inserting the toll-free route is not generated by the software but by the user's insertion of the appropriate intermediate points so that the software is directed to generate a user-tailored toll-free route.

In this context, the only reasonable interpretation of the RFP is that it requires the software to generate a toll-free route in the same way it is required to generate an optimal route based on time or distance. This interpretation is further supported by the RFP's treatment of the analogous requirement for the generation of routes for "heavy trucks" or Class 8 vehicles. This "heavy truck" requirement was listed separately in section C and was identified by the agency in its answers to the questions in amendment No. 3, along with the avoidance of toll roads, as functions that must be capable of being generated by overriding the optimal route. It would simply make no sense to separately specify requirements for the avoidance of toll roads and "heavy truck" routes if the same objectives could be accomplished by the section C requirement that the software permit the insertion of intermediate points to manually generate a particular route. In other words, if Rand McNally's manual "via points" method would suffice, there would be no need for the RFP to elsewhere specify these other two independent requirements.

We therefore find that, with respect to the toll avoidance requirement, the agency reasonably concluded that the protester's product was unacceptable since it does not have the capability to automatically generate toll-free routes. Thus, even if the agency evaluators failed to permit Rand McNally to demonstrate this feature during the FTD and even if that failure was improper, the protester was not harmed since the agency has reasonably concluded that the feature, as described by the protester, does not meet the RFP requirements.

PRINTING REQUIREMENT

The agency also found that the protester's software did not have the ability to print a "route list report" as it argues the RFP required. According to the agency, the only way Rand McNally's software could create a printed copy of the necessary information was to "send the `on screen report'" to the printer by means of the computer's "print screen" function. This feature allows the user to print the data displayed on the computer screen at any given time. In order to generate a "report," the user must repeatedly print the information displayed on the computer screen until all the required data is printed. The agency notes that "print screen" does not send page breaks so that, without additional instructions from the user, printing continues over page perforation The agency concludes that since the process was time-consuming, largely manual, and produced a printed product which included a significant amount of extraneous material, the protester's software did not comply with the RFP printing requirement.

Rand McNally maintains that its software complies with RFP's printing requirement. The protester argues, essentially, that the RFP sets forth only a minimal printing requirement which it has satisfied by demonstrating that it could print the required information by repeatedly using the "print screen" function of the computer. The protester states that the agency's contrary position imposes unstated methodology and print formatting requirements beyond those contained in the solicitation's specifications. Rand McNally contends that if the agency desired a properly paginated report that could be produced with one keystroke, the agency should have so specified.

The RFP printing specification stated that the software must be capable of printing a "route list" and specified that the "route list report" contain six different categories of information. While the protester characterizes the specification as not requiring any particular format, the specification in fact used the term "report" and required specific information.

According to the agency, its use of the term "report" in the specifications should have informed any reasonable offeror of its need for software which has the capability to create a document with the characteristics "commonly understood" to be inherent in a report. The agency states that a "report" as that term is commonly understood is a document which, for example, contains all the required information and no extraneous information, can be generated quickly and easily, and is formatted to include proper page breaks.

Based on our review of the record, and particularly, the sample print-out of Rand McNally's route list information, we think that the agency reasonably determined that the protester's software can not print a report, as that term is normally understood. In reaching this conclusion, we point out that agencies must apply solicitation provisions "in the light of common sense, general knowledge, and experience." Software Innovations, Inc., GSBCA No. 11035-P, Jan. 14, 1991, 91-1 BCA Para. 23,662, 1991 BPD Para. 11.

Here, the protester's interpretation of the printing requirement is at odds with the obvious purpose of the solicitation in that it ignores the fact that the reports are to be used with laptop computers by investigators in the field to conduct audits and inspections. When viewed in this context, we find interpretation of the RFP is that offerors were to provide software that was able to produce a readable and usable report, not merely information which the user must shape into a usable format. Since the protester's software lacked the capability to produce a document which, under these circumstances, could be considered a report, the agency reasonably found the protester's proposal deficient in this area as well.

EXCLUSION OF PROTESTER'S PROPOSAL FROM COMPETITIVE RANGE

We are mindful of the fact that Rand McNally's exclusion from the competition limited the competitive range to one offer. Although we will closely scrutinize an agency decision which results in a competitive range of one, we will not disturb such a determination unless it is unreasonable. Native Am. Consultants, Inc.; ACKO, Inc., B-241531; B-241531.2, Feb. 6, 1991, 91-1 CPD Para. 129.

The RFP specifically warned that the failure of an offeror's software to meet all mandatory specifications at the FTD could result in rejection of the offer. Here, for the reasons cited above, the agency reasonably concluded as a result of the FTD that Rand McNally's software failed to satisfy two of the mandatory specifications. Further, the record shows that, without major revisions, the protester's software simply could not perform in accordance with the specifications. Based on our review of the record, we find that the FTD results provided "strong evidence" that, in terms of the RFP, Rand McNally's proposed software was fundamentally flawed. NCR Corp., supra; Burroughs Corp., B-202316, June 8, 1981, 81-1 CPD Para. 460. We thus conclude that the agency had a sufficient basis to exclude the offer from the competitive range even though only one other remained.

The protest is denied.