Matter of: Nevada Automotive Test Center File: B-251137 Date: March 9, 1993

Matter of: Nevada Automotive Test Center File: B-251137 Date: March 9, 1993

PROCUREMENT Competitive Negotiation Offers Competitive ranges Exclusion Administrative discretion Proposal was properly eliminated from the competitive range where the agency reasonably concluded that deficiencies in technical approach, personnel qualifications, facilities, and project management rendered the proposal technically unacceptable.

Attorneys

DECISION Nevada Automotive Test Center (NATC) protests the elimination of its proposal from the competitive range under request for proposals (RFP) No. DTNH22-92-R-07003, issued by the National Highway Traffic Safety Administration (NHTSA), Department of Transportation (DOT), for vehicle dynamics research. NATC contends that the agency improperly rejected its proposal on the basis of an undisclosed evaluation criterion.

We deny the protest.

The Office of Crash Avoidance at NHTSA is involved in research to evaluate current and new vehicle technologies and to analyze the heavy truck operating environment, including truck use and accident involvement. The RFP contemplated award of an indefinite quantity/task order contract to perform vehicle dynamic analysis in areas such as vehicle design and performance characteristics; effectiveness of crash avoidance countermeasures; environmental factors; and empirical and test-track evaluations. The statement of work (SOW) provided a nonexclusive list of 10 types of task orders which included testing and demonstrations of heavy trucks and equipment, development of computer models for performance attributes, and performance of analyses and comparisons of heavy vehicle components.[1] The SOW stated that specific methods for analysis could be specified by the agency or developed by the contractor.

To assist in the evaluation of the offerors' technical capability, the RFP included a hypothetical task.[2] Each offeror was required to submit a detailed outline and discussion of its response including specific tasks and methods the offeror would use to develop a relevant approach for studying the issue. The RFP provided that technical considerations were of primary importance, with award to be made to the responsible offeror whose proposal was most advantageous to the government, price and other factors considered.

Three offerors, including NATC and the University of Michigan, Transportation Research Institute (Michigan), submitted proposals by the closing date. The proposals were evaluated by an Evaluation Board in four areas in descending order of importance: "Technical Approach to Hypothetical Task" (350 points); "Personnel Qualifications" (300 points); "Facilities, Capabilities and Related Corporate Experience" (250 points); and "Project Management" (100 points), for a total of 1,000 points.

Initial evaluations of the three proposals resulted in an assessment of Michigan's proposal as technically acceptable (875 point score), NATC's proposal as technically unacceptable, but with correction potential (630), and the third proposal as technically unacceptable (390). The Board determined to include the Michigan and NATC proposals in the competitive range and to reject the third offeror's proposal. While the Board found deficiencies in NATC's proposal in all four areas, the evaluators were most concerned with NATC's response to the hypothetical task. Among other things, they found that NATC had misinterpreted the task, proposed a "shotgun" approach including a number of unnecessary tests, and a solution based primarily on testing with insufficient analytical work.

The Board conducted written discussions with the offerors in both technical and cost areas. Based upon its review of NATC's responses to the discussion questions, the Board downgraded the protester's proposal in the areas of technical approach, personnel, and facilities, to a score of 542 points. Michigan's responses resulted in an improvement in its proposal's score to 878 points. Since NATC's proposal did not improve as a result of discussions, the Board recommended that the proposal be eliminated from the competitive range. The contracting officer found that NATC could not be expected to improve its technically unacceptable offer through further discussions to a point where it would have a reasonable chance of receiving the award. The agency notified NATC of its proposal's elimination and continued negotiations with Michigan. DOT awarded a contract to Michigan on September 24, 1992. NATC filed a protest with DOT and, after a debriefing in October, filed this protest with our Office.

In a negotiated procurement, the contracting agency may select a competitive range which consists of all proposals that have a reasonable chance of being selected for award; that is, it includes those proposals that are technically acceptable as submitted or that are reasonably susceptible of being made acceptable through discussions. Delta Ventures, B-238655, June 25, 1990, 90-1 CPD Para. 588; Federal Acquisition Regulation Sec. 15.609. Since agencies are responsible for defining their needs and for deciding the best method of accommodating them, the evaluation of proposals and the resulting determination of whether an offer is in the competitive range are matters within the discretion of the contracting agency. Information Sys. & Networks Corp., 69 Comp.Gen. 284 (1990), 90-1 CPD Para. 203. In reviewing protests against these determinations, our function is not to reevaluate the proposal, but instead to examine the agency's evaluation as a whole to ensure that it has a reasonable basis and is in accord with the evaluation criteria listed in the solicitation. Abt Assocs. Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD Para. 223.

NATC contends that the agency preferred computer analysis over empirical testing even though the RFP called for a balance of testing and analysis. According to NATC, this "preference" constituted an undisclosed evaluation factor and thus, rendered the evaluation defective. The agency responds that it has no preference for analysis over testing. Rather, it views the requirements of each task as determining whether testing, analysis, or a blend of both is appropriate. According to the agency, NATC's proposal was evaluated as unacceptable because of its deficient response to the hypothetical task and other weaknesses which were not resolved by the protester's responses to discussion questions. We agree.

The protester does not specifically take issue with the deficiencies identified by the Evaluation Board; rather, NATC concentrates on the evaluators' narrative comments and statements at the debriefing which it believes establish the agency's preference for computer analysis. According to NATC, these comments and statements establish an agency preference for computer analysis which pervaded the entire evaluation.

Based on our review of the record, we conclude that the comments and statements relate to the Board's reasonable assessment that NATC incorrectly approached the hypothetical task, and do not reflect any improper undisclosed preference on the part of the agency. For example, one evaluator observed that NATC's staff members were competent, but were engineers rather than "researchers," while praising Michigan's ability in research. The RFP required offerors to show evidence of qualified personnel with relevant research experience including the ability to develop effective research programs in addition to developing testing methods and procedures. Michigan identified its primary staff members as research scientists/engineers and emphasized their relevant experience in developing and implementing research projects including testing and computer analysis. While NATC's staff list some experience in testing and computer analysis, NATC characterized its primary staff as test engineers and emphasized their testing capability in various projects without making clear the staff's experience or capability in overall research project development. We believe the evaluation comment related to the accurate assessment that NATC was more test oriented in its approach.

NATC also focuses on evaluation comments which express a belief that NATC's university team member, which was evaluated as having substantial computer analysis expertise, should have been in the prime role, with NATC as the subcontractor. NATC contends that these comments, when coupled with certain statements made during the debriefing, indicate a preference for computer analysis. At the debriefing, one DOT representative stated that what the agency wanted was primarily a computer-generated analysis capability with testing as a subsidiary capability to be used mainly at the component level. The representative continued that DOT did not need or want testing on the scale that NATC proposed, and that it was concerned that NATC would place too much emphasis on testing while the agency was looking for analysis capability.

The agency does not deny that these statements were made, but points out that they were only part of the debriefing. According to the agency, the debriefing covered NATC's overall technical unacceptability. Our own review of the context of debriefing and evaluation statements lead us to the conclusion that the agency did evaluate NATC's proposal on the basis of any inappropriate overall preference for computer analysis.

For example, after noting the university team member's extensive experience in vehicle dynamics modeling, one evaluator expressed concern that NATC, as prime, would be inclined to try to do the work itself. Based on past experience, another evaluator expressed concern that NATC could have control problems with its analysis team member with adverse consequences to timely accomplishment of a task. Based on NATC's poor engineering judgment in overemphasizing testing at the expense of analysis in the hypothetical task (discussed below), it is plain that the evaluators were concerned with how NATC would perform other tasks under the contract. The debriefing statements, as a whole, appear to be aimed at advising NATC of its overall technical deficiencies. In doing so, the statements highlighted NATC's emphasis on testing on a scale inappropriate for the hypothetical task and as evidence of a lack of judgment regarding the overall research effort. Both the evaluations and the debriefing reflect the evaluators' assessment of the protester's problems in this regard, but do not establish any improper agency preference for computer analysis over testing.

With regard to the exclusion of its proposal from the competitive range, we will not disturb the decision unless it was unreasonable or in violation of applicable procurement statutes and regulations. See Institute for Int'l Research, B-232103.2, Mar. 15, 1989, 89-1 CPD Para. 273.[3] Based on our review of the RFP, the proposals, and evaluations, we find that the evaluation of NATC and the resulting competitive range determination were reasonable.

According to the RFP, the technical approach to the hypothetical task factor was the most important and heavily weighted factor in the evaluation. The RFP advised offerors to demonstrate that their approaches were sound, including clarity, feasibility, and technical correctness. Offerors were advised to balance analytical evaluation versus empirical test-track evaluation and were given notice that the proposal should indicate competence in both areas.

In reviewing NATC's task proposal, the Board found that NATC had extensive capabilities in testing and had teamed with a university with extensive computer modeling capability. However, it found that NATC had proposed a "shotgun" approach, and had incorrectly interpreted the task requirements to include matters well beyond the evaluation required by the task. NATC had proposed an extensive series of tests and evaluations of such things as pavement designs and tire dynamics analyses which would be irrelevant or unnecessary to successfully complete the delineated task. The Board found the proposal lacking in clarity as to how NATC would measure or otherwise successfully obtain necessary parametric data for use in its proposed computer analysis. Further, contrary to the requirements of the RFP, the proposal failed to list all the equipment necessary to obtain necessary parametric data and failed to identify which personnel would be responsible for which parts of the task. Overall, the Board concluded that the task response was unacceptable because it placed too much emphasis on testing and not enough on the analysis necessary to complete the particular task. NATC received an average score of 196 of the 350 points allotted to this evaluation factor.

In discussions, the Board requested a matrix of labor mix, roles, and level of effort of staff and subcontractors to complete the task. The Board also questioned the length of time NATC proposed for development of its computer model and asked if it could be performed more efficiently or faster. In response, NATC provided three alternative solutions to the task, along with the requested matrix. The Board found the response largely unacceptable because two of the alternative solutions expedited the task as a whole, emphasizing testing with little computer analysis, and reflected an unrealistic time commitment, while the third alternative, the original solution, continued to present an inadequate "shotgun" approach. Although the Board believed that the testing and analysis mix was better, it still found that this solution placed an inappropriate emphasis on testing. The Board downgraded NATC's average score for this factor to 167.

In addition to the deficiencies noted in the hypothetical task response, the Board found deficiencies and weaknesses in other factors. For example, under project personnel, the Board was concerned that despite NATC's teaming with a university for computer analysis work, its proposal indicated that the offeror would concentrate more on testing than on using the computer capabilities of its team member. With regard to the facilities factor, the initial proposal raised a question of whether NATC possessed a tilt table capable of making full-scale vehicle rollover- threshold determinations. NATC's response to a technical question to list all relevant equipment possessed or needed to be acquired continued to leave unclear whether it possessed an adequate tilt table. NATC's average score for the remainder of the factors was originally 426 of the total 650 points, but was downgraded to 375 points.

It is proper for an agency to determine whether to include a proposal within the competitive range by comparing the proposal evaluation scores and the offeror's relative standing among its competitors. A proposal that is technically unacceptable, but capable of being made acceptable, need not be included in the competitive range when, relative to other acceptable offers, it is determined to have no reasonable chance of being selected for award. See Information Sys. & Networks Corp., supra; Institute for Int'l Research, supra. Here, NATC's proposal was included in the competitive range during the first round of discussions. NATC failed to improve its rating of "unacceptable" after revising its proposal, and its total score dropped from 630 to 542 points out of the possible 1,000, while Michigan's proposal score increased to 878 points. In view of NATC's proposal's continuing deficiencies and its resulting relatively low score, the agency reasonably determined that the protester had no reasonable chance for award and, accordingly, properly eliminated its proposal from the competitive range.

NATC also argues that its price was more advantageous than Michigan's price. While NATC may have offered to perform the contract at the lower price, it also submitted a technically unacceptable proposal. NATC's lower price is therefore irrelevant, since once a proposal is found technically unacceptable, it cannot be considered for award. Drytech, Inc., B-246152.2, Feb. 24, 1992, 92-1 CPD Para. 217.

The protest is denied.

1. The RFP expresses no preference for one type of research over another. While foreseeable tasks could include pure testing, pure analysis, or a mixture, DOT was unable to anticipate what tasks or how much of which type of research would be required. In fact, only $2,500 in work is guaranteed and there is no guarantee of the type of work to be involved.

2. The task was entitled "Analysis of a Semi-Trailer Load-Sensing and Self-Leveling Pneumatic Suspension." The task advised offerors to assume that a new pneumatic trailer suspension had been developed, fabricated, and installed on a van trailer. The objective was to assess the performance of the suspension with a stated purpose of gauging the potential benefits that such a suspension may offer in reducing the rollover propensity of heavy trucks. The level of effort was listed as 2.5 person years over a 15-month performance period.

3. NATC points out that our Office will closely scrutinize an agency decision that results in a competitive range of one. Herley Indus., Inc., B-237960, Apr. 5, 1990, 90-1 CPD Para. 364, aff'd, B-237960.2, Aug. 29, 1990, 90-2 CPD Para. 173. In this case, NATC was afforded an opportunity to improve its proposal before the competitive range was reduced to a single offer.