B-234250, May 30, 1989, 89-1 CPD 517

B-234250, May 30, 1989, 89-1 CPD 517

PROCUREMENT - Competitive Negotiation - Requests for proposals - Evaluation criteria - Sufficiency DIGEST: 1. Technical evaluation that found awardee's proposal superior based on factor (expert review) not explicitly identified in solicitation was proper because this factor was reasonably related to stated technical factor measuring the quality and pertinence of technical approach. PROCUREMENT - Competitive Negotiation - Discussion - Adequacy - Criteria 2. Agency was not required to discuss matters with an offeror that are not related to the solicitation's minimum requirements, i.e., matters that do not render a proposal deficient.

Jeffrey A. Cantor:

Jeffrey A. Cantor, protests the award of a contract to the Institute for the Study of Adult Literacy, Penn State University (Institute), under request for proposals (RFP) No. 88-1, issued by the National Commission for Employment Policy (NCEP). The protester principally alleges that discussions were inadequate and that the evaluation proposals was not conducted in accordance with the solicitation's stated evaluation criteria.

We deny the protest.

The solicitation requested offers for the design of an evaluation plan to assess the appropriateness and effectiveness of computer-based instructional packages used in the remediation of basic skills for individuals served by the Job Training Partnership Act (JTPA) (i.e., unemployed, economically-disadvantaged youths and adults and dislocated workers). The solicitation specified that this design was to include, among other elements, a description and preliminary assessment of current knowledge regarding the use of such instructional packages by JTPA clients, or individuals with similar needs, as well as a summary of existing studies regarding the cost effectiveness of these packages. The solicitation cautioned offerors to indicate the methodology and sources used for this information, such as reliance on literature reviews, vendor interviews, and expert panels. The solicitation also provided that proposals would be evaluated on the basis of the following five technical factors: (1) understanding of objectives (20 Percent); (2) quality and pertinence of technical approach (30 percent); (3) knowledge and experience in computer assisted instructional packages (20 percent); (4) knowledge and experience in evaluation design (20 percent); and (5) knowledge of the use of packages with JTPA clients (10 percent).

Nine offerors responded to the solicitation. A technical evaluation panel evaluated and scored the initial proposals for technical merit. Based on these findings, the contracting officer included three firms, including Cantor and the Institute, in the competitive range. The initial weighted scores for Cantor and the Institute for the five technical factors were 86.7 and 83, respectively. Price was not point scored during the initial evaluation.

Discussions were held with each of the three offerors concerning deficiencies noted in their proposals, and each was requested to submit a best and final offer (BAFO). Specifically, NCEP identified two principal deficiencies in Cantor's proposal: (1) it proposed a limited number of instruction sites and instructional packages to be included in the study; and (2) it demonstrated insufficient expertise by Cantor in the required work.

The evaluation panel conducted a final review of the BAFOs for each of the evaluation factors and for price; the panel did not rescore proposals but, instead, merely summarized the strengths and weaknesses of the three remaining offerors. Cantor was found to have submitted a well-written proposal which demonstrated a great deal of experience and knowledge in both the JTPA and computer-assisted instructional packages. The panel was concerned, however, that Cantor, despite assurances in his BAFO, would not expand his examination and analysis of computer-assisted instructional packages beyond the four packages listed in his initial proposal and, more importantly, that because he intended to work alone, agency personnel would be solely responsible for reviewing the quality and comprehensiveness of Cantor's work product. In contrast, the panel found that the Institute's proposal, which along with Cantor's was "judged the best," offered extensive institutional support as well as use of expert panels to review the adequacy of its work product prior to delivery to the NCEP.

Although it found Cantor's proposal to be impressive, and Cantor offered the low price of $14,593 as compared to the Institute's evaluated price of $14,593 as compared to the Institute's evaluated price of $27,360, the panel concluded the "the vast amount of institutional support, outside reviewers and other consultants provided by the Institute" outweighed the cost advantages of Cantor's proposal, and thus recommended the Institute for award. The contracting officer agreed and made award to the Institute shortly thereafter.

Cantor argues that the selection of the Institute for contract award on the basis of its institutional support was inconsistent with the stated evaluation factors and in fact demonstrated a predisposition on the part of the NCEP to secure a college-based organization offering more manpower and institutional support than the RFP required. Cantor asserts that noe of the five listed evaluation factors encompassed institutional support provided by an offeror, and that had he been advised of this new requirement during discussions, he would have provided this support. concludes that, had the award selection been made in accordance with the stated factors, he would have received the award. In this regard, Cantor notes that his proposal received the highest initial technical score and also was nearly 50 percent less expensive than the Institute's.

It is well-settled that a solicitation must inform all offerors of the basis for evaluation of proposals and that the evaluation must in fact be based on the schemem set forth in the solicitation. Human Resources Research Organization, B-203302, July 8, 1982, 82-2 CPD Para. 31. While agencies thus must inform offerors of all major evaluation factors, they need not expressly identify the various aspects of each which might be taken into account, provided that such aspects are reasonably related to or encompassed by the stated criteria. Id.; Bell and Howell Corp., B-196165, July 20, 1981, 81-2 CPD Para. 49.

As Cantor correctly points out, the availability of institutional support, outside researchers, and expert consultants to review the finished work product prior to its delivery to the government were not specified as requirements in the RFP. This was because the NCEP did not intend to require any of these elements. In fact, the RFP did not require any particular methodology, leaving it instead to each offeror to approach the literature reviews and other research requirements in the manner the offeror believed was best suited to successful completion of the project. Both the Institute and Cantor did this in their proposals, the resulting relevant difference in the two being that the Institute offered expert panel review of the final product, while Cantor proposed performing all the work himself with no review. Both offerors' approaches were acceptable to the NCEP, but the agency determined that expert review of the final product made the Institute's proposal more desirable because it would assure consideration of the product by other experts in the field while reducing the burden on agency personnel to conduct an in-depth review of the quality of the product.

This evaluation was consistent with the RFP's evaluation scheme. The most significant evaluation factor (30 percent of the technical evaluation) measured the "quality and pertinence" of the technical approach. This obviously is a very broad standard, and we think it clearly must be read to encompass all aspects of an offeror's proposed methodology for preparing a quality final product. Expert review and other institutional support were aspects of the Institute's methodology bearing on the quality of the work to be performed and as such, in our view, fell within the bounds of the "quality and pertinence" evaluation factor. We think the benefits of expert review are self-evident (Cantor does not argue otherwise) and thus find nothing unreasonable in the NCEP's conclusion that this review rendered the Institute's proposal superior to Cantor's.

Moreover,, as the remainder of the evaluation focused on the offerors' knowledge of instructional packages and evaluation design, we think consideration of the additional knowledge lent to the project by the Institute's proposed expert reviewers was permissible under these factors as well. Although Cantor received high scores in the original evaluation for his experience and knowledge, we think it was entirely reasonable for the NCEP to conclude in the final analysis that having the final product reviewed by other experts in the field warranted rating the Institute superior under this factor. In sum, the stated evaluation factors were broad enough to encompass consideration of the potential advantages of the Institute's expert review approach. See ORI, Inc., 67 Comp.Gen. 600 (1988), 88-2 CPD Para. 192.

Furthermore, although Cantor's offered price of $14,593 was almost 50 percent lower than the Institute's $27,360 price, we see nothing unreasonable in the NCEP's ultimate determination that the potential higher quality final product and reduced burden on agency personnel that would result from the Institute's approach warranted the additional $12,767 in cost. ORI, Inc., 67 Comp.Gen. 600, supra.

Finally, we do not agree with Cantor that the NCEP should have advised him during discussions that more favorable consideration would be given to an offeror proposing expert panel review of the final product than one that did not. Agencies are not required to discuss matters that are not related to minimum requirements, that is, that do not render a proposal deficient. Here, as indicated above, the NCEP was impressed with Cantor's proposal and considered Cantor fully capable of performing the required work without the assistance of other individuals having expertise in the areas to be studied; the NCEP did not consider Cantor's proposal deficient for its failure to provide for expert review of the final product, but rather simply viewed the Institute's use of expert review to be a desirable feature warranting a superior technical rating. See ORI, Inc., 67 Comp.Gen. 600, supra.

The protest is denied.