Skip to main content

Matter of: JW Associates Inc. File: B-275209 Date: January 30, 1997 * Redacted Decision

B-275209 Jan 30, 1997
Jump To:
Skip to Highlights

Highlights

Agency fails to rebut protester's specific allegations that its proposal and that of the awardee were misevaluated. JW alleges that the Forest Service's evaluation of proposals was inconsistent with the terms established in the RFP. The RFP was issued on August 6. Offerors were instructed to submit separate technical and business proposals. The RFP at M4 stated that award would be made to "that offeror (1) whose proposal is technically acceptable and (2) whose technical/cost relationship is the most advantageous to the Government.". Which were then averaged. Proposals were ranked on the basis of the average scores. Two proposals were determined to be technically unacceptable. Business proposals were evaluated based on the total price offered for the base and option items.

View Decision

Matter of: JW Associates Inc. File: B-275209 Date: January 30, 1997 * Redacted Decision

Source selection cannot be determined to be reasonable where evaluation record contains no meaningful evaluation of offers, but only conclusory statements which do not permit an understanding of the technical differences between proposals, and agency fails to rebut protester's specific allegations that its proposal and that of the awardee were misevaluated.

Attorneys

DECISION

JW Associates Inc. protests the award of a firm, fixed-price contract to Natural Resources Management Corporation (NRMC) under request for proposals (RFP) No. RM-96-37, issued by the Forest Service for services to complete an environmental impact statement for the Cold Springs Analysis Area on the Medicine Bow National Forest in Wyoming. JW alleges that the Forest Service's evaluation of proposals was inconsistent with the terms established in the RFP, and that the awardee's proposal did not conform with the RFP's requirements.

We sustain the protest.

The RFP was issued on August 6, 1996, as a total small business set-aside procurement. Offerors were instructed to submit separate technical and business proposals, to allow technical merit and cost to be considered separately. The RFP at M3 listed the following five evaluation criteria in descending order of importance: qualifications of the firm; qualifications of the personnel assigned to the project; past experience of the firm and its employees assigned to the project; geographic location; and price. The RFP characterized the first three criteria as "very important" and the remaining two as "important." The RFP also listed the type of information that should be provided under each criterion. The RFP at M4 stated that award would be made to "that offeror (1) whose proposal is technically acceptable and (2) whose technical/cost relationship is the most advantageous to the Government." The RFP further confirmed that it would perform a cost/technical trade-off to determine the most advantageous offer.

The Forest Service received proposals from [deleted] firms, including JW and NRMC. An evaluation board, made up of three Forest Service specialists, evaluated the technical proposals. Each member assigned a technical score to each proposal, which were then averaged; proposals were ranked on the basis of the average scores. Two proposals were determined to be technically unacceptable. Of the remaining [deleted] proposals, NRMC's received the [deleted] averaged technical score with [deleted]. JW's proposal received the [deleted] highest score with an average of [deleted] points.

Business proposals were evaluated based on the total price offered for the base and option items, as provided in the RFP. NRMC's price was lowest among the proposals that were rated technically acceptable, and JW's was sixth low.

The board prepared a "Memorandum for Record" that consisted of a one- paragraph narrative summary of each proposal's evaluation. The contracting officer concluded that NRMC's proposal was the one that was most advantageous to the government, and awarded the contract to this firm on October 7. When JW received notice of the award on October 9, it requested a debriefing. JW received a debriefing letter and a copy of the awardee's technical proposal on October 17. [1] This protest followed on October 22. Because our Office notified the Forest Service of the protest within 5 days of the statutorily-required debriefing, the agency was required, under 31 U.S.C. Sec. 3553 (1994), to stay the performance of the contract pending resolution of the protest. Although the agency initially did stay performance, on November 6, the head of the contracting activity approved a request by the forest supervisor to override the stay and proceed with performance. [2]

JW argues that the Forest Service's evaluation of proposals and its resulting source selection decision were improper and inconsistent with the evaluation scheme that was established in the RFP. Comparing its own proposal to the awardee's, JW concludes that it should have received a significantly higher score under those criteria, if properly applied. For example, under the first (and therefore most heavily weighted) criterion, "qualifications of the firm," JW points out that it has recently completed two timber sale environmental impact statements and is currently performing a third one for the Forest Service in roadless areas in National Forests in Idaho. JW notes a number of factors that are very similar to the requirement at issue here, including the location in roadless areas, extensive areas of older lodgepole pine, the presence of risk/damage from mountain pine beetles, extensive intermixed land ownership, and the importance of water quality and biodiversity to the site. In addition, JW points out its direct experience with ecosystem management under those contracts, and that it offers extensive experience with the National Environmental Policy Act of 1969 (NEPA), 42 U.S.C. Sec. 4321 et seq. (1994), the legislation that prescribes environmental impact studies. JW asserts that its current experience with the Forest Service includes project management, technical management, public involvement, technical writing, and hydrology expertise, all skills essential to performing this contract. On the other hand, JW points out that the awardee's proposal reflects no experience in completing an environmental impact statement. The protester notes that NRMC does not mention in its proposal experience with NEPA, project management, technical management, public involvement, technical writing or ecosystem management; it lists only forestry experience. JW concludes that its own score for this criterion should have been higher than NRMC's. JW provides a similar comparison of its proposal with the awardee's on the remaining technical factors which supports its view that its technical proposal should have been evaluated significantly superior to the awardee's.

In reviewing protests against allegedly improper evaluations, it is not our role to reevaluate proposals. Rather, our Office examines the record to determine whether the agency's judgment was reasonable and in accord with the RFP's stated evaluation criteria. Abt Assocs., Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD Para. 223. In order for us to review an agency's selection determination, an agency must have adequate documentation to support its selection decision. Arco Management of Washington, D.C., Inc., B-248653, Sept. 11, 1992, 92-2 CPD Para. 173. While both adjectival ratings and point scores are useful as guides to decision-making, they generally are not controlling, but rather, must be supported by documentation of the relative differences between proposals, their weaknesses and risks, and the basis and reasons for the selection decision. Federal Acquisition Regulation (FAR) Sec. 15.608(a)(3) and 15.612(d)(2); Engineering and Computation, Inc., B-261658, Oct. 16, 1995, 95-2 CPD Para. 176.

After reviewing all of the supporting documentation submitted by the Forest Service, we conclude that the technical evaluation is not adequately supported. We also conclude that without adequate support for the technical evaluation, a proper award determination could not be made. Engineering and Computation, Inc., supra; see Redstone Technical Servs.; Dynamic Science, Inc., B-259222 et al., Mar. 17, 1995, 95-1 CPD Para. 181.

The evaluation record consists of a chart which lists the offerors, the three evaluators' total scores for each offeror, the average score of the evaluators, and each firm's price. This chart ranks firms according to their price. The record also contains a memorandum for the record. The memorandum consists of a one-paragraph evaluation summary of each offeror. For NRMC, the memorandum states:

"Natural Resources Management Corporation presented a proposal which offered to do the most for the money, or in other words, providing the most acceptable proposal whose technical/cost relationship was the most advantageous to the Government. The firm was considered to be highly qualified having lots of experience in the land management field and forestry. The firm proposed establishing a field office in Wyoming for the duration of this project which would be of benefit to the Forest. The only concern of the Board's was that the firm did not have any roadless experience. But this lack of experience was not enough to eliminate the firm from being placed in the competitive range."

Regarding JW's proposal, the memorandum states:

"JW Associates, Inc., was ranked [deleted] making the most advantageous offer to the Government. The proposed project schedule was considered to be good by the Board. The firm proposed using primarily two employees and using a lot of subcontractors. It appeared that the subcontractors had good experience working together. The primary reason for this firm's ranking was due to its cost."

Further, in response to the protest, the Forest Service prepared a document entitled "Agency Reply to Protest," in which it generally states that the evaluation was fairly performed in accordance with the solicitation and responds to each of the protester's allegations with conclusory statements. For example, for the protest basis at issue, that technical proposals were misevaluated, the agency reply repeats the evaluation criteria listed in the RFP and states that:

"Each Board member evaluated each technical proposal independently and rated each proposal based upon what was submitted. Once all of the technical proposals were independently evaluated, the Board members then discussed amongst themselves the technical proposals. As a result of this discussion, technical proposals were determined to be either acceptable or not acceptable to the Government.

The Contracting Officer directed the Board members to evaluate each proposal using only the evaluation criteria listed in the solicitation. An evaluation form was used by the Board to assist with the evaluations. The form listed each of the evaluation criteria and their relative value. No other criteria were used during the evaluation. A Memorandum for Record [from which we quoted in pertinent part, above] was prepared providing reasons for the rankings."

The report does not include any individual technical evaluation sheets or other contemporaneous documentation of the evaluation of the various evaluation criteria, but provided only the chart with the total score that each evaluator assigned to each proposal. In contrast to the very specific and detailed allegations that the protester has raised, pointing out specific areas in its proposal that allegedly should have been evaluated more highly, such as under "qualifications of the firms," discussed above, the Forest Service has provided no specific rebuttal, nor has it provided any support or explanation for its conclusions. Other examples of unrebutted, specific allegations include the protester's assertion that it should have been evaluated higher than the awardee because its own proposed personnel are more highly qualified than the awardee's, an allegation that it supports by comparing the education, background and experience of its own personnel and the awardee's proposed personnel; its assertion that the level of effort that NRMC proposes is inadequate to complete the work, pointing out, for example, that the awardee proposes to spend a total of two-thirds fewer hours on project management compared to JW's proposed hours, and the assertion that NRMC's proposal fails to include a detailed work plan or to disclose the firm's technical approach in sufficient detail, as required by the RFP. None of these issues is addressed in the one-paragraph summary, or otherwise explained in the record. In fact, there is nothing in the record furnished by the Forest Service which shows how each proposal was evaluated under each technical evaluation factor. Thus, we have no basis on this record to assess the reasonableness of either JW's or NRMC's evaluation.

In addition to the Forest Service's failure to support its technical evaluation in any meaningful way, the record suggests that the source selection decision was based primarily on price--the least important factor listed in the evaluation scheme. With the exception of the paragraph in the Memorandum concerning NRMC's proposal, quoted above, every paragraph in the Memorandum summarizing the agency's basis [3] for its ranking of the individual proposals concludes that "[t]he primary reason for this firm's ranking was due to its cost." Indeed, as stated above, the ranking of the proposals [deleted]. The contracting officer's debriefing letter confirms that price was the basis for the agency's award decision.

While price, where lower in importance, can become the determining factor between proposals that have been evaluated as technically equal in a "best value" award decision, Ogilvy, Adams & Rinehart, B-246172.2, Apr. 1, 1992, 92-1 CPD Para. 332, the record in this case does not contain sufficient information and analysis to establish the equality of these two proposals; moreover, there is no evidence in the record that any such conclusion was reached. To the contrary, as discussed above, the protester has provided an analysis of its proposal and the awardee's which reasonably shows that its proposal is superior to the awardee's in the three most heavily weighted technical areas--qualifications of the firm, personnel, and past experience. Further, the protester points to the RFP evaluation language which ranked technical merit as significantly more important than price. JW states that its higher price was based on its proposal of high quality personnel and a level of effort commensurate with the agency's needs as announced in the RFP.

FAR Sec. 15.612(d)(2) requires that the documentation supporting selection decisions show the relative differences among proposals; their strengths, weaknesses and risks; and the basis and reasons for the decisions. This required explanation provides protesters and this Office a basis upon which to judge the reasonableness of the agency's decision and, ultimately, its compliance with the procurement statutes and regulations. TFA, Inc., B-243875, Sept. 11, 1991, 91-2 CPD Para. 239. Where, as here, the agency has not adequately documented its evaluation process and has failed to rebut the protester's very specific arguments that its own proposal should have received a higher technical score than the awardee's, we have no basis to conclude that the selection decision was reasonable.

The protest is sustained.

By letter of today to the Secretary of Agriculture, we are recommending that the agency, in accordance with the FAR, reevaluate the proposals, document its evaluation, and make a new selection decision. [4] If after reevaluation, the Forest Service believes discussions with offerors are warranted, it may open discussions and request best and final offers. If the agency selects an offeror other than NRMC for award, it should terminate the contract to NRMC and make award to that other offeror. We also recommend that the protester be reimbursed its costs of filing and pursuing its protest. Bid Protest Regulations, Sec. 21.8(d)(1), 61 Fed. Reg. 39039, 39046 (1996) (to be codified at 4 C.F.R. Sec. 21.8). The protester should submit its certified claim for costs to the contracting agency within 60 days of receiving this decision. Bid Protest Regulations, Sec. 21.8(f)(1), supra.

The protest is sustained.

Comptroller General of the United States

1. For reasons that are not clear, the Forest Service provided JW with a copy of the awardee's technical proposal along with its debriefing letter.

2. The statute permits an override of the stay on the basis of either urgency or best interests of the government. The override determination does not identify the basis for the override.

3. The RFP creates confusion concerning the evaluation scheme. While M3 and M4(2) envision a best value approach, M4(1) introduces the concept of technical acceptablility which is a scheme that results in award to the low cost offeror. It appears from the record that the Forest Service may have inappropriately followed the latter approach.

4. Although JW raises a number of challenges to the evaluation and source selection decisions in its protest, we have not discussed each protest basis. However, we recommend that the Forest Service, during its reevaluation of proposals, take into consideration JW's allegations that NRMC's proposal does not meet the requirements of the RFP because it fails to provide certain information.

DOCUMENT FOR PUBLIC RELEASE A protected decision was issued on the date below and was subject to a GAO Protective Order. This version has been redacted or approved by the parties involved for public release.

GAO Contacts

Office of Public Affairs