Matter of: J.A. Jones Management Services, Inc. File: B-254941.2 Date: March 16, 1994 94-1 CPD 244 REDACTED VERSION[*]

Matter of: J.A. Jones Management Services, Inc. File: B-254941.2 Date: March 16, 1994 94-1 CPD 244 REDACTED VERSION[*]

PROCUREMENT Competitive Negotiation Requests for proposals Evaluation criteria Cost/technical tradeoffs Weighting Where the RFP contains separate and independent technical evaluation factors encompassing separate subject areas, including staffing, technical plans, safety, key personnel, past experience, corporate support, and continuity of services, with each factor assigned separate numerical weights, an agency may not double count, triple count, or otherwise greatly exaggerate the importance of any one listed factor; stated differently, where an RFP lists a number of evaluation factors of stated importance, a single one cannot be accorded more than the weight prescribed in the RFP's evaluation methodology by the agency repeatedly considering the same factor in conjunction with the other major factors.

Attorneys

DECISION

BACKGROUND

The RFP, issued March 3, 1993, sought proposals to provide labor, supervision, equipment, and materials (except those specified as government-furnished) for operation and maintenance support services for the Department of Agriculture Albany Research Centers, California. Services to be provided included facilities maintenance and management (boiler plant systems, utilities systems, buildings maintenance, grounds care, and sanitation services); material receiving, storage, and shipping; and administrative support services. The RFP contemplated an initial phase-in period, a basic performance period of 12 months, as well as four option periods. The RFP stated that options would be evaluated and contained the standard clause, "Contract Award" (Federal Acquisition Regulation Sec. 52.215-16), which stated that award would be made to the responsible offeror whose offer conforming to the solicitation was most advantageous to the government, cost or price and other factors considered. Section M of the RFP ("Basis for Award") stated that cost and cost-related factors would play a significant role in determining contract award and that the contracting officer would "determine what trade-off between technical and cost promises the lowest overall cost to the government, price and other factors considered." The RFP also stated that as the technical merit of the offerors' proposals became more equal, "cost may become the determining factor."

The RFP contained the following technical evaluation factors:

I. Technical Requirements Point Values

A. Staffing 260 B. Technical Plans 220 C. Safety 70 Subtotal 550

II. Management Requirements

A. Key Personnel 200 B. Past Experience 150 C. Corporate Support and Policies 50 D. Continuity of Services 50 Subtotal 450

Total Possible Technical Points[1] 1,000

The agency received four proposals on April 28, 1993, the closing date for receipt of initial proposals. The technical proposals were submitted to the agency's source evaluation board (SEB) which determined that two proposals did not stand a reasonable chance of being selected for award and were excluded from the competitive range. The two remaining offerors, the protester and RFSS, were considered technically acceptable with correctable deficiencies and were included in the competitive range. The SEB furnished a detailed narrative and point score report to the contracting officer on the results of its evaluation. The SEB ranked the offerors as follows:

Offeror Score Cost Risk

RFSS [DELETED] [DELETED] [DELETED] J.A. Jones [DELETED] [DELETED] [DELETED]

Briefly, the SEB found that the protester had submitted [DELETED], but repeatedly found that the protester's proposed project manager was "not an acceptable Project Manager or leader" for the proposed contract.[2] [DELETED] SEB's finding that the protester's proposed project manager "cannot execute the plans" and "does not possess the necessary skills to implement" the work requirements.[3]

Discussions were then held with the offerors. As relevant here, various discussion questions were posed to J.A. Jones by the agency regarding the protester's project manager, such as: (1) "what special training will be given to the proposed Project Manager to educate him to efficiently utilize [a certain computer system]?"; (2) "what training will be provided to the Project Manager to address the proposed work prioritization methodology?"; (3) "what training is proposed for the Project Manager to improve communication, facilitating, and organization needed to implement the proposed Quality Control Plan?". The agency also expressed concern with the project manager's ability to communicate with his staff and government personnel; his ability to manage/implement the proposed organization, plans and reporting requirements; and his leadership skills.

Best and final offers (BAFO) were received on July 30, 1993. In its BAFO, the protester proposed [DELETED]. The SEB met the week of August 2 to evaluate BAFOs, but was unable to reach a documented consensus score based on the merits and deficiencies of each BAFO; the contracting officer then decided to do her own evaluation.[4] The contracting officer's independent evaluation of BAFOs was as follows:

Offeror Score Cost Risk

RFSS [DELETED] [DELETED] [DELETED] J.A. Jones [DELETED] [DELETED] [DELETED]

In rating the protester's BAFO, the contracting officer's narrative report, like the SEB's initial report, was replete with negative references about the project manager. Indeed, she flatly stated that the contracting officer "feels strongly that [the Project Manager] is not an acceptable Project Manager or leader [for this contract]." Nonetheless, the contracting officer found that J.A. Jones's overall score [DELETED]. The contracting officer recommended to the source selection official (SSO) that award be made to RFSS [DELETED]. The SSO approved the award to RFSS, and this protest followed.[5]

PROTESTER'S CONTENTIONS

The protester contends that the agency's evaluation of proposals was arbitrary and unreasonable. With respect to the project manager, the protester complains that the agency "focused almost exclusively on [alleged] performance problems that were perceived with the Project Manager." The protester insists that, under its current contract, the agency's Award Fee Determination Board has consistently awarded the firm a [DELETED] rating for its current performance ("satisfactory" under Program and Cost Management--the area most directly concerned with the project manager), and that during the BAFO evaluation some of the evaluators correctly noted that marked improvement had occurred in the performance of the project manager during the past several months. According to the protester, the contracting officer unreasonably ignored these findings by the evaluators in her final evaluation of BAFOs. The protester also contends that the agency deviated from the RFP's evaluation structure, methodology, and weights by making the qualification and experience of the project manager the "overriding evaluation factor" which played an important and exaggerated part in every major evaluation factor under the RFP. We agree with the protester that the agency's evaluation was improper.[6]

ANALYSIS

The RFP here contained separate and independent technical evaluation factors encompassing separate subject areas, including Staffing, Technical Plans, Safety, Key Personnel, Past Experience, Corporate Support, and Continuity of Services. Each factor was assigned separate numerical weights, and the subject areas encompassed by each factor that were to be evaluated were specifically set forth in the solicitation. Under such an evaluation methodology, which we have approved, see Mutual of Omaha Insurance Co., B-201710, Jan. 4, 1982, 82-1 CPD Para. 2, an agency may not double count, triple count, or otherwise greatly exaggerate the importance of any one listed factor; stated differently, where an RFP lists a number of evaluation factors of stated importance, a single one cannot be accorded more than the weight prescribed in the RFP's evaluation methodology by the agency repeatedly considering the same factor in conjunction with the other major factors. See The Center for Educ. and Manpower Resources, B-191453, July 7, 1978, 78-2 CPD Para. 21; Management Servs., Inc., B-206364, Aug. 23, 1982, 82-2 CPD Para. 164; Earth Envtl. Consultants, Inc., B-204866, Jan. 19, 1982, 82-1 CPD Para. 43; Mutual of Omaha Insurance Co., supra.

As stated above, the contracting officer's final evaluation of BAFOs consisted of a narrative and scoring report for each offeror that was based in part on the SEB's initial evaluation report, portions of the SEB's inconclusive BAFO draft reports, as well as her own findings. We will review her report and findings in major technical evaluation areas.

The RFP contained a major technical evaluation factor, "Key Personnel" (worth 200 of 1,000 possible points), which required offerors to show who will manage and supervise the work under the contract and each key person's qualifications and recent experience. The SEB, in its initial evaluation report, [DELETED], the proposed project manager, in its view, could not "carry out these plans." The SEB found that the project manager was not a "motivator", caused his current work force to become disgruntled, has not praised his employees for excellent workmanship but has looked for ways to criticize them, has routinely forgotten about recurring monthly reporting requirements, and has very poor communications skills. The SEB "[felt] strongly that [he] is not an acceptable Project Manager or leader for the Albany Facility's contract." In her final evaluation, the contracting officer essentially adopted the initial evaluation results "not[ing] [that the protester] did not change any Key Personnel." Because of this "major deficiency" (unchanged project manager) she assigned the protester a score of [DELETED] which she felt was "extremely generous." She also adopted a lengthy narrative from one of the SEB draft reports which contained extremely disparaging findings about the capabilities of the proposed project manager. She found all other key personnel of the protester to be acceptable.

We think the agency's downgrading of the protester in the area of Key Personnel was reasonable. While the protester argues that the project manager improved his performance during the few months prior to the final evaluation (as stated by some of the evaluators during BAFO evaluations) and that the protester received high Award Fee Determination Board ratings during its performance, we think the contracting officer and other agency personnel with direct knowledge of the protester's performance obviously were in the best position to determine that the project manager's performance was unacceptable. Generally, agencies evaluating proposals may properly consider their own past unsatisfactory experience with an offeror's performance, rather than relying solely on references provided by the offeror. George A. and Peter A. Palivos, B-245878.2; B-245878.3, Mar. 16, 1992, 92-1 CPD Para. 286. The contracting officer's final evaluation report documents specific instances of unacceptable performance by the project manager, including his failure to understand the difference in funded and unfunded special tasks, his inability to read his own fund expenditure report or answer questions as to the basis of his findings, as well as his failure to perform work requests in a timely manner or meeting subcontracting goals. Simply put, the contracting officer found that the project manager "does not have an understanding of the fundamentals of contracts," and we are not persuaded by the protester's arguments that this finding was unreasonable.

The agency, however, did not restrict evaluation of the protester's proposed project manager to the evaluation factor, Key Personnel, which specifically encompassed this area. Rather, the record shows that the perceived deficiencies of the project manager pervaded the entire evaluation by the SEB and the final evaluation by the contracting officer, including all major evaluation factors.[7] Stated differently, and as explained below, the agency, during the evaluations, greatly exaggerated the evaluation factor, Key Personnel, which included evaluation of the project manager, by repeatedly evaluating the substance of this single evaluation factor under all major evaluation factors of the solicitation.

For example, under Technical Plans, the second most important technical factor, the SEB found that the protester's work request system plan [DELETED].[8] However, the SEB found that while the implementation plan was outlined very well, the "current Project Manager [cannot] execute the plans [since he] cannot [even] execute the current antiquated Work Request System." Similarly, the SEB favored the protester's work prioritization methodology but again determined that the project manager would not be able to carry it out. The contracting officer adopted the initial scoring (technical downgrade) of the SEB in her final evaluation and again doubted whether the project manager could successfully implement the system.

As another example, under the technical area of quality control (Technical Plans factor), the SEB found that the protester's quality control plan was [DELETED], but "will not and cannot be executed" by the project manager. The contracting officer, in her final evaluation, did not take any exception to this initial evaluation downgrading the protester's technical proposal, but rather adopted it and found that the protester had not offered sufficient training to "bring the proposed Project Manager up to the government's desired level of performance."

[DELETED]

The record shows that the contracting officer adopted the SEB's evaluation on these matters and rated the protester's proposal as [DELETED] because she determined that the project manager "does not possess the necessary skills to implement the plans outlined in the technical proposal." We therefore find, based on our review of the entire record, that because of the agency's determination that the project manager was unacceptable, it downgraded the protester's proposal in all major evaluation areas. In short, the agency greatly exaggerated the weight the RFP assigned the evaluation factor, Key Personnel, and thereby deviated from the stated evaluation criteria. We therefore sustain the protest.

We understand the agency's strong concern about the project manager proposed by the protester. An agency can reject a proposal where the deficiencies of key personnel render the entire proposal unacceptable, whether or not the rest of the proposal is acceptable. See Sach Sinha & Assocs., Inc., 69 Comp.Gen. 154 (1990), 90-1 CPD Para. 50. Here, however, the agency never determined that the protester's initial proposal or BAFO was unacceptable based on the proposed project manager. Indeed, the agency found the protester's initial proposal [DELETED] and its BAFO [DELETED], even with the proposed project manager. Instead, the agency conducted discussions without advising the protester what it appears to have concluded--that the protester's project manager had to be replaced in order for the protester to be awarded the contract. If the agency did reach that conclusion and had it so advised the protester, the protester could have substituted for its proposed project manager or risked the rejection of its proposal. Instead, the agency greatly exaggerated the importance of one evaluation factor, Key Personnel, and thereby deviated from the stated evaluation criteria.

We recommend that the agency either reevaluate proposals giving each evaluation factor its appropriate weight, or reopen discussions with the protester and RFSS. If discussions are reopened, the protester should be advised that its proposed project manager must be replaced, if that is the agency's position, any other matters of concern should be discussed, and a new round of BAFOs should be evaluated consistent with the solicitation. Following either evaluation, the agency should determine which offer is most advantageous to the government as provided in the RFP. In the event the protester's proposal is determined to be most advantageous to the government, the agency should terminate the contract with RFSS and award the contract to the protester. We also find that the protester is entitled to the cost of filing and pursuing this protest, including reasonable attorneys' fees. 4 C.F.R. Sec. 21.6(d)(1) (1993). In accordance with 4 C.F.R. Sec. 21.6(f), the protester's certified claim for such costs, detailing the time expended and costs incurred, must be submitted directly to the agency within 60 days after receipt of this decision.

The protest is sustained.

* The decision issued March 17, 1994, contained proprietary information and was subject to a General Accounting Office protective order. This version of the decision has been redacted. Deletions in text are indicated by "[DELETED]."

1. The RFP disclosed the point value subtotals for the two broad areas of Technical Requirements and Management Requirements and also disclosed the total possible technical scores available; however, the RFP did not further subdivide the point values. Instead, the RFP contained a narrative of the relative importance of the various factors. Further, the agency's evaluation system was supplemented (as reflected in the source selection plan) by an adjective rating methodology and a risk assessment methodology. The adjective rating methodology consisted of possible ratings of outstanding, good, fair, poor, and unacceptable. Risk factors were evaluated as high, moderate, and low.

2. The SEB, in its evaluation report, stated that the [DELETED]. However, the SEB found "deficiencies [which were] a direct result of [the agency's] current experience with the incumbent Project Manager."

3. The SEB's initial evaluation report shows that the agency, based on its past and current experience with the protester's incumbent project manager, considered the proposed project manager to be totally unacceptable and wholly incapable of performing the work. As explained below, the agency proceeded to downgrade the protester's technical proposal in all major technical areas due to the perceived deficiencies of the proposed project manager.

4. The SEB was unable to reach consensus evaluation ratings for the BAFOs because, as relevant here, there was disagreement among the members as to the adequacy of the protester's project manager. Some members felt that the project manager had improved his performance during the past few months (he had become the project manager about 1 year before the evaluations) and that therefore the protester should receive a higher score. Other members of the SEB continued to believe that the proposed project manager was wholly unacceptable. Conflicting and inconsistent draft reports were prepared. As a result, the contracting officer decided to independently evaluate [DELETED], the deficiencies initially identified by the SEB, the discussion questions, and the technical merits of the BAFOs. In doing so, the contracting officer relied, among other things, upon the initial evaluation report of the SEB (technical scoring and narrative results) concerning the initial proposals as well as portions of the draft BAFO narrative evaluations with which she agreed.

5. The cost/technical trade-off was based on further cost realism adjustments to the protester's BAFO [DELETED]. The protester disputes these upward cost adjustments [DELETED]; however, since we are sustaining the protest on another ground, we do not reach the issue of the propriety of the cost adjustments.

6. The protester does not argue lack of meaningful discussions with respect to the project manager. However, the protester does state that it "was completely unaware that its only real chance to improve its proposal was to offer a new Project Manager," and that the contracting officer's virtually sole reliance on this factor constituted an undisclosed evaluation criterion.

7.[DELETED]

8. The RFP required offerors under this factor to discuss their plan for establishing a work request system for routine and nonroutine maintenance, preventive maintenance, and special tasks.