Consolidated Engineering Services, Inc.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Consolidated Engineering Services, Inc.

File: B-311313

Date: June 10, 2008

DIGEST

1. Protest is sustained where agency's evaluation of offerors' experience relied on factors not identified in the solicitation and on distinctions between offerors' experience that were not supported by the record.

2. Protest is sustained where source selection decision was based on a flawed technical evaluation, and also considered an undisclosed evaluation criterion of transition risk in assuming that any non-incumbent contractor would likely cause mistakes in performance that would result in costs for the agency.

Consolidated Engineering Services, Inc. (CES) protests the award of a contract to Meridian Management Corporation (MMC) under request for proposals (RFP) No. SSA-RFP-08-1004 by the Social Security Administration for facility management services. The protester contends that the agency's evaluation of the offerors' technical proposals was unreasonable, and that the selection decision was flawed.

We sustain the protest.

BACKGROUND

The RFP was issued on June 1, 2007, and sought proposals to provide facilities management services for the HaroldWashingtonSocialSecurityCenter, also known as the Great Lakes Program Service Center (GLPSC), in Chicago, Illinois. Offerors were required to propose the following services: facilities management, operation and maintenance of building and mechanical equipment, elevator maintenance, utilities, custodial and related services, sustaining maintenance, security, protective signaling service, and uninterruptible power supply service maintenance. RFP sect. B. MMC is the incumbent contractor currently providing these services for the GLPSC.

The RFP anticipated the award of a fixed-price contract, with a 1-year base performance period and nine 1-year option periods. The RFP stated that proposals would be evaluated on the basis of the following factors: experience, past performance, and price. RFP sect. H-1. The non-price factors were of equal importance and, combined, were –slightly more important than price.— RFP amend. 1 sect. H-1. The RFP also stated that the agency would evaluate offerors' experience and past performance to develop a –confidence/performance risk— assessment for each proposal. The confidence/performance risk assessment was to be used, along with price, to conduct the selection decision. RFP sect. H-2.2.

As relevant here, the RFP stated that the agency would evaluate offerors' proposals under the experience factor based on the following criteria:

Experience is the opportunity to learn by doing. The Government will evaluate the extent to which an offeror possesses experience in work similar to the Government's requirements under this solicitation in terms of size, scope and complexity, to enable the offeror to:

-identify performance uncertainties and risks;

-identify potential performance problems and their symptoms;

-identify, fashion and select prospective and appropriate solutions;

-implement effective corrective actions;

-develop and implement actions that improve efficiency and/or useful life expectancy of facilities and equipment; and

-achieve the overall objectives of the contract.

RFP sect. H-1(d)(1).

The RFP further stated that proposals would be evaluated based on offerors' experience in the following –critical areas—: custodial services, elevator maintenance, energy management, life safety systems maintenance, mechanical systems operation and maintenance, security services, utilities service management, and working productively with unions. Id.Offerors were advised that the agency would –evaluate the offeror's experience in providing relevant and similar services to other corporations or Government agencies . . . in terms of the size, scope and complexity of the operation,— and that the evaluation –may also include information available from other sources both public and private.— Id.The RFP stated that offerors' proposals would be assigned one of the following ratings under the experience factor: extremely similar, very similar, somewhat similar, slightly similar, or neither similar nor relevant –in terms of size, scope and complexity, to the Government's requirements under this solicitation.— Id.

For the confidence/performance risk assessment, the RFP stated that the agency would consider offerors' past performance and experience to create the composite assessment that –reflect[s] the Government's level of confidence that the offeror, if selected for award, would successfully perform the requirements of this contract.— RFP sect. H-2.2. The RFP advised that offerors' proposals would be assigned one of the following ratings under the confidence/performance risk assessment:

Full Confidence/No Risk -- Essentially no doubt/risk exists that the offeror will successfully perform the required effort.

High Confidence/Little Risk -- Little doubt/risk exists that the offeror will successfully perform the required effort.

Confidence/Some Risk -- Some doubt/risk exists that the offeror will successfully perform the required effort.

Unknown Confidence/ Unknown Risk -- Where no relevant performance record was identified, confidence level/performance risk is unknown.

Little Confidence/Substantial Risk -- Substantial doubt/risk exists that the offeror will successfully perform the required effort.

No Confidence/Extreme Risk -- Extreme doubt/risk exists that the offeror will successfully perform the required effort.

Id.

The agency received proposals from six offerors by the closing date of July 11, 2007, including CES and MMC. A technical evaluation team (TET) evaluated each offeror's technical proposal and prepared a report to the contracting officer (CO), who also served as the source selection authority for the procurement. In testimony provided to our Office, the CO stated that she relied primarily on the TET's analysis for her understanding of the technical aspects of the offerors' proposals. Audio Disc (AD) at 1:11:06.[1]

The agency established a competitive range of the most highly-rated proposals, consisting of CES, MMC, and a third offeror. Agency Report (AR), Tab 6, Competitive Range Determination, at 4-5. The agency then conducted discussions with each offeror in the competitive range, received revised proposals, and conducted new evaluations.

The final ratings for each offerors' proposal were follows:

	CES	MMC
Experience	Extremely Similar	Extremely Similar
Past Performance[2]	Excellent	Excellent
Confidence/ Performance Risk	High Confidence/ Little Risk	Full Confidence/ No Risk
Price	$[deleted]	$46,368,279

AR, Tab 18, Source Selection Decision (SSD), at 4, 6, 7.

In the selection decision, the CO concluded that although CES's and MMC's proposals received similar ratings, MMC's proposal had several advantages that warranted selection for award. The selection decision first discussed each offeror's proposal, and the CO's basis for assigning ratings under each evaluation factor. Although CES and MMC received equal ratings under the experience and past performance evaluation factors, the CO concluded that MMC's proposal merited a confidence/performance risk rating of –full confidence/no risk,— whereas CES merited a rating of –high confidence/little risk.— AR, Tab 18, SSD, at 6-7.

In comparing the offerors, the CO concluded that MMC was superior to CES under the past performance and experience evaluation factors. Under the past performance evaluation, the CO noted that both offerors were rated excellent, and that there were no performance problems reported for either. The CO concluded that –while both firms received equal past performance ratings, there is somewhat less past performance related risk associated with award to Meridian.— Id.

Under the experience evaluation, the CO noted that both offerors were rated as having experience –extremely similar— to the solicitation requirements. Nonetheless, the CO stated that the –extremely similar— ratings represented a range of possible merit for each offeror's experience, and that MMC's experience was superior to CES's experience. The CO first concluded that while CES had experience in managing all of the eight critical areas required under the solicitation, the company had performed them in a number of different multi-service contracts, –none of which required the performance and management of these functions at one site.— Id.By comparison, the selection decision stated that –Meridian has experience performing all these functions on one contract (the GLPSC [incumbent contract]).— Id.On this basis, the CO concluded that MMC's experience was more relevant that CES's experience.

The CO then cited two examples to illustrate her view that MMC's experience was more relevant than CES's experience, explaining that while –Meridian and CES both appear to be excellent contractors with extensive experience . . . [t]he differences are in the details.— Id.First, the CO concluded that MMC's experience in managing centrifugal chillers, a requirement under the mechanical systems operation and maintenance critical area, was more relevant than CES's experience because –CES has nine years of demonstrated experience operating and maintaining 300-ton York centrifugal chillers,— whereas –Meridian has 10 years of demonstrated experience operating the three 533-ton York centrifugal chillers at the GLSPC.— Id.Second, the CO concluded that MMC's experience in managing guard services, a requirement under the security services critical area, was more relevant than CES's experience. In this regard, the CO notes that MMC had provided guard services through its subcontractor under the incumbent contract, whereas CES had provided guard services through a subcontractor on a military base operations contract--experience the agency viewed as somewhat less relevant to the solicitation requirements. Id.

In the final tradeoff between CES's and MMC's proposals, the CO stated that the selection decision would be based on the following consideration: –Given Meridian's clear superiority in experience, the question becomes is this superiority, and associated reduction in risk, worth the slightly over $[deleted] over 10 years that could be saved if [the contract were] awarded to CES.— Id.The CO determined that the lower price of CES's proposal was not the better value to the government because CES did not have experience in performing all of the solicitation requirements under one contract, and that CES lacked familiarity with the GLPSC. Id.The CO also concluded that –[i]t is reasonable to assume that CES will make some mistakes during the learning phase at a new facility,— and that –[s]uch a mistake (i.e., one that damages equipment, injures an employee, or causes a building closure) could easily cost the agency more than $[deleted].— Id.

The agency selected MMC's proposal for award on February 20, 2008, and notified CES of that decision. CES requested a debriefing, which was provided on February 27. This protest followed.

DISCUSSION

CES challenges the reasonableness of the agency's evaluation of the offerors' experience and the selection decision. The evaluation of an offeror's proposal, including experience, is a matter within the agency's discretion. IPlus, Inc., B-298020, B-298020.2, June 5, 2006, 2006 CPD para. 90 at 7, 13. Although we will not substitute our judgment for that of the agency, we will question the agency's conclusions where they are inconsistent with the solicitation criteria, undocumented, or not reasonably based. Sonetronics, Inc., B-289459.2, Mar. 18, 2002, 2002 CPD para. 48 at 3. On the other hand, a protester's mere disagreement with the agency's judgment in its determination of the relative merit of competing proposals does not establish that the evaluation was unreasonable. C. Lawrence Constr. Co., Inc., B-287066, Mar. 30, 2001, 2001 CPD para. 70 at 4.

Although the protester raises several allegations regarding the agency's evaluation of the offerors' proposals and assignment of adjectival ratings, our decision here primarily addresses the agency's direct comparison of the offerors' proposals.[3] As discussed below, we agree with the protester that the agency's comparison of the offerors' proposals and the selection decision were unreasonable.

Comparison of Offerors' Experience

The protester argues that the agency's evaluation of the offerors' experience was unreasonable and not supported by the record. As discussed above, the agency rated both offerors under the experience factor as having experience that is –extremely similar— to the requirements of the solicitation, the highest possible rating. The agency concluded, however, that MMC's experience was more relevant than CES's based on a general comparison of the relevance of the offerors' experience, and on two specific examples which the agency concluded illustrated differences between the offerors. We discuss first the agency's general evaluation of the offerors' relative experience, and then address the two specific examples. As discussed below, we conclude that the agency's evaluation of the offerors' relative experience was not reasonable.

1. Overall Relevance of Experience

The selection decision first emphasized that CES's experience was less relevant than MMC's because the protester had not performed all of the eight –critical areas— listed in the experience factor under one contract. The protester contends that the agency's reliance on this distinction was unreasonable. We agree.

In the comparison of the overall experience records of CES and MMC, the CO explains twice that she found a significant distinction based on whether the offerors had performed all of the eight –critical areas— under one contract:

Unlike CES, Meridian has experience performing all eight critical areas combined under one contract, [the incumbent] GLPSC combined facilities management contract. . . .

However, while Meridian has experience performing all these functions under one contract (the [incumbent contract]), CES' experience is spread over a number of contracts, none of which required the performance and management of all of these functions at one site.

AR, Tab 18, SSD, at 12.

During the hearing, counsel for the agency asked the CO to explain –what the government was looking for in the overall solicitation.— AD at 1:57:45. The CO stated the agency was seeking to identify offerors that had –experience in providing all services under all the categories under one contract.— Id. at 1:57:55. The CO further stated that in order to be considered equivalent in terms of experience to the incumbent, an offeror would need to have performed at least one contract involving every service sought under the solicitation. Id. at 2:02:45, 2:04:50.

Agencies are required to evaluate proposals based solely on the factors identified in the solicitation, and must adequately document the bases for their evaluation conclusions. Intercon Assocs., Inc., B-298282, B-298282.2, Aug. 10, 2006, 2006 CPD para. 121 at 5. While agencies properly may apply evaluation considerations that are not expressly outlined in the RFP where those considerations are reasonably and logically encompassed within the stated evaluation criteria, there must be a clear nexus between the stated criteria and the unstated consideration. Global Analytic Info. Tech. Servs., Inc., B-298840.2, Feb. 6, 2007, 2007 CPD para. 57 at 4.

The solicitation stated that offerors experience would be evaluated as follows: –Experience will be evaluated in terms of the size, scope and complexity of the operation. Experience in the following [eight] critical areas, listed in alphabetical order, will be evaluated.— RFP sect. H-1(d)(1). We agree with the agency that the solicitation anticipated award of a single contract, and that the ability to perform all of the services under a single contract is a basis for evaluation. We do not agree, however, that the CO's –one contract— criterion for evaluating the relevance of offerors' experience was a reasonable proxy for that evaluation. Instead, we think that the RFP does not support the agency's –one contract— interpretation for evaluating experience, and that the interpretation was not reasonably related to the stated evaluation criteria.

The RFP does not state that the agency was seeking proposals from offerors who had performed all of the eight critical areas of experience under –one contract—; instead, the solicitation states that the agency will evaluate offerors' experience to determine whether their experience with the eight critical areas is similar to the requirements of the solicitation. The CO's –one contract— approach meant that the agency did not consider any offeror to have equivalent experience to the incumbent contractor unless the offeror had performed the exact same services required under the solicitation in the same manner as the incumbent, i.e., under one contract.

In addition to being inconsistent with the RFP, the CO's interpretation appears to have unduly restricted competition by unreasonably disfavoring non-incumbent firms competing with the incumbent. Specifically, the agency's action prejudiced CES by precluding consideration of whether offerors' references demonstrated, individually or collectively, experience that was similar in size, scope or complexity to the solicitation requirements.[4] In this regard, the CO simply assumed, without meaningful analysis, that CES's collective experience, which the record shows clearly included multi-function contracts involving all of the services required under the solicitation, could not have been the equal to MMC's under the incumbent contract.[5] On this record, we conclude that the agency's evaluation was not reasonably consistent with the stated evaluation factors.

2. Centrifugal Chillers

In the first of the two specific examples of areas where the CO concluded that MMC's experience was more relevant than CES's, the selection decision states that MMC had more relevant experience managing centrifugal chillers. The selection decision states that the offerors' experience was distinguishable because CES has 9 years of experience with 300-ton York chillers, whereas MMC has 10 years of experience with –the three 533-ton York centrifugal chillers at the GLPSC.— AR, Tab 18, SSD, at 12. The protester contends that the agency's distinctions between the offerors' experience here was unreasonable. We agree.

During her testimony, the CO stated that although CES was given credit for having experience in centrifugal chiller maintenance, she distinguished between the experience of CES and MMC because she concluded that CES did not have experience with maintenance of the same type or models of chillers as those at the GLPSC. AD at 1:13:50. The CO stated that she believed that there are differences between various models of centrifugal chillers, and that CES's experience was less relevant than MMC's experience managing the 533-ton York centrifugal chillers at the GLPSC. AD at 1:16:24, 1:17:05. The CO stated, however, that she performed only a –cursory— review of the offerors' proposals, and –mostly relied on the technical evaluation team— for analysis of the technical merits of the proposals. AD at 1:11:06. Further, the CO stated that she does not understand what a centrifugal chiller is, but instead relied on the TEP for all of her understanding of the technical issues regarding chillers. AD at 1:35:05. Thus, the CO explained that any conclusions she made regarding differences between the experience of CES and MMC in maintaining centrifugal chillers was based on information she received from the TET. AD at 1:18:10.

The TET report, however, does not discuss differences between the offerors' experience, and does not support the conclusions made by the CO in the selection decision.[6] The TET report stated that CES has –performed maintenance on Trane and York chilled water equipment which is very similar to the chilled water equipment at the [GLPSC].— AR, Tab 5, TET Report, at 20. The TET report does not identify which models of centrifugal chillers with which CES or MMC has experience, nor does the report draw any distinctions between the offerors' experience with centrifugal chillers or discuss why their experience is different.

Additionally, the record shows that the CO did not have a clear understanding of which types of chillers CES had experience maintaining. The CO stated, for example, that her reference in the selection decision to CES's experience with York chillers was an error, because she believed that the CES proposal showed experience with Trane chillers. AD at 1:13:40, 1:43:45. In fact, CES's proposal states that the company has experience maintaining –[deleted] chiller equipment . . . including [deleted] Trane, York[deleted].— AR, Tab 4, CES Proposal, Executive Summary, at 6. CES's proposal lists experience with [deleted] Trane centrifugal chillers, [deleted] York centrifugal chillers, [deleted] Trane CentraVac centrifugal chillers, [deleted] Trane CentraVac centrifugal chillers, and [deleted] Trane centrifugal chillers. Id. at 9-15.

On this record, we conclude that the evaluation of the offerors' proposals regarding their experience with centrifugal chillers was not reasonable. As the CO acknowledges, she does not understand whether the experience each offeror had with the chillers was different in any meaningful way, and the report provided to her by the TET does not discuss or otherwise provide any basis to distinguish between the offerors' experience. Moreover, the selection decision's assessment of MCC's experience again appears to reflect an unsupported, and anti-competitive, preference for the incumbent.

3. Security Services

The CO also concluded that MMC's experience in managing security services was more relevant than CES's experience. The CO noted that both offerors proposed to meet the security requirements under the contract through subcontractors, and that each offeror had previously performed contracts where they had managed security services subcontractors. The CO concluded, however, the MMC's experience was more relevant because –Meridian and [its] proposed subcontract[or] have demonstrated experience providing Guard Services at the GLPSC for the last ten years.— AR, Tab 18, SSD, at 12. The protester contends that the CO's evaluation was unreasonable because both offerors had proposed to perform the security requirements through subcontractors with similar experience records. We disagree.

The record shows that the CO rated the experience of both offerors' proposed subcontractors as extremely similar to the security requirements in the solicitation. AR, Tab 5, TET Report, at 6, 20. The CO, however, distinguished between the experience of the CES and MMC in their management of security subcontractors. AR, Tab 5, TET Report, at 6, 20; Tab 18, SSD, at 6, 12. In this regard, the CO determined that MMC had experience managing a subcontractor at the GLPSC on the incumbent contract, and therefore had extremely similar experience. AR, Tab 5, TET Report, at 6; Tab 18, SSD, at 12. In contrast, CES had experience managing a subcontractor at a military installation, which the CO concluded was –not entirely similar to managing armed guard services in a multi-story office building setting like that of the GLPSC.— AR, Tab 18, SSD, at 6. Specifically, the CO concluded that CES had not adequately explained in its proposal why the requirements of the security subcontract it managed were similar to the requirements of the solicitation. AD at 1:48:35. Thus, in contrast with the unsupported preference for the incumbent's experience discussed above, we think the CO reasonably distinguished the nature of the protester's prior work from the work required under the solicitation. We therefore find no basis to sustain this protest ground.

In sum, we conclude the agency's evaluation of the offerors' relative merits under the experience factor was flawed, and thus the agency's conclusion that MMC's experience was more relevant than CES's experience lacked a reasonable basis. We further conclude that CES was prejudiced by this flawed evaluation because the agency relied upon it in selecting MMC's higher-priced proposal for award, and sustain the protest on this basis. See McDonald Bradley, B-270126, Feb. 8, 1996, 96-1 CPD para. 54 at 3; Statistica, Inc. v. Christopher, 102 F.3d 1577, 1581 (Fed. Cir. 1996).

Source Selection Tradeoff Analysis

The protester argues that the agency's selection decision was flawed for two reasons: (1) the tradeoff analysis relied on the flawed evaluation of the offerors' experience, and (2) the tradeoff analysis concluded that non-incumbent contractors generally pose risks in their transition to contract performance, and that those concerns warranted discounting CES's lower proposed price--notwithstanding the fact that the solicitation did not call for evaluation of offerors' approaches to transition. We agree with the protest on both issues.

First, the tradeoff analysis concluded that MMC's experience was –clearly superior— to CES's experience, and that this difference required the CO to address whether MMC's higher-priced proposal merited selection as opposed to CES's lower-priced proposal. Because, as discussed above, the agency's underlying evaluation of the offerors' experience was unreasonable, we conclude that CES's was prejudiced by the agency's reliance on that evaluation in the selection decision.

Next, the agency's tradeoff determination relied on assumptions concerning the risk posed by award of a contract to a non-incumbent offeror that were unreasonable and inconsistent with the solicitation. As explained above, the CO determined that although CES's proposed price was $[deleted] lower than MMC's, this difference was –more than offset[]— by MMC's lower performance risk based on that offeror's successful performance of the incumbent contract. AR, Tab 18, SSD, at 12. The CO also concluded that the $[deleted] price differential was not significant because of the possibility that a non-incumbent contractor, such as CES, could experience problems during contract transition that would eliminate the differential: –It is reasonable to assume that CES will make some mistakes during the learning phase at a new facility,— and that –[s]uch a mistake (i.e., one that damages equipment, injures an employee, or causes a building closure) could easily cost the agency more than $[deleted].— Id.

The CO acknowledged in her testimony that her concern regarding the transition risk was based on general assumptions regarding non-incumbents, rather than any aspect of CES's proposal. AD at 1:34:05. Specifically, the CO cited two factors for her assumptions regarding transition risk: (1) what the CO termed –general knowledge— concerning the difficulties that a non-incumbent might have in contract transition, and (2) the CO's experience with a contract in Birmingham, Alabama where a non-incumbent, lower-priced offeror was selected for contract award, but did not perform successfully. AD at 1:31:15, 2:05:30. We do not think that either of these rationales provides a reasonable basis to conclude that CES's proposal posed a $[deleted] transition risk. Neither factor relates to the risk posed by CES's proposal, and instead is based on a general assumption that a non-incumbent will likely experience costly problems during transition.

Additionally, the solicitation did not require offerors to identify their approaches to contract transition in their proposals, nor did the agency state that it would take transition costs or risk into account in evaluating offerors' proposals.[7] We think that the agency's use of assumptions concerning transition risk here constituted an improper use of an undisclosed and anti-competitive evaluation criterion that was not related to any risk or price evaluation identified in the solicitation. See Global Analytic Info. Tech. Servs. Inc., supra. The undisclosed evaluation factor of transition risk and associated cost clearly prejudiced CES because it effectively negated the price advantage CES's proposal had as compared to MMC.

On this record we conclude that the agency's selection decision was not reasonable, and that CES was prejudiced by the agency's reliance on the flawed evaluation of offerors' experience and the agency's unreasonable assumptions concerning transition risk. We therefore sustain the protest on these bases.

RECOMMENDATION

We recommend that the agency reevaluate the offerors' proposals, consistent with this decision. If the agency concludes that its needs require a consideration of transition risk, it should issue a revised solicitation and obtain new proposals from the offerors. At the conclusion of this process, we recommend that the agency make a new selection decision. If MMC is not found to offer the best value to the government, the agency should terminate MMC's contract for the convenience of the government.

We also recommend that CES be reimbursed the costs of filing and pursuing this protest, including reasonable attorney fees. Bid Protest Regulations, 4 C.F.R. sect. 21.8(d)(1) (2008). CES should submit its certified claim for costs, detailing the time expended and cost incurred, directly to the contracting agency within 60 days after receipt of this decision. 4 C.F.R. sect. 21.8(f)(1).

The protest is sustained.[8]

Gary L. Kepplinger
General Counsel