Matter of: SDA Inc. File: B-248528.2 Date: April 14, 1993

Matter of: SDA Inc. File: B-248528.2 Date: April 14, 1993

PROCUREMENT Competitive Negotiation Offers Evaluation errors Evaluation criteria Application Agency's undisclosed source-selection plan and resulting proposal evaluation were unreasonable where offerors' experience, a critical evaluation criterion, was evaluated in an arbitrary and unsupportable manner which effectively ignored the evidence in proposals of the actual extent of the competing offerors' relevant experience. PROCUREMENT Competitive Negotiation Contract awards Administrative discretion Cost/technical tradeoffs Technical superiority Cost/technical tradeoff is unreasonable where it fails to set forth a reasonable basis for selecting a higher cost proposal and offers no explanation of what benefit the agency can realize from the fact that even though a higher percentage of the awardee's prior projects were of a size and type similar to the current procurement, the protester offered more experience, in absolute terms, in such projects.

Attorneys

DECISION SDA Inc. protests the award of a contract to Dominion Leasing, Inc. under solicitation for offers (SFO) No. SNM92384, issued by the General Services Administration (GSA) for the lease of approximately 55,000 square feet of office and warehouse space in Albuquerque, New Mexico, for use by the Internal Revenue Service. SDA contends that the proposals were evaluated in a manner inconsistent with the SFO's evaluation criteria and that the award decision was based on an unreasonable cost/technical tradeoff.

We sustain the protest.

Originally issued on November 30, 1990, and amended several times in the ensuing 2 years, the SFO anticipates a 20-year lease, with the government entitled to terminate after 10 years. The SFO states that award will be made on the basis of the offer "most advantageous to the government," with price less important than the combination of other factors. Those other factors are identified in the SFO, in descending order of importance, as: (1) design criteria; (2) experience of the offeror, the general contractor, and the management company; (3) public transportation and parking facilities; and (4) child care and fitness facilities. The SFO explains that the design criteria factor is "significantly more important" than the others. The SFO states that price evaluation will be based on the "net present value" of the annual rent per square foot, and explains how that figure will be calculated for price evaluation purposes.

The agency's source-selection plan, which was not disclosed to the offerors, contains rating sheets that detail how the various SFO evaluation criteria are to be applied. The rating sheets establish 100 points as the maximum number of technical points, allotting 50 points as the maximum for design criteria, 30 points for experience, 15 points for public transportation and parking facilities, and 5 points for child care and fitness facilities.[1]

Offers were received from five firms, of which four remained in the competitive range until the source selection. For reasons not relevant here, the agency conducted multiple rounds of discussions and afforded the offerors several opportunities to revise their proposals, including three rounds of best and final offers (BAFOs). As a result, the source selection evaluation board (SSEB) conducted five rounds of evaluations.

In the final analysis, the agency conducted a cost/technical tradeoff between SDA's and Dominion's proposals, because SDA's proposal was evaluated as lower in cost, but also lower in technical rating than Dominion's. The evaluated annual net present value price for SDA's proposal was $9.64 per square foot, while Dominion's was $9.95. The difference, in net present value terms, meant that, over the 20-year lease term, Dominion's proposal was evaluated as costing the government approximately $290,000 more than SDA's.

On the technical side, the agency found SDA's and Dominion's proposals essentially equal in most areas. The two proposals received identical scores (18 out of a possible 20 points), and were considered substantially equal, under the two least important factors, public transportation and parking and availability of child care and fitness facilities. Under the most important technical factor, design criteria, SDA's proposal received a higher point score (32 vs. 27 out of 50); nonetheless, the agency determined that the two proposals were essentially equal in that area.

Only under the factor of experience--and then only under two subfactors within that factor, the offeror's experience and the management company's experience--did the agency assign more points to Dominion's proposal than to SDA's. SDA's proposal received zero points for the offeror's own experience and three points for the proposed management company's experience, while Dominion's proposal received seven and eight points, respectively, for those two subfactors.[2] That difference led to Dominion's proposal receiving a higher overall technical score than SDA's (69 vs. 62 points). It was solely on the basis of Dominion's perceived superiority in those two subfactors that the agency concluded that Dominion's proposal was worth the associated price premium. Accordingly, we now turn to examine the evaluation of those two subfactors in detail.

The SFO provides no explanation of how the experience factor will be evaluated. The sole references to it in the solicitation documents (other than the above-cited listing among the evaluation criteria) are the following two items contained in a "checklist of documents required to constitute a complete offer," which was distributed to potential offerors:

"List of all buildings, or tenant improvements in existing buildings, of similar size and design constructed in the last 15 years by offeror and general contractor and references, including name and phone number of individual qualified to assess performance."

"Name and address of proposed operations and maintenance firm, including [a list] of buildings of similar size and design which the firm has operated and maintained during the past 15 years. Also provide a list, including telephone number and address, [of building] managers, tenants, property managers, service companies, mechanical contractors, custodial contractors who can testify as to the quality of service previously provided by offeror and primary subcontractor."

The agency's internal rating sheets state that it is "the Government's desire to contract with an individual or firm who has demonstrated the ability to successfully construct and operate facilities of this size and type." The rating sheets break down the 30 points within the experience factor equally among the three subfactors: 10 points for the offeror's experience, 10 points for the construction contractor's experience, and 10 points for the management company's experience. The internal rating sheets detail the full breakdown of evaluation criteria and associated points for the offeror's experience and the maintenance company's experience as follows:[3]

OFFEROR Maximum Score 10

a. Number of Years (1) 1-5 0 points (2) 6-10 2 points (3) 11 or more 5 points

b. Size of majority of successfully completed projects (1) Less than 25,000 sq. ft. 0 points (2) 25,000-50,000 sq. ft. 2 points (3) Over 50,000 sq. ft. 3 points

c. Type of majority of successfully completed projects (1) Residential or no prior experience 0 points (2) Office/computer/light industrial 2 points

MANAGEMENT COMPANY Maximum Score 10

a. Number of Years (1) 1-5 0 points (2) 6-10 2 points (3) 11 or more 4 points

b. Size of majority of satisfactorily managed projects (1) Less than 25,000 sq. ft. 0 points (2) 25,000-40,000 sq. ft. 1 point (3) Over 40,000 sq. ft. 3 points

c. Type of majority of satisfactorily managed projects (1) Light industrial only 1 point (2) Office only 2 points (3) Office and computer 3 points

The evaluation of proposals thus required the evaluators to determine, for the offeror and its proposed management company, the size of the majority of successfully completed (or satisfactorily managed) projects.[4] The evaluators also had to determine the type of the successfully completed (or satisfactorily managed) projects for each entity. Since the size and type ratings were based on whether a majority of the entity's projects fell into one or another category, determining the "universe" of relevant projects was critical. For example, seven large office/computer projects would lead to maximum scores if they were counted as a majority of 11 projects overall; but they would merit zero scores if counted as a minority of 15 projects. SDA and Dominion did not disclose, either in their proposals or during discussions, how the offerors had determined the universe of their reported prior projects--i.e., whether they had listed all prior projects, or only those projects deemed of similar size and design to the current procurement.

As part of the information concerning offeror experience, SDA submitted a detailed description of eight offices that the offeror stated evidenced its "prior history in constructing this type of building [referring to `the proposed new building in Albuquerque']." Of those projects, all are office projects; seven of the eight cover more than 50,000 square feet, while one (presumably deemed relevant despite its smaller size because it is a GSA facility built for the Internal Revenue Service) involves 21,000 square feet.

Similarly, SDA provided a description of three government office buildings that its management company was then managing. This document also mentions that the management company manages 600 residential apartment units in Colorado. All three office buildings are owned by GSA and used by IRS; all presumably contain computer facilities as well as offices; and each exceeds 100,000 square feet. In addition, SDA submitted a separate list of 19 projects that SDA's management company had managed; of those, 9 are identified as office developments (of which 7 were larger than 50,000 square feet), with the remaining 10 equally divided between retail and apartment developments.

Included with SDA's proposal was also what was plainly a preexisting brochure identifying a total of 48 prior projects: 12 office projects, 8 retail sites, 2 industrial developments, 6 apartment complexes, and 20 land developments. That list does not identify the square footage or provide any other detailed information about the projects.

Concerning the size of the offeror's prior projects, the evaluators relied on SDA's preprinted literature as a basis to find that SDA had identified 50 projects, of which the agency found that 20 involved land development only, so that they were excluded from consideration.[5] This left a universe of 30 projects, so that 16 projects would constitute a majority of that universe. Of the 17 SDA projects for which the agency had square footage information, 13 exceeded 50,000 square feet. By assuming that all projects for which it lacked square footage involved less than 25,000 square feet, the SSEB, in its final report, concluded that "the majority of [SDA's] projects, based on the brochure provided by the offeror and excluding land only development . . . were less than 25,000 square feet."[6] Accordingly, the evaluators assigned SDA's proposal a score of zero for the size of SDA's prior projects.

The evaluators used the same universe of 30 projects to determine the type of the majority of SDA's prior projects. The final SSEB report states that the majority of SDA's projects were "residential and retail." The evaluators' score sheets, however, make clear that the agency viewed SDA's 30 non-land-development projects as consisting of 14 office projects, 8 retail sites, 2 industrial developments, and 6 apartment complexes. Of those, only 14 were residential and retail (the apartments and the retail sites); the majority (16) were office or industrial and, as explained above, the maximum score was to be assigned where the majority of offeror's experience was with office/computer/light industrial projects. There is no further explanation for the agency's determination to assign SDA's proposal a zero score for the offeror's experience, which, according to the rating sheets, meant that the majority of SDA's prior projects were "residential [or] no prior experience."

Concerning the size of the management company's prior projects, the evaluators assigned SDA's proposal the maximum number of points (three). However, the evaluators concluded that the majority of the management company's projects were "retail and apartment," and it assigned SDA a score of zero for the type of the management company's prior projects. The SSEB report explains that this conclusion was based on a count of 9 office projects and 10 retail and apartment sites, with the rating based on the fact that 9 is less than a majority of 19.

The SSEB report does not explain how the evaluators decided to assign SDA zero points for this subfactor, where the rating scheme shows one point as the minimum available. In addition, the SSEB report does not indicate how the evaluators translated SDA's management company's experience in managing apartments and retail sites into the rating scheme's limited matrix, which provided that one point was to be assigned for "light industrial only," two points for "office only," and three points for "office and computer."

The agency assigned maximum scores to Dominion's proposal for the size and type of the offeror's and the management company's prior projects. By the agency's count, Dominion had identified 7 relevant projects--the same number as SDA--but Dominion received the highest score (3 points) for size and the highest score (2 points) for the type of prior projects, because the total number of projects the agency counted for Dominion was 10, so that the 7 relevant projects constituted a majority.[7] As explained above, SDA received zero points for both criteria, because the 7 large projects that SDA had identified as relevant were counted as less than a majority of the 30 projects the evaluators found listed in SDA's brochure.

Concerning the management company's experience, by the agency's count, Dominion had identified fewer projects (6) as office/computer projects than SDA, but Dominion received the highest scores both for size and for type (3 points each), because Dominion had identified only 11 projects overall for its management company. Dominion's six office/computer sites thus represented a majority, albeit by half a project. Although SDA had identified greater experience for its management company in managing projects of the size and type GSA found relevant (9 projects as opposed to Dominion's 6), SDA's proposal received a zero for the type of prior project, because the agency concluded that SDA's greater experience constituted less than a majority of the 19 projects that the agency treated as the relevant universe for SDA.

SDA contends that, for the experience evaluation subfactor, GSA's source selection plan added evaluation criteria which were not disclosed in the SFO, and which are inconsistent with the SFO. Further, SDA alleges that the zero scores assigned to SDA's proposal for its experience and for that of its management company are unreasonable. In particular, SDA complains that the agency's methodology was irrational, because GSA simply counted the number of prior projects listed in offerors' proposals and based the scores on the percentage of those projects which are large office developments. SDA also challenges the reasonableness of the cost/technical tradeoff that supported the selection of Dominion. Finally, SDA alleges that the agency displayed bias against the company.

In our review of an agency's evaluation of proposals, we confine our analysis to a determination of whether the evaluation was reasonable and consistent with the solicitation evaluation criteria. SeaSpace, 70 Comp. Gen. 268 (1991), 91-1 CPD Para. 179. Similarly, where an agency chooses between a higher cost, higher rated proposal and a lower cost, lower rated one, our review is limited to a determination of whether the cost/technical tradeoff that the agency performed was reasonable and consistent with the solicitation evaluation criteria. Central Tex. College, 71 Comp.Gen. 164 (1992), 92-1 CPD Para. 121.

Here, we find that the agency's evaluation was unreasonable, both in terms of the mechanical formula used in the source selection plan and in the way the proposals were actually evaluated. The agency's source selection plan's rating scheme for evaluating experience was arbitrary, and the resulting evaluations bore no meaningful relation to offerors' actual experience. The undisclosed evaluation scheme would assign zero scores for experience to a proposal demonstrating recent experience with dozens of projects of size and design similar to this procurement, if the proposal also mentioned larger numbers of projects of different sizes and design; while a proposal citing nothing but two large office/computer projects completed 15 years ago would receive the maximum ratings.

In effect, GSA was actually measuring how few irrelevant projects the offerors mentioned in their proposals: proposals mentioning the smallest proportion of irrelevant projects were rated as having the most experience. Such an evaluation scheme has no rational basis.[8]

The unreasonableness of the evaluation was exacerbated by the way the source selection plan was implemented. The agency simply ignored SDA's identification of eight projects as most relevant to the current procurement--thus plainly indicating SDA's judgment that these eight were similar in size and design to the current procurement--and instead based its evaluation of SDA's proposal on the preprinted literature referring to other prior SDA projects. The agency's action in this regard plainly prejudiced SDA. If the agency had limited its evaluation of SDA's experience to the eight identified projects, and the evaluation of SDA's management company's experience to the management projects that SDA indicated were relevant, SDA's proposal's score would have been within four points of Dominion's for experience, and higher than Dominion's for the overall technical evaluation.[9]

SDA thus received a lower score only because its preprinted literature mentioned many other projects and the agency unreasonably took that larger number as the relevant universe of projects; within that inflated universe, SDA was found to have one or two projects fewer than a majority of the preferred size and type. Dominion received a higher score because it had one project (or even one-half of a project) more than a majority falling into the agency's definition of the preferred size and type.

We conclude that the agency's methodology did not represent a rational approach to measuring offerors' experience. Even within the context of that methodology, the agency reached conclusions which are inconsistent with the rating scheme or which do not accurately reflect the experience- related information presented in the proposals.[10]

The agency's evaluation under the experience criterion was apparently based solely on the point scores.[11] Although there is narrative in the SSEB report, that narrative only explains how the point scores were assigned. The record contains no indication that the agency ever considered whether Dominion actually possesses superior or greater experience than SDA. Indeed, on this record, the agency could hardly have considered the reality behind the point scores and still have awarded a contract to Dominion, since the respective proposals unambiguously indicated that SDA possesses more experience than Dominion in large office projects, the kind favored under the agency's evaluation plan.

In sum, there was no rational basis for GSA to determine that SDA's proposal indicated less relevant experience than Dominion's, and there was no rational basis for GSA to downgrade SDA's proposal for including literature indicating the breadth of the offeror's experience. GSA's evaluation of SDA's proposal was therefore not reasonable.[12]

Moreover, because the cost/technical tradeoff was grounded solely on the point scores, we find that the tradeoff lacked a reasonable basis. Other than a generic statement that Dominion's "technical superiority . . . represents the greatest value to the Government," the agency offers no basis to support a determination that this slight difference in the distribution of the two offerors' experience justified paying the price premium associated with Dominion's proposal. In light of the way in which the agency reached its conclusion that Dominion's experience was superior, we find that, on the current record, there is no reasonable basis to conclude that Dominion's experience justified paying the associated higher price.

Accordingly, we sustain the protest. Normally, we would recommend that the agency reevaluate the proposals in a way both reasonable and consistent with the SFO, with a view to possible termination for convenience of Dominion's contract, depending on the outcome of that reevaluation. That remedy is not feasible here, because the lease does not contain a termination for convenience clause. In such circumstances, we will not recommend termination of an awarded contract, even if we sustain the protest and find the contract award improper. Peter N.G. Schwartz Cos. Judiciary Square Ltd. Partnership, B-239007.3, Oct. 31, 1990, 90-2 CPD Para. 353.[13]

Since there is no basis for termination of the lease, we find that SDA's relief is limited to recovery of its proposal preparation costs and the reasonable costs of pursuing its protest, including attorneys' fees. 4 C.F.R. Sec. 21.6(d) (1992). SDA should submit its certified claim directly to GSA within 60 working days of receipt of this decision. 4 C.F.R. Sec. 21.6(f)(1).

The protest is sustained in part and denied in part.

1. The 50/30/15/5 distribution of points is consistent with the SFO's weighting of the various technical factors. Until late in the proposal evaluation process, the agency was multiplying the "raw" scores resulting from that distribution by the same weight distribution stated as decimal fractions (i.e., 0.50, 0.30, 0.15, 0.05), which had the effect of substantially magnifying the difference in weights assigned to the various factors. When the agency realized that the "raw" scores already incorporated the SFO's weighting and that multiplying those scores by the listed decimal fractions essentially constituted "double-weighting," it stopped doing so--clearly a reasonable decision. Our analysis refers solely to the raw scores, not to the double-weighted scores.

2. Both proposals received nine points for their proposed construction contractors' experience.

3. We have not reproduced the point breakdown for the evaluation of the construction contractor's experience because that subfactor is not at issue in the protest.

4. The agency's evaluation of the length of each entity's experience is a collateral issue in the protest, which we address briefly below. In addition, we do not address the question of whether the offerors had successfully completed (or satisfactorily managed) prior projects, because the evaluators appear not to have eliminated prior projects from consideration on that basis.

5. The agency reached the total of 50 projects by adding to the 48 projects in the brochure 2 office projects which were listed in the detailed descriptions but not in the brochure.

6. Actually, in the immediately preceding round of offers, the consensus evaluation had identified the size of SDA's prior projects as a "major strength," and had indicated that the majority of those projects were larger than 50,000 square feet. The maximum number of points assigned as a result (three) was later crossed off and replaced with a zero, with a date indicating that the change was made during the final BAFO evaluation. SDA's final BAFO was no different in this area from the company's prior offer.

7. Dominion did not initially identify the square footage of its prior projects, but the agency requested that information during the written discussions. The agency did not ask SDA to provide comparable information regarding SDA's projects for which the company's brochure did not disclose square footage.

8. Although an agency might wish to evaluate offerors' understanding or ability to discern the importance of experience with projects of size and design similar to the current procurement, such a criterion was plainly not present here. First, the SFO identified the evaluation criterion as the offerors' experience, not their ability to identify projects of similar size and design. Second, SDA did, in fact, demonstrate its ability to identify such projects by describing those projects separately and in considerable detail--but GSA chose to ignore that part of SDA's proposal. Third, on the current record, GSA had no way of knowing whether Dominion's more limited universe of listed projects resulted from that company's superior ability to identify projects of the relevant size or design--or whether the company simply had more limited experience than SDA.

9. That is, it would have gained three points for the size and two points for the type of the offeror's prior projects, and three points for the type of the management company's prior projects. The total of 8 additional points would have led to SDA's proposal being assigned 70 points, as opposed to the 69 points that Dominion's proposal received.

10. For example, there does not appear to have been any basis for the agency to assign SDA zero points for the type of projects its management company had managed, where the source selection plan's provided for a minimum of one point in that area. In addition, the agency appears to have arbitrarily assigned points for the number of years its management company had been in existence, the most heavily weighted element within the experience factor. SDA's proposal stated that its management company had been in existence earlier than 1988, albeit under a different name, but the agency ignored that statement. Furthermore, even if we assume, arguendo, that the agency acted reasonably in basing its evaluation on the preprinted list of projects mentioned in SDA's proposal, rather than the list of projects relevant to the instant procurement, the determination that the majority of SDA's prior projects involved "residential or no prior experience" was still unreasonable and inconsistent with the record before the agency, since only 14 of the 30 projects the agency considered were residential. Moreover, nothing in SDA's proposal supported the SSEB's determination that a majority of SDA's prior projects involved less than 25,000 square feet. Finally, we note that, while neither offeror's proposal disclosed the square footage of all prior projects (since SDA identified square footage only for projects of similar size or design), the agency brought the matter to Dominion's attention and provided that company an opportunity to furnish the missing information. With SDA, however, the evaluators did not raise the matter, but instead drew a negative inference from SDA's failure to provide the information and assumed that every one of SDA's projects for which no square footage had been identified involved less than 25,000 square feet. The two offerors were thus not treated equally.

11. With respect to the area of design criteria, the agency articulated a plausible basis for its conclusion that SDA's and Dominion's proposals were essentially equal, notwithstanding SDA's significant advantage in terms of the point scores alone. Such a determination is permissible, since point scores are intended only as guides to intelligent decision- making and are not binding on agencies. See Bunker Ramo Corp., 56 Comp. Gen. 712 (1977), 77-1 CPD Para. 427.

12. SDA alleges that the improper evaluation of its proposal resulted from bias on the part of the evaluators. We find no merit to this ground of protest. Government officials are presumed to act in good faith and, therefore, for our Office to conclude that bias existed, the record must establish that contracting officials intended to injure the protester. Jaycor, B-240029.2 et al., Oct. 31, 1990, 90-2 CPD Para. 354. Our review of the record does not indicate that such convincing evidence exists here.

13. SDA attempts to distinguish the instant protest on the theory that Dominion's contract was void because Dominion's construction contractor failed to satisfy the New Mexico licensing requirements at the time of award. We deny this protest ground, both because, in fact, Dominion's construction contractor obtained the state license prior to award, and because the lack of a license is not a bar to award, absent a specific solicitation provision to the contrary, which is not present here. Mid-Am. Mgmt. Servs., Inc., B-244103, June 5, 1991, 91-1 CPD Para. 537.