Helicopter Transport Services LLC

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Helicopter Transport Services LLC

File: B-400295; B-400295.2

Date: September 29, 2008

DIGEST

1. Protest that agency improperly evaluated proposals under one of four technical factors on a pass/fail basis is sustained where the solicitation, by stating that the four technical factors were listed in descending order of importance and calling for a price/technical tradeoff, contemplated that each technical factor would be evaluated qualitatively.

2. Protest that agency improperly evaluated protester's past performance is sustained where the record contains no contemporaneous documentation of the oral discussion that the agency states provided the basis for the evaluation, and the evaluators relied on unidentified written documents.

Helicopter Transport Services LLC (HTS) protests the award of contracts to Erickson Air-Crane, Inc. and Columbia Helicopters, Inc. under request for proposals (RFP) No. AG-024B-S-08-9003, issued by the U.S. Forest Service, NationalInteragencyFireCenter, Boise, Idaho, for fire support helicopter services. HTS alleges that the agency: improperly evaluated proposals under the most important technical evaluation factor on a pass/fail basis; improperly evaluated the protester's past performance; and failed to conduct an adequate price/technical tradeoff.[1]

We sustain the protest.

BACKGROUND

On March 3, 2008, the Forest Service solicited offers for exclusive use of heavy or medium helicopters for firefighting service at 34 locations, each of which would be the home base for one helicopter. The agency designated each location a separate contract line item number (CLIN). Each offeror could submit an offer for any or all of the 34 CLINs, propose as many helicopters as it chose for each CLIN, and propose the same helicopter for multiples CLINs.

The solicitation identified four technical factors, listed in descending order of importance: aircraft technical capability, safety/risk management, past performance, and organizational experience.[2] These four non-price factors, when combined, were significantly more important than price. RFP at 189. Past performance included four subfactors of equal weight, described as follows:

a) that you [the offeror] were capable, efficient, and effective
b) that your performance conformed to the terms and conditions of your contract
c) that you were reasonable and cooperative during performance
d) and that you were committed to customer satisfaction.

RFP at 189. The RFP specified that the agency's evaluation of past performance would include the offeror's contract performance for the years of 2005-2007. In preparing their proposals, offerors were instructed to utilize Form E-6, –Offeror's Past Performance and Organizational Experience,— included in the RFP. Awards were to be made to the offerors whose proposals were technically acceptable and –whose technical/price relationships [were] the most advantageous to the Government.— Id. at 190.

Thirty-four offerors submitted proposals, offering 88 helicopters.[3] The agency scored the proposals under the technical factors using a scale of one to five points--best to worst--corresponding to the adjectival ratings of exceptional, acceptable, neutral, marginal, and unacceptable. The agency provided no further definitions for the ratings. As explained further below, under the most important technical factor, aircraft technical capability, proposals could receive one of only two possible scores, two points (acceptable) or five points (unacceptable). Because of the complicated calculations required to make 34 simultaneous awards, where CLINs received multiple offers and aircraft were offered for multiple sites, the agency employed a computer program, called the Optimization Model, to assist it in making price/technical tradeoffs. CO's Supplemental Statement of Facts at para. 3.

The protester's proposal contained the requested Form E-6, which listed four Forest Service contracts, one from 2007 and three from 2005, as past performance references. The references contained only the contract number and not the required contact information. The protester's past performance documentation also included certificates of appreciation from 2007, a letter of appreciation from 2002, and an undated Forest Service evaluation report.

HTS's proposal received the following technical scores: aircraft technical capability, [DELETED]; safety/risk management, [DELETED]; past performance, [DELETED]; and organizational performance, [DELETED]. Multiplying those point scores by the assigned weighted averages, HTS's proposal received an overall technical score of 2.56. The technical evaluation team's (TET) summary comments on the protester's proposal stated, in full:

[DELETED]

AR, Tab 5, Memorandum from TET to Contracting Officer (CO), May 24, 2008, at 5.

The source selection authority adopted the recommendations of the TET without comment, awards were made, including four to HTS,[4] and this protest followed. Performance of the contested contracts has been stayed.

DISCUSSION

The protester challenges the award of Item Nos. 3, 7, and 8 to Erickson, and Item No. 6 to Columbia.[5] HTS argues that the agency failed to follow the evaluation criteria announced in the solicitation when it evaluated factor 1 on a go/no-go basis, and that it conducted an improper evaluation of HTS's past performance. HTS alleges that for each contested award the agency selected a higher-priced, higher-rated offer but failed to document a price/technical tradeoff.

Factor 1--Aircraft Technical Capability

HTS contends that the agency improperly failed to qualitatively evaluate the offerors' proposals under technical factor 1, aircraft technical capability, even though the RFP identified it as the most important technical evaluation factor. The agency asserts that the RFP provided that aircraft technical capability would be evaluated on a pass/fail basis, and that it was reasonable to assign all proposals that were rated acceptable or better the same score of two points (acceptable).

Where a dispute exists as to the meaning of solicitation language, we will resolve the matter by reading the solicitation as a whole and in a manner that gives effect to all provisions of the solicitation. See Honeywell Regelsysteme GmbH, B-237248, Feb. 2, 1990, 90-1 CPD para. 149 at 5. To be reasonable, an interpretation must be consistent with the solicitation when read as a whole and in a reasonable manner. Id.Here, we conclude that, when taken as a whole, the RFP contemplates the qualitative evaluation of all the technical factors, including factor 1.

As noted above, the solicitation listed all four technical factors and stated that, –when combined, [they were] significantly more important than price.— RFP at 189. The solicitation also stated that those four factors were –listed in descending order of importance.— Id.The RFP language here is similar to the language at issue in Lithos Restoration, Ltd., B'247003.2, Apr. 22, 1992, 92-1 CPD para. 379. In that case, the RFP provided that the –combined weight of the technical factors— was more important than price, Lithos Restoration, Ltd., supra at 4, and set forth the technical evaluation factors in descending order of importance. Id.As in Lithos Restoration, Ltd., we think that the RFP here, by listing the four technical factors in descending order of importance and calling for a price/technical trade-off that considered all four factors, clearly contemplated a qualitative evaluation of the technical factors, balanced against price, to determine the offers that are most advantageous to the government. Evaluating proposals under factor 1 on a go/no-go basis, as the agency did, is inconsistent with this announced evaluation scheme because it effectively gives no weight to factor 1 in the trade-off decision and makes the three less important factors the determining factors for award.

The agency argues that making factor 1 a go/no-go criterion does not diminish its importance in the evaluation scheme. We disagree. Assigning all technically acceptable offers the same score for the primary technical factor has essentially the same effect on the evaluation ratings as not ranking the most important factor at all, an action we found improper in Lithos Restoration, Ltd.. Assigning a score of two points to all technically acceptable offers created an evaluation rating floor; all offers would start off with the same points for that factor. This evaluation scheme effectively neutralized the influence of the most important factor and, as noted above, made the evaluation discriminators the three less important technical factors the determining factors for award.[6]

Past Performance

The protester alleges that the agency improperly focused its evaluation on the final year of the 3 years of past performance that should have been considered, and that the agency has not produced the necessary contemporaneous documentation to support the score [DELETED] given the protester. As a general matter, the evaluation of an offeror's past performance is a matter within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based past performance ratings. In determining whether a particular evaluation conclusion is reasonable, we examine the record to determine whether the judgment was reasonable and in accord with the evaluation criteria listed in the solicitation. Abt Assocs., Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD para. 223 at 4. Implicit in that examination is that the evaluation must be documented in sufficient detail to show that it was not arbitrary. Federal Acquisition Regulation (FAR) sections 15.305(a), 15.308; Quality Elevator Co. Inc., B-276750, July 23, 1997, 97'2 CPD para. 28 at 3.

According to the agency, the TET members relied primarily on an oral discussion of their individual experiences with HTS in arriving at the past performance rating of [DELETED]. Supplemental AR at xviii. There is no contemporaneous documentation of that discussion. In an evaluation that takes into account the agency's own knowledge of offerors' performance, the fundamental requirement that evaluation judgments be documented in sufficient detail to show that they are reasonable and not arbitrary must still be met. Omega World Travel, Inc., B'271262.2, July 25, 1996, 96-2 CPD para. 44 at 4.

The agency argues that evidence of HTS's past performance, first introduced into the record as part of the agency report on the protest, supports the reasonableness of the evaluation, citing Omega World Travel, Inc..[7] In that case, however, the record showed that the agency evaluators relied in their deliberations on specific evaluation material subsequently produced in the protest record. Here, there is no contemporaneous account of the discussion of HTS's past performance, let alone one that references the evaluators' reliance on any of the written documentation the agency supplied in response to the protest. The TET chair asserts that the team based its evaluation of the protester's past performance on –the written evaluations that it had,— Declaration of TET Chair at para.15, but fails to identify any specific document or to state whether the evaluators considered any of the contracts listed in the protester's proposal. In contrast, the TET chair goes into considerable detail, in response to the protest, recounting the personal experiences of the TET members that contributed to the rating. Id. at paras. 6'14. The documents produced in the agency report may or may not be part of the written evaluations that the TET asserts it relied on, but the TET chair's recollection of the evaluation process indicates that greater weight was given to the undocumented team discussion of past performance.

As discussed above, evaluations must be documented in sufficient detail to show that they were not arbitrary. FAR sections 15.305(a), 15.308. Here, without any contemporaneous documentation of that oral evaluation, and with no record, contemporaneous or otherwise, of what contract performance information was considered and how much relevance the information was accorded, we have no basis on which to conclude that the past performance evaluation was reasonable.[8]

PRICE/TECHNICAL TRADEOFF

Because we are sustaining the protester's challenges to the agency's evaluation of proposals under factor 1 on a go/no-go basis and the evaluation of HTS's past performance, we need not address the protester's challenge to the agency's price/technical trade-off. In this regard, we recognize that the agency, faced with evaluating a substantial number of proposals from multiple offerors, decided to use a computer program to assist in the evaluation and selection decisions. Under these circumstances, an agency's reasonable attempts to simplify the source selection process are not objectionable. Apex Marine Ship Mgmt. Co., LLC; American V-Ships Marine, Ltd., B-278276.25; B-278276.28; Sept. 25, 2000, 2000 CPD para. 164 at 21. Moreover, when numeric scores are reasonable and supported by the contemporaneous narrative record, an agency may use the results of the numeric scoring as indicative of whether evaluated technical superiority is worth the associated cost premium. Id.Nevertheless, as was the case here for a number of the award decisions, when a price/technical tradeoff is made, the source selection decision must be documented, and the documentation must include the rationale for any tradeoffs made, including the benefits associated with additional costs. FAR sect. 15.308.

RECOMMENDATION

For the four contested awards, we recommend that the agency either qualitatively evaluate the eligible aircraft under technical factor 1, or revise the solicitation to state that it will evaluate factor 1 on a no/no-go basis and solicit revised proposals for the challenged CLINs. In any event, we also recommend that the agency reevaluate HTS's past performance. We recommend that the agency then conduct and document a new source selection decision for those four CLINs and make award to the offeror(s) whose proposal(s) are found to represent the best value to the government. We also recommend that the protester be reimbursed its costs of filing and pursuing its protest, including reasonable attorneys' fees.

The protest is sustained.

Gary L. Kepplinger
General Counsel