Skip to main content

Matter of: Telos Field Engineering File: B-253492.6 Date: December 15, 1994

B-253492.6 Dec 15, 1994
Jump To:
Skip to Highlights

Highlights

Protest challenging agency's technical evaluation of proposals is sustained where the evaluation was neither reasonable nor consistent with the solicitation. This is the second protested award under this RFP. The contract is for a 2-year base period. That award would be made to the responsible offeror whose proposal was determined to be "most advantageous" to the agency. Under which 465 points (50 percent of the total) were assigned through the technical evaluation. The other 465 points were reserved for cost. The technical points were assigned according to how well each proposal scored on various aspects of the RFP's evaluated optional features. While other proposals' cost scores were based on how close their price was to the lowest one.

View Decision

Matter of: Telos Field Engineering File: B-253492.6 Date: December 15, 1994

Protest challenging agency's technical evaluation of proposals is sustained where the evaluation was neither reasonable nor consistent with the solicitation, and the errors in the evaluation affected the outcome of the competition.

Attorneys

DECISION

Telos Field Engineering protests the award of a contract to Concept Automation, Inc. (CAI) under request for proposals (RFP) No. YJ- 93525E, issued by the Tennessee Valley Authority (TVA). Telos challenges the contracting agency's evaluation of proposals and the conduct of discussions.

We sustain the protest.

This is the second protested award under this RFP. Telos had earlier protested TVA's award of a contract to Employee Owned Maintenance Company, Inc. (EOMC). We sustained that protest because the agency improperly failed to request BAFOs at the conclusion of discussions. Telos Field Eng'g, 73 Comp.Gen. 39 (1993), 93-2 CPD Para. 275. After receiving our decision, TVA requested BAFOs, performed an evaluation, and recommended that award be made to CAI. The present protest challenges the agency actions leading up to the selection of CAI.

BACKGROUND

TVA issued the underlying RFP on September 3, 1992, seeking proposals for a time-and-materials contract to provide maintenance services for computer hardware at a number of different locations. The contract is for a 2-year base period, with four 1-year options. Section M of the RFP stated that technical factors (specifically, what the RFP called evaluated optional features) and cost would be given equal weight in source selection, and that award would be made to the responsible offeror whose proposal was determined to be "most advantageous" to the agency.

To implement the RFP award criteria, the agency established a 930- point scheme, under which 465 points (50 percent of the total) were assigned through the technical evaluation; the other 465 points were reserved for cost. The technical points were assigned according to how well each proposal scored on various aspects of the RFP's evaluated optional features. On the cost side, the proposal with the lowest proposed price received 465 points, while other proposals' cost scores were based on how close their price was to the lowest one. Technical and cost points were then added, and the proposal with the highest total score was deemed to be the most advantageous to the agency.

After receiving initial proposals in November 1992, the agency sent offerors letters, dated December 9, seeking further information. Based on the initial proposals and the offerors' responses to the agency's December 1992 letters, the evaluators assigned point scores to each technical and cost proposal. Without requesting BAFOs, the agency totaled the technical and cost points, determined that, since EOMC's initial proposal received the highest combined point score, it was the most advantageous to the agency, and recommended that award be made to EOMC. Telos's initial protest followed.

In our decision on that protest, we concluded that Telos was prejudiced by the agency's failure to request BAFOs. We recommended that TVA reopen negotiations with all offerors whose proposals were in the competitive range and then request BAFOs.

In January 1994, in response to our recommendation, TVA requested BAFOs from 11 offerors, including Telos.[1] CAI and Telos proposed practically identical prices, which were lower than any other offeror's. Accordingly, both received the maximum cost score. Overall the two firms' BAFOs were evaluated as follows:

Technical Score Cost Score Total

Telos 428 465 893 CAI 445 465 910

The agency determined that, since CAI's proposal received the highest combined point score, it was the most advantageous to the agency. The TVA Board approved the award to CAI on July 20.

Upon notification of TVA's determination, Telos filed the present protest with our Office, and then Telos filed a supplemental protest following a July 25 debriefing. After deciding that urgent and compelling circumstances necessitated an award notwithstanding the protest, the agency awarded a contract to CAI on August 19, and issued a notice to proceed on the same date.

TECHNICAL EVALUATION OF PROPOSALS

The evaluation of technical proposals is primarily the responsibility of the contracting agency. Our Office will not make an independent determination of the merits of technical proposals; rather, we will examine the record to ensure that the agency's evaluation was reasonable and consistent with the stated evaluation criteria. Litton Sys., Inc., B-237596.3, Aug. 8, 1990, 90-2 CPD Para. 115.

We have reviewed Telos's allegations regarding the technical evaluation and, as set forth below, we find that there were several proposal evaluation deficiencies, the correction of which would result in Telos's proposal receiving the highest score under the evaluation formula.[2]

EVALUATION OF TELOS'S PROPOSAL

Local Stocking Warehouses

Section M of the RFP required offerors to respond to the following: "Dedicated repair parts inventory with local stocking warehouses. Where and how many?" An offeror was credited under this evaluation factor with 4 points for each specified local stocking warehouse, at up to a maximum of five locations (20 points), and 4 points for a parts inventory dedicated to this contract, for a possible total of 24 points.

The agency credited Telos with four points for one local stocking warehouse, and four points because Telos's BAFO stated that the parts would be dedicated to this contract. Telos argues that the agency improperly scored its proposal under this evaluation factor because it "unambiguously listed seven locations where it would store its spare inventory of $1,000,000."

TVA acknowledges that Telos stated that it would provide seven dedicated local repair parts facilities, but argues that Telos's proposal, when read as a whole, was ambiguous regarding the number of local stocking warehouses that it was proposing. TVA points to the following language from Telos's BAFO as evidence of the ambiguity regarding this matter:

"Telos will concentrate its assigned personnel and parts inventories at the main TVA locations in Chattanooga and Knoxville, with additional technical personnel and spares located at Sequoyah, Browns Ferry, Watts Barr, Bellafonte, and Muscle Shoals. The proposed `on-site' Telos inventory of approximately $1,000,000 will be assigned at these facilities on a rata basis to the equipment types and models listed by TVA."

The agency interpreted this statement to mean that Telos was proposing local stocking warehouses at the main TVA locations in Chattanooga and Knoxville, with spares located at other locations, and that the parts would be dedicated to TVA. Transcript (Tr.) at 104.[3] The agency concedes that Telos should have received 8 points for proposing two local stocking warehouses, and 4 points for dedicating the parts to this contract, for a total of 12 points. Tr. at 106 and 158. The agency did not interpret Telos's intention to stock spares at five locations to mean that those five locations were local stocking warehouses. Tr. at 121. Regarding spare parts, one evaluator noted that, "A spare part to me could mean one hard drive. . . ." Tr. at 120.

We find that the imprecise language in Telos's BAFO reasonably led the agency to conclude that Telos was committing to provide only two local stocking warehouses, even though some spare parts might be located at the additional five facilities. Since we agree that Telos's proposal was ambiguous regarding its commitment to provide more than two stocking warehouses, and the agency's interpretation was reasonable, we find the agency's evaluation under this criterion unobjectionable, except for the four-point error conceded by the agency.[4]

Service Contract Sales

Under this criterion, evaluators examined the number and type of contracts that offerors had performed. TVA's internal evaluation plan, which was not disclosed to offerors, reserved 2 of the available 28 points to proposals where prior work was performed on- site. Telos received only 26 out of the possible 28 points because the agency determined that none of Telos's previous contracts were performed on-site.

Telos points out that its proposal explicitly stated that Telos had provided on-site maintenance at Letterkenny Army Depot and Blue Grass Army Depot. Our review confirms that Telos's proposal stated that it had provided "on-site maintenance services" at those two locations, and TVA has provided no basis to discount this information. Accordingly, under TVA's evaluation scheme, Telos should have been credited with two points for listing on-site contracts in its proposal.

Warranty Service

Under this criterion, evaluators were to consider the offeror's ability to provide warranty service. In this regard, an offeror was credited with four points for each manufacturer listed in its proposal for which it was authorized to provide warranty service, with a maximum of 40 points available.

Telos listed nine manufacturers for which it provides warranty service, and received 36 points. Telos argues that it should have received an additional four points because its proposal included a list of 125 vendors "supported by Telos." The agency responded that Telos was not credited with an additional four points for listing manufacturers that it supported, because Telos did not indicate in its proposal that it provided warranty service for these manufacturers. Tr. at 93.

We find that TVA reasonably credited Telos only for the nine manufacturers listed in its proposal for which it stated that it provided warranty service. The agency properly did not credit Telos for "supporting" additional vendors since, as TVA argues, supporting a particular vendor does not necessarily mean receiving the authorization generally needed from the manufacturer to provide warranty service for its products. Tr. at 93. We conclude that there is no basis to question this aspect of the evaluation.[5]

Demonstrated Success With Customers

Under this evaluation factor, evaluators examined each offeror's demonstrated success with hardware service and support provided to customers similar to TVA. An offeror was credited with 4 points for each such customer listed in its proposal, up to a possible total of 40 points.

Telos listed 15 contracts with nine customers similar to TVA. Based on Telos having nine such customers, its proposal received 36 points. Telos contends that it deserved full credit under this criterion because it listed 15 contracts. In support of its argument, Telos notes that, according to the agency's internal evaluation plan, an offeror was to receive four points for each "contract/customer."

We disagree. It is clear that this evaluation criterion was included in the RFP to examine each offeror's success with customers, particularly customers similar to TVA. To the extent that Telos relies on language in the TVA internal evaluation plan to support its argument that TVA was required to evaluate an offeror's contracts, its reliance is misplaced. Evaluation plans are internal agency instructions and as such do not give outside parties any rights. Aerospace Design, Inc., B-247793, July 9, 1992, 92-2 CPD Para. 11. The agency is required to follow the evaluation scheme set forth in the RFP. Id. In this case, the RFP evaluation scheme specifically referred to "customers equivalent to TVA." Accordingly, counting customers rather then contracts was consistent with the RFP, and we find the agency's evaluation under this criterion unobjectionable.[6]

Total Quality Program

Offerors under this evaluation criterion were required to show a company-wide commitment to quality, including an active total quality management (TQM) program, a supplier quality assurance program, a statistical quality management program, and a reference to ISO 9001 standard. This evaluation criterion had a maximum score of seven points, and, under the agency's undisclosed evaluation plan, was scored on an all-or-nothing basis.

Telos's proposal received zero points under this evaluation criterion. TVA noted that Telos's proposal exhibited a "company- wide commitment to quality" and referenced the ISO 9001 standard. Tr. at 66 and 127-128. However, TVA also noted that Telos proposed a quality control (QC) program in which TVA's only participation would consist of Telos's customer visits (that is, visits by Telos to TVA) apparently in response to problems that arose. One evaluator stated, "[Telos] said there would be customer visits but it was for service and not so much for customer satisfaction." Tr. at 113. TVA viewed Telos's proposed customer visits as involving TVA later in the process than would be the case with a TQM program, where the customer would be involved as part of a quality council convened regularly throughout the contract period.

In contrast, CAI's proposed TQM program included a quality council. While TVA was impressed by CAI's customer-driven response to this evaluation factor, it did acknowledge that CAI's proposal did not provide a supplier quality assurance program and did not reference ISO 9001. Tr. at 132-133. Nonetheless, CAI's proposal received the full seven points under this evaluation factor.

We find that TVA had a reasonable basis for finding CAI's proposal superior under this factor. CAI's proposal evidenced a TQM program that involved TVA early in the process in the context of a quality council, while Telos proposed a QC program with TVA's involvement apparently limited to customer visits. TVA has not provided a reasonable basis to assign the full seven points to CAI, whose proposal did not meet all the requirements of this factor and zero points to Telos, whose proposal satisfied some of those requirements. In this regard, TVA appears to have given the use of TQM methodology weight in excess of that indicated to the offerors by the RFP evaluation criteria. Whatever point differential could be justified by the evaluated difference in the merits of the two proposals, the record provides no reasonable basis for a seven-point difference in scores.

EVALUATION OF CAI'S PROPOSAL

One Number Calling

Under the "one number calling" factor, proposals could receive 30 points for offering a single point of contact for all support services and 20 points for offering remote diagnostic capability. Telos argues that, upon receipt of BAFOs, TVA unreasonably increased CAI's score by 20 points under this evaluation factor, even though CAI made no relevant changes to its technical proposal in its BAFO.

TVA explains that in its initial evaluation it gave CAI credit for one number calling (30 points), but none for remote diagnostic capability. TVA states that, upon receipt of BAFOs, it reexamined the initial proposals as well as the BAFOs. Tr. at 59. In reviewing CAI's initial proposal, TVA concluded that CAI had demonstrated remote diagnostic capability, and CAI's score was raised accordingly. Telos contends that the Dispatch-1 System offered by CAI is no more than a telephone hotline with access to an automated data base, and does not have remote diagnostic capability.

Section M of the RFP required offerors to demonstrate "automated procedures such as remote diagnostic capability." While our review confirms that CAI's proposal included a reference to "multiple level structure of decision support information (including technical data) for solving customer problems by phone," we see no basis to conclude that this or other references in the proposal could reasonably be interpreted as demonstrating remote diagnostic capability, as TVA found, or any other relevant automated procedure.[7] Accordingly, we find no basis for the 20-point increase in CAI's score at BAFO.

Experience

Similarly, Telos argues that TVA improperly increased CAI's score at BAFO by 30 points under the experience evaluation factor, even though CAI had made no relevant changes to its proposal. Under this factor, an offeror was credited with 10 points for each year of experience with similar contracts beyond the 3 required years, up to 100 points. Accordingly, an offeror with 12 years experience would receive 90 points (10 points for each of the 9 years beyond the required 3), while one with 13 or more years would receive 100 points.

In the initial evaluation, CAI's proposal was awarded 70 points for this evaluation factor. Upon reexamination of CAI's initial proposal after submission of BAFOs, TVA determined that CAI had listed "at least" 13 years of experience with similar contracts and raised its score to 100 points. We find no reasonable basis for this determination.

The earliest contract that CAI listed in its proposal began on April 5, 1982, and the proposal listed contracts that CAI performed up through the time that BAFOs were received and evaluated in February 1994. Therefore, at the time that BAFOs were submitted, and assuming all CAI's contracts were relevant, CAI had 11 years and 11 months of experience. TVA explains that it credited CAI with an entire year of experience for 1982, even though CAI claimed only 9 months of experience in that year, and for each year up to and including all of 1994, even though BAFOs were submitted in February, totaling 13 years.

While TVA could reasonably round CAI's 11 years and 11 months of experience up to 12 years (which would give CAI 90 points out of the total 100) it was unreasonable for TVA to credit CAI with experience which it never claimed and is not supported by its proposal. We therefore conclude that the 10 points for the 13th year were added to CAI's score without a rational basis.

CONCLUSION

There were many irregularities throughout the source selection in this procurement. The scoring methodology, as implemented by TVA, was not consistent throughout the RFP, and the methodology used in some instances was inconsistent with the RFP evaluation criteria. Additionally, TVA's evaluation of technical proposals was poorly documented at best, and at times not documented at all. Agencies are required, in order to protect the integrity of the public procurement process, to document their evaluation of proposals and their selection decisions so as to show the relative differences between proposals, their weaknesses and risks, and the basis and reasons for the selection decision. See KMS Fusion, Inc., B-242529, May 8, 1991, 91-1 CPD Para. 447. In several instances in the course of this protest, TVA provided our Office with after-the-fact explanations for the technical scoring, where the record was bereft of any contemporaneous narrative or analysis.

Finally, as detailed above, there were several instances in which the evaluation of proposals was based on plain error or was otherwise unreasonable. The net result of the deficiencies in the evaluation of proposals was that Telos's proposal was improperly denied four points for stocking warehouses (an error conceded by the agency), two points for "on-site" contracts (apparently due to an oversight), and several points due to the way in which TVA applied the all-or-nothing scoring method for the total quality program factor. In addition, CAI was improperly credited with 10 points for experience not claimed in its proposal, 20 points for a remote diagnostic capability which it did not offer, and several points for its total quality program. The correction of these scoring errors raises Telos's technical score from 428 to at least 434, and lowers CAI's technical score from 445 to no more than 415.[8] We conclude that, but for TVA's evaluation errors, Telos's total score would have substantially exceeded CAI's (as noted above, the two proposals' cost scores were identical).

When point scores are used, they typically are not controlling, but are used as guidance by the source selection officials. Grey Advertising, Inc., 55 Comp.Gen. 1111 (1975), 76-1 CPD Para. 325. These officials have broad discretion to determine the manner and extent to which they will make use of point scores, as well as other elements of technical and cost evaluation results. Dyncorp, B-245289.3, July 30, 1992, 93-1 CPD Para. 69. Thus, TVA could have made award to CAI notwithstanding Telos's proposal receiving a higher score, if the agency reached a documented determination, based on a reasoned analysis of the relative merits of the proposals, that the evaluated optional features which CAI offered in its proposal were, in fact, superior to those offered by Telos.

In this procurement, however, TVA twice appears to have based its award decision entirely on the total point score, as a result of which the errors noted in our decision were determinative. In view of TVA's insistence on this mechanical award determination formula, Telos's proposal should have been selected. Secure Servs. Technology, Inc., B-238059, Apr. 25, 1990, 90-1 CPD Para. 421.

Since it is relatively early in the contract performance period, we recommend that TVA terminate CAI's contract for the convenience of the government and make award to Telos, if otherwise eligible, unless TVA reaches a reasonable, documented determination that CAI's proposal is the most advantageous to the government. In any event, TVA should reimburse Telos for its reasonable costs of filing and pursuing this protest, including attorneys' fees. 4 C.F.R. Sec. 21.6(d)(1) (1994). In accordance with 4 C.F.R. Sec. 21.6(f), Telos's certified claim for such costs, including the time expended and costs incurred, must be submitted directly to the agency within 60 days after receipt of this decision.

The protest is sustained.

1. TVA did not conduct discussions prior to requesting BAFOs because it determined that offerors had been adequately apprised of deficiencies in their offers by the December 1992 letters.

2. TVA is subject to the bid protest provisions of the Competition in Contracting Act because it falls within the definition of "federal agency" to which those provisions apply. 31 U.S.C. Sec. 3551(1) (1988); 40 U.S.C. Secs. 472(a) and (b) (Supp. V 1993). See also Telos Field Eng'g, B-257747, Nov. 3, 1994, 94-2 CPD Para. 172.

3. Transcript citations refer to the transcript of the hearing conducted by our Office in connection with this protest.

4. CAI's BAFO was evaluated similarly. CAI was given four points for proposing one local stocking warehouse and four points for stating that the parts would be dedicated to this contract. CAI received no credit for its proposed plan to equip each field engineer with a spare parts kit based on the equipment being maintained at the assigned TVA location. Tr. at 22.

5. CAI's proposal received the entire 40 points under this evaluation criterion because it listed 38 manufacturers for which it currently provides warranty service.

6. CAI received the full 40 points for this evaluation criterion, because it listed contracts with 10 customers that were similar to TVA.

7. In contrast, Telos proposed a single point of contact for all support services and the capability for remote diagnostics through a dial-up modem, and therefore properly received the full 50 points under this evaluation criterion.

8. Telos also contends that TVA failed to conduct meaningful discussions with offerors before the request for BAFOs was made. Telos points to our November 16, 1993, decision, which recommended that TVA reopen discussions with all offerors whose proposals were in the competitive range, before requesting BAFOs. As noted above, after receipt of our decision, TVA determined not to conduct further discussions prior to requesting BAFOs. It does appear that certain of the evaluation deficiencies might well have been ameliorated if, prior to the request for BAFOs, offerors had been afforded meaningful discussions regarding aspects of their proposal which were unclear; for example, discussions could have corrected the agency's apparent misreading of Telos's on-site service contract experience and may have led the agency to give Telos credit for more local stocking warehouses. We need not address this issue, however, because the effect of the extensive pleadings in this protest, as reflected in our recommendation, has been to provide the agency with essentially all the information that it would have learned if discussions had been held.

GAO Contacts

Office of Public Affairs