Skip to main content

Matter of: TRESP Associates, Inc.; Advanced Data Concepts, Inc. File: B-258322.5; B-258322.6 Date: March 9, 1995

B-258322.5,B-258322.6 Mar 09, 1995
Jump To:
Skip to Highlights

Highlights

Evaluation of protester's proposal was unreasonable where agency identified weaknesses during discussions. Record indicates that many of the weaknesses were thereby eliminated. Proposal rating was not increased at all and record does not explain why increase was not warranted. Cost/technical tradeoff was unreasonable where agency selected higher-cost proposal for award over lower-cost. Selection is not supported by the record. Finding that awardee had the ability to respond more quickly was based largely on source selection official's general recollection of the two firms' prior performance. Which is not documented in either the contemporaneous or protest records. It is not apparent. Awardee's initial proposal was evaluated similarly to protester's.

View Decision

Matter of: TRESP Associates, Inc.; Advanced Data Concepts, Inc. File: B-258322.5; B-258322.6 Date: March 9, 1995

Evaluation of protester's proposal was unreasonable where agency identified weaknesses during discussions, protester addressed the weaknesses, and record indicates that many of the weaknesses were thereby eliminated, but proposal rating was not increased at all and record does not explain why increase was not warranted. Cost/technical tradeoff was unreasonable where agency selected higher-cost proposal for award over lower-cost, similarly rated proposal based primarily on awardee's ability to respond more quickly on short notice, and the fact that awardee's initial proposal contained fewer weaknesses, purportedly indicating better understanding of requirement, and selection is not supported by the record. Finding that awardee had the ability to respond more quickly was based largely on source selection official's general recollection of the two firms' prior performance, which is not documented in either the contemporaneous or protest records; and record contains no explanation, and it is not apparent, why the types of weaknesses identified bear on understanding and, in any case, awardee's initial proposal was evaluated similarly to protester's.

Attorneys

REDACTED DECISION

A protected decision was issued on the date below and was subject to a GAO Protective Order. This version has been redacted or approved by the parties involved for public release.

DECISION

TRESP Associates, Inc. and Advanced Data Concepts, Inc. (ADC) protest the Department of Energy's (DOE) award of a contract to M.H. Chew & Associates, Inc. (CAI), under request for proposals (RFP) No. DE-RP03-93SF19686, for technical and management services. TRESP primarily challenges the evaluation of its proposal as improper, and ADC primarily argues that the cost/technical tradeoff was improper.

We sustain the protests.

The RFP, issued on May 14, 1993, as a small business set-aside, contemplated the award of a 5-year--2 base years plus 3 option years--cost-plus-fixed-fee contract to support DOE programs in San Francisco, California, by furnishing technical and management services in several areas (e.g., technical studies, environment, safety, health and facility oversight, and safeguards and security technical services). The solicitation included an estimate of the agency's total requirement--including 68,100 (approximately 33 persons) direct productive man-hours for the first year and 79,100 man-hours (approximately 38 persons) for the second year--and an estimate of the total man-hours for each of the 27 required key and non-key labor categories. Offerors were required to propose certain key and non-key personnel with "L" or "Q" security clearances, and to show that they could obtain a DOE-approved classified facility.

Award was to be made to the responsible offeror whose offer conformed to the RFP and was most advantageous to the government under three broad criteria: (1) technical, which was more important than (2) business/management, which was more important than (3) cost. There were four technical evaluation factors (in descending order of importance): (1) qualification of proposed personnel and personnel management, including subfactors for technical experience and offeror's approach to staffing the required work; (2) technical experience and past performance; (3) technical approach; and (4) task management. There were five business/management factors, all of equal importance: (1) business/management organization; (2) business/management experience on similar contracts; (3) cost management; (4) subcontracting and subcontract management; and (5) labor management relations.

Seven proposals were received by the closing date. Three--TRESP's, CAI's, and ADC's--were included in the competitive range. Following site visits and oral and written discussions with the offerors, the agency requested best and final offers (BAFO). Based upon its evaluation of the BAFOs, the source evaluation board (SEB) concluded that only the responses of ADC and CAI warranted increases to their original scores. The evaluation scores (received/available) were as follows: [1]

TECHNICAL ADC CAI TRESP Personnel Qualifications Experience 16/22.5 14 15 Staffing 18/22.5 17 15 Technical Experience 31/35 28 25 Technical Approach 19/25 20 16 Task Management 9/15 12 10 SUBTOTAL 95/120 90 82 BUSINESS/MANAGEMENT Organization 15/20 16 16 Experience 16/20 15 16 Cost Management 14/20 17 12 Subcontracting 14/20 15 13 Labor Mgmt. Relations 13/20 15 15 SUBTOTAL 72/100 77 70

TOTAL 167/220 167 152

PROPOSED COST $21,262,193 $23,380,018 $23,771,519 EVALUATED COST $20,669,538 $21,917,802 $23,600,000 [2]

In its report to the source selection official (SSO), the SEB summarized its evaluation of BAFOs without recommending any firm for award.

The SSO conducted a cost/technical tradeoff based on his own knowledge and experience, the SEB report, the most probable cost analysis, and information presented to the SSO by the SEB chairman. In comparing ADC's and CAI's BAFOs, the SSO determined that although both BAFOs received almost the same technical ratings, the technical strengths of CAI's BAFO outweighed its evaluated cost premium ($1,248,264). CAI's BAFO also was found more advantageous than TRESP's on the basis of its higher scores under the technical and business/management factors and its lower probable cost.

TRESP's PROTEST

TRESP challenges DOE's evaluation of its BAFO on the basis that the agency improperly failed to recognize the improvements in its BAFO. Specifically, the protester notes that although the final SEB report stated that all of the 12 "weaknesses" that it found in, and "observations" that it made with respect to, TRESP's initial technical and business/management proposals were "adequately addressed in the BAFO" or "remedied," TRESP's final evaluation score was not increased under any factor or subfactor.

In reviewing protests against allegedly improper evaluations, our Office will examine the record to determine whether the agency's determination was reasonable and consistent with the evaluation criteria listed in the solicitation. Amtec Corp., B-240647, Dec. 12, 1990, 90-2 CPD Para. 482, recon. denied, Dept. of the Army--Recon., B-240647.2, Feb. 26, 1991, 91-1 CPD Para. 211. Implicit in the foregoing is that the agency must document its judgments in sufficient detail to show that they are not arbitrary. U.S. Defense Sys., Inc., B-245563, Jan. 17, 1992, 92-1 CPD Para. 89; Beckman Instruments, Inc., B-246195.3, Apr. 14, 1992, 92-1 CPD Para. 365. In this regard, Federal Acquisition Regulation (FAR) Sec. 15.608 requires the contracting agency to prepare specific documentation showing the basis for evaluation, an analysis of the technically acceptable proposals, including an assessment of each offeror's ability to accomplish the technical requirement, and a summary of findings. DOE did not meet this standard here.

The record shows that TRESP submitted a BAFO which responded to the concerns raised by the agency during discussions and which led the SEB to state in its final report that the concerns had been "adequately addressed in the BAFO" (or, with respect to one area, had been "remedied"). Further, the agency concedes in the agency report that the SEB members in fact believed that most of the weaknesses identified in TRESP's initial proposal had been corrected. In this regard, the agency has submitted declarations from four of five SEB members stating that, except for the evaluated weakness of TRESP's proposed non-key Health Physicist and Environmental Engineer, "most of the weaknesses I identified in TRESP's initial proposal were corrected," or simply "the weaknesses I identified in TRESP's initial proposal were corrected."

Although the agency generally maintains, and these declarations further state, that the "clarifications and improvements were not significant enough . . . to change [the] scoring for TRESP's BAFO," there is nothing in the record that explains the specific reasons underlying the determination not to increase TRESP's BAFO score under any factor or subfactor. [3] While TRESP may not have been entitled to an increase in points under each of the factors and subfactors where the SEB noted that its concerns were adequately addressed, we can find no reasonable basis from our review of the record for concluding that TRESP was entitled to no additional points under any evaluation subfactor or criteria as a result of the changes it made in its BAFO in response to the concerns raised by the agency during discussions.

For example, under the technical experience and past performance factor of the technical criterion, where TRESP received 25 of 35 points, and the business management experience on similar contracts factor of the business/management criterion, where TRESP received 16 of 20 points, the agency was concerned that TRESP's initial proposal failed to show that TRESP, as opposed to its proposed subcontractors, had prior experience in performing contracts as wide and diverse in scope and of a similar level of effort--approximately 33-38 persons during the base period--as contemplated under the current statement of work. To alleviate the concern raised during discussions in this regard, TRESP submitted in its BAFO a detailed description of numerous tasks and services performed by TRESP itself under current and prior government contracts, including contracts involving levels of effort of up to 25, 27, and 80 employees. The record shows that the SEB not only was satisfied that TRESP's response alleviated its concerns but, indeed, considered TRESP's experience a strength. In this regard, the SEB's final report noted that TRESP's BAFO "adequately addressed" its concerns and described as a strength TRESP's "good" and "similar" contract experience with DOE. Nevertheless, TRESP's BAFO scores did not increase. DOE does not explain, nor is it otherwise evident from the record, why an increase in TRESP's scores was not warranted by the more detailed description in its BAFO of its experience.

As another example, under the cost management factor of the business/management criterion, where TRESP received only 12 of 20 points, the SEB determined that TRESP's initial proposal did not demonstrate an ability to track and control costs and achieve cost targets. In response to the agency's request during discussions that TRESP provide examples of a proven ability to track and control costs with its proposed automated accounting system, TRESP described in detail its system for tracking and analyzing costs. Although the SEB report stated that this concern was "adequately addressed" in TRESP's BAFO, there was no increase in TRESP's BAFO score under this factor. Again, DOE does not specifically explain, and it is not evident, why an increase in TRESP's score was not warranted.

As a further example, under the subcontracting and subcontract management factor of the business/management criterion, where TRESP received 13 of 20 points, the SEB determined that TRESP's initial proposal had not clearly defined how it would, nor demonstrated the ability to, manage subcontractor performance through contract completion. In response to the agency's request during discussions to explain how it would manage subcontractor performance, TRESP explained in its BAFO that it would monitor subcontractor performance in three different ways: (1) by analyzing milestones accomplished against the subcontractor task plan; (2) by comparing forecasted expenditures with the costs incurred; and (3) by making frequent quality control checks throughout subcontractor task performance to discover potential problems, and then redirecting resources from noncritical tasks, revising the subcontractor task plan or expected milestones, and developing and implementing a recovery plan. In addition, TRESP explained that subcontractor management would be facilitated by co-locating subcontractor personnel with TRESP personnel at DOE or TRESP sites. Again, although the SEB report stated that this concern was "adequately addressed" in TRESP's BAFO, there was no increase in TRESP's BAFO score under this factor. And again, nothing in the record shows why TRESP's score was not increased.

Given the evaluators' own statements that TRESP's BAFO resolved the deficiencies raised during discussions, our own review which supports these statements, and the absence of documentation showing otherwise, it appears that TRESP's BAFO was not properly evaluated; although the SEB found that TRESP had largely remedied the initial proposal weaknesses, TRESP's score was not increased at all. At minimum, the evaluation record is inadequate to establish that the TRESP's BAFO was properly scored. As it also is not possible to determine the precise impact of any evaluation improprieties on TRESP's score, we can only conclude that a proper evaluation could result in TRESP's score increasing sufficiently to displace both CAI and ADC. We therefore sustain the protest on this basis.

ADC's PROTEST

ADC's protest turns on the same essential issue as TRESP's--whether the record supports the SSO's cost/tradeoff decision. The SSO selected CAI for award based primarily on three factors: (1) CAI's superior past performance; (2) CAI's superior depth of staffing; and (3) CAI's submission of a superior initial proposal. ADC maintains that the tradeoff--resulting in selection of CAI's BAFO, which received the same evaluation score as ADC's but was evaluated as $1,248,264 (or 6 percent) higher in cost--was not adequately supported by the record. We agree.

Past Performance

The SSO's determination that CAI's past performance was superior to ADC's was based on his personal knowledge and belief that ADC's proposed subcontractor (PAI) had failed to timely accomplish several task assignments under a current DOE Oakland contract (and thus could not be relied upon to timely support the contemplated contract effort here). Such personal knowledge of course may be considered in an evaluation. See NDI Eng'g Co., B-245796, Jan. 27, 1992, 92-1 CPD Para. 113. However, reliance on personal knowledge of the offerors does not eliminate the fundamental requirement that evaluation judgments be documented in sufficient detail to show that they are not arbitrary. U.S. Defense Sys., Inc., supra. DOE did not meet this standard in its cost/technical tradeoff decision.

The record contains no factual information supporting the SSO's conclusion that CAI's prior experience was superior to ADC's. Specifically, the SSO does not cite--and the record does not reference--any specific CAI contracts, or examples of CAI's performance that was deemed superior, or any similar details of ADC's or PAI's performance that would support a conclusion that their performance was inferior to CAI's. Rather, the SSO merely references his "years of personal experience dealing with CAI and their proposed subcontractors and ADC and their proposed subcontractors," and states that CAI has demonstrated superior performance in the areas of environment, safety and health, and management support, while ADC's performance in the area of safety and security "in my opinion was not superior." These conclusionary statements, without some specific explanation, are an inadequate basis for the SSO's evaluation of CAI and ADC, since it is not possible to determine whether the SSO's judgment was reasonable. In fact, regarding PAI, the only detailed information in the record is inconsistent with the SSO's conclusion; ADC has furnished three letters from DOE to PAI commending PAI for its performance on prior DOE contracts. Indeed, one of the letters from DOE commends PAI for its "quick turnaround on request changes" on the DOE Oakland contract, stating that PAI's "efforts were instrumental in completing this [facility representative training and certificate] plan on time to allow me to submit it to management."

In light of the FAR Sec. 15.608 requirement for adequately documented source selection decisions, we do not think the SSO's nonspecific, undetailed references provided a reasonable basis for distinguishing between two similarly evaluated proposals in a cost/technical tradeoff.

Depth of Staffing

The SSO states that "the most important factor that caused me to choose CAI over ADC was the breadth and depth of immediately available support staff. This capability to expeditiously respond, and the depth of varied and experienced talent was very persuasive to my decision." However, while CAI did have 400 individuals within 100 miles of the Livermore area and 750 "Q" cleared personnel nationwide, ADC's proposal showed 277 "Q" or "L" cleared personnel. Given that the required staffing under the contract was only 33-38 individuals, and that the contractor would be obligated to provide no more than 110 percent of the estimated level of effort, we do not think there was a reasonable basis for assigning great significance to CAI's larger staff in the tradeoff decision. While redundancy in staffing may be a legitimate consideration for a requirement such as the one here, it is not reasonable to evaluate a proposal as superior to another where the larger staff will not benefit the government. Both CAI and ADC have what appears to be significant redundancy for the level of effort required here. CAI's staff certainly is much larger than ADC's, but the record does not explain why CAI's large number of additional personnel was considered beneficial in any significant way. DOE does not assert, for example, that such a large number of employees (in relation to the required staffing) has been necessary in the past to properly perform similar requirements.

The SSO's conclusion regarding the superiority of CAI's available staffing was based in part on his "concern about ADC's ability to provide timely support staff." This concern stemmed from (1) his personal knowledge of PAI's performance on the DOE Oakland contract, and (2) the SSO's opinion that while ADC's BAFO provided as an example of expeditious performance the fact that it pulled together a 35 member "Q" cleared team in 2 weeks, CAI could have assembled a similar team in a much shorter period of time. As stated previously, the SSO's personal knowledge of PAI's past performance consists entirely of conclusionary statements. His opinion is also questionable because CAI's proposal does not reference similar accomplishments (and the SSO did not state that he was aware of any), and the SEB actually cited this mobilization example as a strength in ADC's evaluation because it shows a "demonstrated ability to respond to short notice/high volume task assignments." The SSO properly may reject the SEB's evaluation conclusions where he disagrees with them, see Wyle Labs., Inc.; Latacoere Int'l, Inc., 69 Comp.Gen. 648 (1990), 90-2 CPD Para. 107, but his different judgment must be reasonable. Id. The SSO's conclusion regarding CAI's capability, and his essential disagreement with the SEB that ADC actually was strong in this area, are unsupported by any references to CAI's proposal or specific information regarding the firms' past performance.

Superior Initial Proposal

The SSO stated in the source selection statement, under the technical experience of proposed personnel subfactor, that:

"CAI's proposed personnel are characterized as exceeding minimum requirements with no weaknesses identified. This characterization is based upon CAI's initial offering as opposed to ADC's initial offering which resulted in a number of weaknesses related to the qualifications of proposed personnel. This ability to understand and anticipate the needs of [DOE] and respond timely is crucial to meeting the future responsibilities of this office. ADC's weaknesses were resolved during the BAFOs, but demonstrate an initial lack of understanding of the personnel requirements."

Likewise, in responding to ADC's protest, the SSO explained that his determination that CAI's proposal was superior to ADC's, even though ADC received five more points under the technical factor, was based in part upon the fact that in several performance areas ADC had proposed personnel who were not qualified, and that this needed to be discussed and resolved during discussions. According to the SSO, if "ADC truly understood the needs of this Operations Office, this should not have been necessary." The SSO noted that, in contrast, CAI had "submitted a superior original proposal which demonstrated a better understanding of the needs of this Operations Office."

In judging an offeror's understanding of the contract requirements, an agency properly may take into consideration a lack of understanding demonstrated in the initial proposal, even where it subsequently is remedied in the BAFO. See Communications Int'l Inc., B-246076, Feb. 18, 1992, 92-1 CPD Para. 194. However, the record here does not support the SSO's conclusion that ADC's initial understanding was inferior to CAI's. First, CAI's initial proposal received only 14 of 22.5 available points under the technical experience of proposed personnel subfactor, while ADC's proposal received 15 points. In fact, CAI's BAFO still received only 14 points under this subfactor, while ADC's received 16 points.

Second, contrary to the SSO's statement, CAI's initial proposal did have a cited weakness with respect to the technical experience of the proposed personnel. [4] The SSO's conclusion that ADC's initial proposal reflected a weaker understanding than CAI's is inconsistent with the SEB's scoring of ADC's initial proposal as slightly superior to CAI's in this area. Again, although the SSO may disagree with the SEB, Wyle Labs., Inc.; Latacoere Int'l, Inc., supra, the reasons for his disagreement must be set forth in the record. The record does not indicate why the SSO came to different conclusions regarding the two initial proposals.

We conclude that the cost/technical tradeoff is not supported in the record. As a result, there is no basis for concluding that DOE reasonably determined that CAI's proposal provided technical advantages which offset ADC's $1.2 million cost advantage.

CONFLICT OF INTEREST

TRESP and ADC argue that DOE's award to CAI gives rise to an actual or apparent conflict of interest, due to the actions of three DOE officials who were involved in the procurement--Messrs. Keheley, Warner, and Taylor. The protesters allege that the three engaged in, or were influenced by others', employment discussions with Mr. Chew, president and chief executive officer of CAI.

There is no evidence of any impropriety here. First, the record indicates that Mr. Warner was not involved with any aspect of the procurement. Mr. Warner states in his affidavit that he did not participate in, discuss, or have any knowledge of the procurement. Although Mr. Warner was appointed an ex-officio, nonvoting member of the SEB on the basis of his position as a division director affected by the solicitation, the record indicates that he did not know that he had been designated an ex-officio member of the SEB until he was told for the first time on October 26, more than 2 months after contract award. Second, Mr. Keheley's affidavit states that he met with Mr. Chew for the first time on August 24, that is, about 3 weeks after his August 2 selection of CAI for award and that, in any case, he never discussed employment at CAI or received an employment offer. Further, there is nothing to suggest that Mr. Keheley, Mr. Warner's supervisor, may have improperly based his source selection on any knowledge or belief that Mr. Warner was having employment discussions with Mr. Chew. Finally, with respect to Mr. Taylor, there is no evidence that his evaluation may have been improperly influenced by any knowledge or belief that Mr. Warner had employment discussions with Mr. Chew. Further, although Mr. Warner was Mr. Taylor's usual supervisor, during the course of the procurement Mr. Taylor was supervised solely by the SEB chairman. [5] We therefore deny this aspect of the protests.

RECOMMENDATION

We recommend that DOE reevaluate proposals in accordance with this decision and fully document the evaluation and the basis for the resulting source selection decision. In the event an offeror other than CAI is found to be entitled to award, DOE should terminate CAI's contract for the convenience of the government and make award to that offeror. We also find that ADC and TRESP are entitled to reimbursement of the reasonable costs of pursuing their protests, including attorneys' fees. 4 C.F.R. Sec. 21.6(d)(1) (1994). The protesters should submit their certified claims for their protest costs directly to the agency within 60 working days after receipt of this decision.

The protests are sustained in part and denied in part.

Comptroller General of the United States

1. The offerors' overall technical and business/management subtotals were not the sum of the offerors' individual subfactor and criteria scores. Rather, according to the agency, the subtotals were obtained by adding the five individual SEB evaluators' total scores for all of the technical and all of the business/management subfactors and criteria, and then dividing the resulting sums by five. The total scores for the technical and business/management proposals were added together to obtain the offerors' overall total scores.

2. The contracting officer reduced proposed costs based on the recommendations of the Defense Contract Audit Agency.

3. Although one of the SEB members stated that he did not change "most of my scores for TRESP's BAFO," in fact, the record shows that no SEB member changed any score for TRESP's BAFO.

4. While not reflected in the SEB report, we note that the technical evaluation committee (first line evaluators) found two additional weaknesses under the technical experience of proposed personnel subfactor, and the SEB chairman expressed several concerns. For example, he noted that CAI's proposal failed to show that its proposed project manager and physical scientist--satisfied the "PC [personal computer] software experience requirement," and lacked specifics regarding its physical scientist/classification specialist's skills with respect to communicating "class[ification] guidance concepts," as required in the RFP. The chairman also expressed concern that CAI's proposal: (1) failed to show that CAI's proposed environmental engineer had the required "specific experience re[garding] geologic, hydrologic, chemical, and environmental investigations," and experience in conducting site appraisals; (2) did not specifically demonstrate the "technical/project leadership" skills of CAI's proposed general engineer; and (3) did not explain the extent and type of experience of CAI's proposed nuclear engineer with respect to audits, appraisals, and conducting support facility tour activities.

5. DOE conducted an internal investigation in response to the protesters' allegations and found, as we have, that there is no evidence of a conflict of interest.

GAO Contacts

Office of Public Affairs