Skip to main content

Matter of: Communication Technologies, Inc. File: B-283491; B-283491.2 Date: November 30, 1999

B-283491,B-283491.2 Nov 30, 1999
Jump To:
Skip to Highlights

Highlights

DIGEST Protester's contentions that the agency unreasonably evaluated its proposal with an eye toward ensuring that the awardee would prevail (after the source selection authority directed a reevaluation of proposals in certain areas before making his selection decision) is denied where the record shows no evidence of bad faith on the part of the agency evaluators. Where the evaluation was reasonable and in accordance with stated evaluation criteria. Comtek argues that the agency's selection decision was improperly based on unreasonable evaluation conclusions in the areas of past performance and experience. BACKGROUND This procurement is the small business set-aside component of two related service contracts awarded by DISA.

View Decision

Matter of: Communication Technologies, Inc. File: B-283491; B-283491.2 Date: November 30, 1999

DIGEST

Attorneys

DECISION

Communication Technologies, Inc. (Comtek) protests the award of a contract to ARTEL, Inc. by the Defense Information Systems Agency (DISA), pursuant to request for proposals (RFP) No. DCA200-98-R-0060, issued to procure engineering services in support of the Defense Switched Network (DSN), the Defense Information Systems Network (DISN), and other related networks. Comtek argues that the agency's selection decision was improperly based on unreasonable evaluation conclusions in the areas of past performance and experience, and an unreasonable view that the ARTEL proposal offered a better approach to working with DISA's large business network support contractor.

We deny the protest.

BACKGROUND

This procurement is the small business set-aside component of two related service contracts awarded by DISA. The instant contract is referenced by DISA as the DISN Network Management Support Services--Global (DNMSS-G) Associate Support Contract (ASC), hereinafter, the ASC contract; the related large business contract is referenced by DISA as the DNMSS-G Network Engineered Contract (NEC), hereinafter, the NEC contract. Both solicitations were issued on December 1, 1998. The ASC RFP anticipated award of an indefinite-delivery/indefinite-quantity contract with a 2-year base period and three 1-year options. RFP Sec. F-1. The RFP sought engineering, program development, testing, installation, provisioning, and operations and maintenance in support of the DSN, the Defense Red Switch Network (which handles secured communications), and DISA's Metropolitan Area Networks. Agency Report (AR) at 4-5. The RFP contained a $2.5 million minimum purchase guarantee, and a $100 million maximum value. RFP Sec. H-2. In addition, the RFP included the limitation on subcontracting clause at Federal Acquisition Regulation (FAR) Sec. 52.219-14, which requires that at least 50 percent of the cost of performance incurred for personnel shall be for employees of the small business prime contractor. RFP Sec. I, at 1.

The RFP advised offerors that the agency would make award to the offeror whose proposal represented the best value after consideration of three major areas--technical/management, past performance/experience, and cost/price. RFP Sec. M-2. The RFP provided the following guidance on the relative weight of these three areas:

In award selection, "Technical/Management" is more important than "Past Performance/Experience," which is significantly more important than "Price/Cost." "Past Performance/Experience" and "Price/Cost" together are approximately equal to "Technical/Management."

RFP Sec. M-3.d. Under the technical/management evaluation area, the RFP identified several detailed evaluation factors and subfactors; however, these details are not relevant to this decision as there is no challenge to the evaluation under these factors or subfactors.

Under the past performance/experience area--the area that is largely the focus of this protest--the RFP advised that past performance and experience would be "approximately equal in importance." RFP Sec. M-6.a. In preparing their proposals, offerors were required to identify for evaluation between three and five prior contracts similar to the ASC contract. RFP Sec. L-12.c.3. In addition, offerors were required to identify three similar contracts for any of the offeror's major subcontractors (defined as any subcontractor expected to perform at least 10 percent of the work). RFP Secs. L-12.c.1., 3. The RFP also included separate specific instructions about the kind of information that should be provided with respect to the past performance and experience portions of this evaluation area: instructions related to experience were set forth at RFP Sec. L-12.c.5., instructions related to past performance were set forth at RFP Sec. L-12.c.6.

With respect to the evaluation of past performance and experience, the RFP stated for both areas that the agency would evaluate similar prior contracts. RFP Secs. M-6.b., e. In addition, the RFP identified four past performance subfactors of equal weight: quality of service, timeliness of performance, cost/price control, and customer satisfaction. RFP Sec. M-6.c. With respect to the evaluation of experience, the RFP advised that the government did not want to preclude qualified new contractors, and indicated that lack of experience in comparable contracts could be mitigated by superior performance on other contracts, experience of corporate officers or key personnel, or teaming arrangements with experienced contractors. RFP Sec. M-6.f. There were no experience subfactors.

On March 11, 1999, the agency received initial proposals. After an initial evaluation, discussions, and oral presentations, the four offerors remaining in the competitive range, including Comtek and ARTEL, submitted final revised proposals on June 16. After completing the final evaluation, the source selection evaluation board (SSEB) scheduled a briefing for the source selection advisory council (SSAC) and the source selection authority (SSA) on June 30. The events of this meeting, and the reevaluation that followed, are relevant to several of the grounds of this protest. /1/

During the June 30 briefing, the SSA realized that the evaluators had reviewed past performance, but had not reviewed experience, even though experience was to be evaluated separately under the RFP. Declaration of SSA, Oct. 21, 1999, at 2. The SSA concedes that there was some discussion of avoiding a formal evaluation by stating that the two offerors with the highest scores and best prices (ARTEL and Comtek) were approximately equal in this area. Id. Despite this discussion, the SSA states that he directed the SSEB to reconvene, evaluate experience in accordance with the terms of the solicitation, and prepare a written SSEB report. Id. In addition, the SSA notes that while he was aware that the briefing concluded with a recommendation for award to ARTEL, he considered the evaluation incomplete, and did not pay attention to the recommendation. Id.

Upon resolution of the experience scoring issues--the mechanics of which are discussed in greater detail below--the SSEB presented its revised briefing slides to the SSAC on July 28. AR at 8. Using a rating scale of 1 to 10, the SSEB rated ARTEL higher than Comtek in 9 out of 10 evaluated technical/management factors or subfactors. AR at 9. The SSEB combined the scores for past performance and experience--both of which were scored on a scale of 1 to 5--to create a 1 to 10 scale for the past performance/experience area similar to the scale used for the technical/management area. With respect to price, the agency evaluated the discounted life cycle cost (DLCC) of the two proposals, as it had advised it would in the solicitation, RFP Sec. M-7.b., and concluded that ARTEL's DLCC of $39.5 million was approximately 4 percent higher than Comtek's DLCC of $38.0 million. AR at 10. A numerical price score was generated by a formula not at issue in this protest.

To compare the relative merits of the proposals, the SSEB converted the numerical score for each area to a composite overall score by multiplying the score by the relative weight for the area established in the source selection plan (SSP). AR at 11. These weights were: technical/management, 50 percent; past performance/ experience, 30 percent; and price, 20 percent. The resulting overall scores are set forth below:

ARTEL COMTEK

Technical/Management 3.98 3.14 Past Performance/Experience 2.34 1.98 Price 1.23 1.28 TOTAL SCORE 7.54 6.39

AR at 11.

On August 4, the SSAC adopted the SSEB's findings, and identified six areas where ARTEL or Comtek exceeded the government's requirements. After concluding that the proposals were essentially equal in three of the areas, the SSAC found ARTEL superior in the remaining three areas--planning and management approach, commitment to a relationship with the NEC contractor, and risk management. AR at 12. Thus the SSAC recommended award to ARTEL despite its slightly higher price.

On August 5, the SSA adopted the SSAC's tradeoff analysis and concluded that ARTEL's technical proposal demonstrated a superior approach to planning and management issues, a firmer commitment to a working relationship to the DNMSS-G/NEC contractor, and significant benefits in risk management. Furthermore, the SSEB found ARTEL significantly superior to all other offerors in terms of past performance and experience. ARTEL proposed the second lowest price, approximately 4% higher than the next offeror's proposed price. The added benefits of ARTEL's overall proposal are meaningful and of such value as to justify the slightly additional cost.

Source Selection Decision Memorandum, Aug. 5, 1999, at 2.

This protest followed.

DISCUSSION

Comtek's initial protest raised nine separate challenges to DISA's award decision, which were supplemented by three additional challenges. Subsequently, almost every issue raised in the protester's initial filing was expressly withdrawn or abandoned, and the remaining portion of Comtek's protest is largely focused on the evaluation of past performance/experience.

Evaluation of Experience

Comtek argues that the agency's evaluation of experience--performed after the SSA noted that the evaluators had overlooked this area--was unreasonable because the evaluation considered only the experience of the offeror, and not the experience of the offeror's team. Comtek contends that this approach was chosen to ensure that ARTEL would continue to be the recommended awardee. Before turning to the specific issues raised by these arguments, further background about the experience evaluation is set forth below.

After the SSA directed the evaluators to consider experience, as required by the solicitation, the chair of the past performance and experience evaluation team (PPEET) first noted that the agency's internal SSP provided no guidance on evaluating experience (as it, too, apparently overlooked the experience evaluation requirements of the RFP). Declaration of PPEET Chair, Oct. 21, 1999, at 2. After discussing this matter with legal counsel, the PPEET Chair adopted the same numerical/adjectival scale used for evaluating past performance. This scale was as follows:

Exceptional 5 Superior performance, no previous problems

Very Good 4 Better than merely acceptable

Satisfactory 3 Acceptable, requirements were met with minor

Agency resources

Marginal 2 Less than fully acceptable

Unsatisfactory 1 Plainly deficient

SSEB Final Report, July 18, 1999, Attach. 2 (Past Performance and Experience Chair Report), at 3-4.

Upon completion of her review, the PPEET Chair noted that there was a significant difference between the experience scores of the offerors and their proposed subcontractors. PPEET Chair Declaration, supra. The relevant scores for ARTEL and Comtek are set forth below:

ARTEL COMTEK

Offeror Experience Only 3.80 3.00

Subcontractor Experience Only 1.42 2.92

Prime/Sub Experience 50/50 2.61 2.96

Second Agency Report (2nd AR) at 27. Thus, the PPEET Chair explained she was unsure whether to include the subcontractor scores, and if so, to what extent (since different offerors relied to different degrees on subcontracting to supplement their proposals). PPEET Chair Declaration, supra. This "dilemma" arose because, in her view, there was an ambiguity between the RFP's sections L and M. Specifically, she concluded that section L of the RFP suggested the agency would consider contract references for both contractors and major subcontractors, while section M suggested that only the experience of the offeror would be evaluated--unless the offeror lacked experience and needed a review of teaming members to mitigate its inexperience. PPEET Chair Declaration, supra, at 2-3; 2nd AR at 24-26.

In attempting to resolve this dilemma, the PPEET Chair again sought advice from agency legal counsel. DISA's legal counsel agreed that the solicitation was ambiguous, and the PPEET Chair ultimately elected to use the higher of either the offeror's own score, or the combined prime/subcontractor score--which in the case of the protester and the awardee meant the score for offeror experience only. Thus, ARTEL and Comtek received experience scores of 3.80 and 3.00, respectively, rather than 2.61 and 2.96, respectively. 2nd AR at 26-27.

In support of its arguments, Comtek points to several facts that, it contends, show that the evaluators were trying to preserve the same selection decision they originally recommended to the SSA. First, Comtek points to the SSA's admission that the evaluators discussed on June 30 whether to simply state that the two offerors were equal in experience and thereby avoid performing an evaluation. Second, Comtek notes that experience was first evaluated using both the offeror and its major subcontractors--an approach which gave Comtek a higher score than ARTEL for experience. Third, Comtek notes that reviewing only the experience of the offeror itself, rather than its team, is the opposite of the approach taken to review the past performance portion of this area.

The agency responds that Comtek is, in essence, arguing that its evaluators are acting in bad faith. The agency counters that the PPEET Chair reasonably concluded that the solicitation was ambiguous in this area, and reasonably sought legal advice on the most appropriate approach to performing the evaluation given the perceived ambiguity. In addition, DISA argues that its approach is supported by our decision in USATREX Int'l, Inc., B-275592, B-275592.2, Mar. 6, 1997, 98-1 CPD Para. 99 at 4, where we concluded that an agency may properly limit its consideration of experience to the prime offeror's experience under a solicitation reserved for firms qualifying under section 8(a) of the Small Business Act, and including the limitation on subcontracting clause set forth at FAR Sec. 52.219-14--as the solicitation does here. /2/

We agree with the agency that despite the protester's arguments to the contrary, Comtek is essentially arguing that the agency's evaluators, its selection official, and its legal counsel, are all acting in bad faith. See, e.g., Protester's 2nd Comments at 5-6. Without strong evidence to support such a conclusion, we will not assume that agency employees act in bad faith. Indian Affiliates, Inc., B-243420, Aug. 1, 1991, 91-2 CPD Para. 109 at 5. Based on our review of the record, we have no basis to conclude that DISA acted in bad faith here. TMI Servs., Inc., B-276624.2, July 9, 1997, 97-2 CPD Para. 24 at 5.

With respect to the agency's explanation that its first attempt to evaluate experience raised concerns about the fairness of considering subcontractors in this review, we see nothing about those concerns that is unreasonable. As the table above shows, consideration of subcontractor experience lowered the experience scores of both offerors, although there is no doubt that the impact was significantly more adverse for ARTEL than for Comtek. Given the considerations set forth below, we cannot conclude that the agency's reason for expressing this concern was to preserve the award to ARTEL.

In the agency's view there is an ambiguity between sections L and M of the RFP. There is little debate that section L of the RFP suggests that the agency intends to review the experience of both the prime offeror and its subcontractors. As stated above, RFP Sec. L.12.c.3. requires offerors to identify between three and five prior contracts for themselves, and three prior contracts for their major subcontractors; RFP Sec. L.12.c.5.A. provides that "[t]he Government will utilize these descriptions in evaluating contractor/subcontractor experience." It is the paragraphs set forth below that lead to the disagreement about whether there is an ambiguity created by section M:

The Government will evaluate the offerors on their experience in performing contracts of similar size, scope and complexity. Accordingly, it will evaluate the degree of similarity of five required contract references to the current work. The Government considers the performance of five similar contracts (performed currently or within the last three years) to demonstrate adequate previous experience. Therefore, a contractor will not increase its experience evaluation by submitting references to more than five similar contracts.

In considering such relevant experience, the Government does not want to preclude qualified new contractors from participation in this solicitation. Lack of experience in comparable contracts may be mitigated by superior performance on other types of contracts (i.e., smaller or commercial contracts), the experience level of corporate officers/key personnel in the company and/or appropriate teaming arrangements (i.e., the teaming of an experienced contractor(s) with a relatively inexperienced one).

RFP Sec. M-6.e.-f. (emphasis added). The agency reads this language to conclude that only the experience of offerors will be evaluated, absent extenuating circumstances.

In recognition of the ambiguity in the solicitation, the agency attempted to ameliorate the possible unfairness of selecting one approach over another, given that an offeror might have tailored its response to the opposite interpretation. The agency decided to use whichever number was higher for the offeror in question. Thus, both Comtek and ARTEL were evaluated using the score for their experience alone, while one of the other offerors was evaluated using the higher score obtained by considering the contribution of its subcontractors.

We think the solution adopted here--while perhaps not an ideal approach to resolving a solicitation ambiguity--reflects a fair resolution of this issue. If we assume that Comtek crafted its proposal hoping to receive the benefit of both its experience and the experience of its subcontractors--and given that Comtek's primary subcontractor is the incumbent large business contractor that performed this work, we can safely assume that it did--Comtek has received a higher score than it would have received otherwise. Comtek's only complaint is that ARTEL received a greater relative advantage than Comtek from the agency's approach. Given our view that the agency was not unreasonable in its interpretation of section M of the RFP, and given the agency's decision to adopt whichever interpretation yielded the highest score for each offeror, we conclude that Comtek has been treated fairly here, and that there was no attempt to select whichever approach would preclude award to Comtek.

Evaluation of Past Performance

In the area of past performance, Comtek filed a supplemental protest arguing that the agency's evaluation record showed that DISA's past performance scores for ARTEL did not fairly represent the nature of the underlying comments received from references. In fact, comments to this effect were found in the materials developed by the PPEET (and provided with the agency report), wherein for each of the four past performance subfactors, the PPEET included comments like the one below:

Although given numerical scores of 4 and 5, the narrative often describe[s] the job as meeting the [Statement of Work] requirements. There is little description of what ARTEL & FCI did that was more than satisfactory. This discrepancy between the score and the narratives is a weakness on the part of the [contracting officer's representative/contracting officer] . . . .

AR, Tab 13, attach. 17, at 1.

In its second agency report, DISA conceded that there were inconsistencies between some of the narrative comments and the numerical scores awarded by the past performance respondents. To address this, DISA conducted a review of each of the narratives and underlying numerical scores for both Comtek and ARTEL before recalculating the past performance score used in its evaluation. The result of the reevaluation was that ARTEL's past performance score of 4.0 stayed the same, while Comtek's score was increased under each of the four past performance subfactors, raising its past performance score overall from 3.59 to 3.88. 2nd Agency Report at 38. In response, Comtek argues that the PPEET Chair's approach to recalculating the past performance scores was unreasonable, and identifies three separate instances where it alleges that the review of narratives and underlying scores unfairly favored ARTEL.

Comtek argues that the agency's approach to recalculating the past performance scores is unreasonable because it allegedly differs from the approach used in the initial evaluation. Specifically, Comtek notes that the initial approach relied upon averaging scores given by the PPEET Chair and her two subordinate evaluators, while the reevaluation was performed using a percentage-based approach. Thus, Comtek complains that this approach must have been adopted to ensure that ARTEL would continue to receive a higher score. Comtek argues that if the scores were averaged, it would have prevailed in this area with a rating of 4.19 to ARTEL's 4.16--rather than the 3.88 and 4.0 ratings received from the reevaluation performed by the agency.

We disagree with Comtek's assertions, and the premises upon which they are based. The record here shows that the agency's review of ARTEL's score in this area considered 54 potential responses for each of the four subfactors in the past performance area. 2nd AR at 36. To reevaluate these responses, the PPEET Chair calculated the percentage of each of the five possible numerical scores that were assigned under each subfactor. A table showing how the responses were reviewed for ARTEL, under each past performance subfactor, is set forth below:

Quality Timeliness Cost Control Customer Satisfaction

5 - Outstanding 38% 38% 31% 49%

4 - Very Good 38% 36% 39% 38%

3 - Satisfactory 25% 25% 27% 13%

2 - Marginal 0 2% 2% 0

1 - Unacceptable 0 0 0 0

2nd AR, attach. 20. By reviewing the range of responses set forth above, the PPEET Chair (acting alone now, as the two other PPEET evaluators were no longer available for this procurement) assigned a rating of 1 to 5 based on where the greatest concentration of responses occurred.

Comtek's claim that scores were initially averaged is only partially accurate. In the earlier evaluation the scores assigned by the three PPEET evaluators were averaged; in the subsequent evaluation, the two PPEET evaluators were no longer available and the PPEET chair was performing the evaluation alone. Thus, averaging the scores of multiple evaluators was no longer an issue. On the other hand, the averaging of the scores awarded by references responding to the past performance questionnaire, which Comtek also seeks, was never done. In our view, there is nothing about the approach described above that is inherently unreasonable, and the approach is not rendered unreasonable simply because it is different from the approach taken initially. In short and again, there is no basis for our Office to conclude that DISA has chosen this approach unreasonably or to inflict harm on Comtek. /3/

We also conclude that Comtek has failed to show how it was prejudiced as a result of the allegedly unfair review of the narrative responses and numerical scores received from the underlying references. In this regard, all of the evaluation materials associated with the agency reassessment of past performance were provided to Comtek under a protective order, yet Comtek raises only three instances of judgments that were allegedly unfair. /4/ Two of the three adjustments involved an increase in the underlying response score for ARTEL; however, the record shows that the agency ultimately made no adjustment to ARTEL's score in the past performance area. Thus, these adjustments had no impact on the evaluation here. The third challenge involves a situation where the agency elected not to increase the score of one of Comtek's subcontractors on one response under one past performance evaluation subfactor. In our view, and considering the fact that Comtek makes no attempt to show that increasing its score for this one of 22 responses under one of 4 past performance subfactors would change its score in the past performance area--and our review of the record strongly suggests it would not--we see no basis to conclude that Comtek was prejudiced by the agency's reevaluation.

Evaluation of the Relationship with the NEC Contractor

In a final remaining challenge, Comtek argues that the SSA erred when he concluded that the ARTEL proposal offered a "a firmer commitment to a working relationship to the [NEC] contractor." Soure Selection Decision Memorandum, supra, at 2. The contractor's ability to work with the large business NEC contractor was one of the management evaluation subfactors identified in the solicitation. RFP Sec. M-4.d.4.C. In Comtek's view, this conclusion was irrational since during the course of the evaluation here, Comtek's major subcontractor, GTE, was selected to be the NEC contractor. Since Comtek and GTE have worked together before, are offering to do so again, and already have in place a non-disclosure agreement related to their current proposal, Comtek contends that using this conclusion as a discriminator between the proposals is irrational.

We have reviewed all of Comtek's allegations, the underlying evaluation materials, and the agency's arguments, and we disagree with Comtek's assertions in this area. First, the agency argues, and our review of the materials shows, that ARTEL's proposal was more thorough in this area. In addition, the agency's review was based on the representations and approaches made in the proposal--especially since during most of the time the agency was evaluating these proposals, GTE had not yet been selected for award. Finally, we also agree with the agency's view that Comtek's agreements with GTE about their relationship as prime contractor and subcontractor for the ASC contract may not necessarily translate to their relationship as prime contractors for the ASC and NEC efforts. As a result, we see nothing unreasonable about the SSA's conclusion that ARTEL's proposal's commitment in this area was preferable to the commitments found in Comtek's proposal.

The protest is denied.

Comptroller General of the United States

1. Since the record shows that the SSEB, SSAC, and SSA all ultimately agreed that ARTEL and Comtek offered the most advantageous proposals--the other two offerors had lower scores and higher prices--this decision focuses only on ARTEL and Comtek.

2. Section 8(a) of the Small Business Act authorizes the Small Business Administration to enter into contracts with government agencies and to arrange for performance through subcontracts with socially and economically disadvantaged business concerns. 15 U.S.C. Sec. 637(a) (1994). For purposes of our consideration here, we see no importance in the fact that USATREX involved an 8(a) set-aside, while the procurement here is reserved for any small business, regardless of whether it qualifies as a socially disadvantaged business under the 8(a) program.

3. In addition, the corrected scores Comtek argues would have resulted from the use of averaging the past performance responses have a clearly de minimis impact on the overall proposal scores assigned here. Specifically, if we use the 4.19 and 4.16 past performance scores that Comtek argues it and ARTEL should receive, respectively, add those scores to the 3.0 and 3.8 experience scores received, and apply the 30 percent weight used for this area, the corrected past performance/experience scores for Comtek and ARTEL are 2.16 and 2.39 respectively. Substituting these scores in the overall proposal score formula, Comtek's and ARTEL's scores both increase--from 6.39 and 7.54, to 6.58 and 7.60, respectively. Thus, even if Comtek's challenges were upheld, the minimal change in scores would not warrant a new cost/technical tradeoff. Advanced Data Concepts, Inc., B-277801.4, June 1, 1998, 98-1 CPD Para. 145 at 4-5.

4. As stated above, the agency reviewed 54 such responses for ARTEL's reevaluation; the corresponding number for Comtek was 22 responses. 2nd AR at 36-37. If the materials had displayed other examples of unfairness, Comtek was in a position to raise them. Since it did not, we assume there were no more.

GAO Contacts

Office of Public Affairs