Intelligent Decisions, Inc.; Abacus Technology Corporation; D&S Consultants, Inc.; CDO Technologies, Inc.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Intelligent Decisions, Inc.; Abacus Technology Corporation; D&S Consultants, Inc.; CDO Technologies, Inc.

File: B-409686; B-409686.2; B-409686.3; B-409686.5; B-409686.6; B‑409686.12; B-409686.14

Date: July 15, 2014

DIGEST

Protests are sustained where record shows that agency’s evaluation and source selection decisions fail to conform to the solicitation’s evaluation scheme and basis for award.

Intelligent Decisions, Inc. (IDI), of Ashburn, Virginia; Abacus Technology Corporation, of Chevy Chase, Maryland; D&S Consultants, Inc., of Eatontown, New Jersey; and CDO Technologies, Inc., of Dayton, Ohio, protest the award of multiple contracts by the Department of the Air Force under request for proposals (RFP) No. FA8771-09-R-0019, issued to acquire a wide array of information technology services and products. The protesters maintain that the agency misevaluated proposals, failed to engage in adequate discussions, and made unreasonable source selection decisions.

We sustain the protests in part, deny them in part, and dismiss them in part.

BACKGROUND

The RFP, issued September 28, 2011, solicited proposals to provide network operations, infrastructure and service oriented architecture information and transformation services and solutions to the Air Force and other Department of Defense agencies at locations worldwide. The overarching acquisition commonly is referred to as Network Centric Solutions-2 (NETCENTS-2), and these protests concern the award of contracts to small business concerns under the small business set-aside portion of the acquisition. The RFP anticipates the award of multiple indefinite-delivery/indefinite-quantity (IDIQ) contracts that include cost reimbursable elements, fixed-price elements, and labor hour elements. RFP at 2-33.[1] The RFP contemplates a 3-year base period, and up to four 1-year option periods, during which the agency can award one or more task orders to the successful contractors. Id. Each awarded contract has a minimum guaranteed value of $2,500; the maximum value of the contracts awarded under the small business set-aside portion of the agency’s acquisition is $5.7 billion. RFP at 36.

The agency was to award contracts on a “best value” basis using a performance price tradeoff procedure. RFP at 223. The agency anticipated awarding between six and nine contracts, but reserved the right to make more, fewer, or no awards. Id. at 224. Firms were advised that the agency would first evaluate proposals for technical acceptability on a pass/fail basis.[2] The agency was to perform a performance confidence assessment and cost/price evaluation on the proposals found technically acceptable or reasonably susceptible of being made technically acceptable. Id. Award decisions were to be based on an integrated assessment of performance confidence and cost/price; performance confidence was significantly more important than price. Id. at 223.

For the performance confidence assessment, the RFP required the agency to perform a detailed and in-depth assessment for each offeror by considering the quality of recent, relevant contract efforts (either government or commercial) performed either by the offeror or the offeror’s significant subcontractors or teaming partners. RFP at 226. The agency was to assign each concern an overall performance confidence rating of substantial confidence, satisfactory confidence, limited confidence, no confidence or unknown confidence. Id. at 227.

Proposals could include up to 10 past performance examples. Id. at 214. Each example had to be described in an individual past performance information sheet (PPIS), and each PPIS had to be prepared in accordance with a form included with the RFP. The RFP further advised that each PPIS could encompass only one stand-alone contract, or a single task order under an overarching IDIQ contract.[3] Id. Firms also were instructed to send out past performance questionnaires (PPQs) to the point of contact identified in each PPIS. These questionnaires also were embodied in a form included with the RFP. If the example was one where the offeror performed as the prime contractor, the PPQ had to be completed by the individual responsible for the contract (a government contracting officer or contracting officer technical representative, a quality assurance official or evaluator, or their civilian equivalent in the case of a commercial contract). Id. at 215. If the example was one where the offeror performed as a subcontractor, the PPQ had to be completed by the entity awarding the subcontract. Id.

The agency was to evaluate each performance example for relevancy[4] and to assign ratings of highly relevant, relevant, somewhat relevant, not relevant or not applicable for each performance example. Id. at 228. Relevancy ratings would be assigned in each of five areas: metadata environments, network management and network defense (NM/ND), network management and enterprise services (NM/ES), information transport systems (ITS), management, and cost. Id. at 228-229. Each specific area also had detailed sub-elements that would be considered in evaluating relevancy. Id.

The RFP stated that more recent and relevant performance examples would have a greater impact on the performance confidence assessment, and a more relevant past record of favorable performance could receive a higher confidence rating and be considered more favorably than a less relevant record of favorable performance. RFP at 227. Efforts demonstrating a “sustained track record of performance” (defined as contracts performed for at least one year), also would have a greater impact on the agency’s overall performance confidence assessment. Id. The RFP also advised that performance examples that covered more relevancy criteria were preferred to those that covered fewer relevancy criteria. Id. Finally, the RFP stated that offerors should reference examples that were performed worldwide to have the greatest impact on the performance confidence assessment; while other references would be accepted, worldwide examples were preferred. Id.

For purposes of evaluating quality, the RFP stated that the agency would evaluate the offerors’ PPISs and the PPQs received, and also would consider a variety of other sources of information such as contractor performance assessment reports (CPARs) information obtained from the past performance information retrieval system (PPIRS), and interviews with cognizant government officials, Defense Contract Management Agency officials, and commercial clients. RFP at 229-230. The agency planned to assign overall performance ratings of exceptional, very good, satisfactory, marginal, unsatisfactory or not applicable. Id. at 230.

Offerors were required to calculate a total evaluated price (TEP) comprised of their cost/price responses to two sample tasks, along with their calculated total price from a maximum labor rates best estimated quantities worksheet. Id. at 217-218. For evaluation purposes, the agency was to consider the reasonableness of the three components of the TEP (the two task order responses, along with the labor rates best estimated quantities worksheet), and the realism of the offerors’ proposed costs for the cost reimbursable task order. Id. at 231.

The agency received 29 proposals in response to the solicitation, two of which were subsequently withdrawn. The agency evaluated proposals, engaged in discussions, and solicited, obtained and evaluated final proposal revisions (FPRs). On the basis of that evaluation, the agency found five proposals technically unacceptable and eliminated them from further consideration. Of the remaining proposals, 12 received substantial performance confidence ratings, and all four protesters (as well as some of the remaining offerors) received satisfactory ratings. The relevant results, along with the offerors’ TEPs, are as follows[5]:

Offeror	Confidence Rating	TEP
Smartronix, Inc.	Substantial	$66,986,539.39
STG, Inc.	Substantial	$67,578,894.33
Abacus Technology Corp.	Satisfactory	$68,398,356.29
American Systems Corp.	Substantial	$68,927,268.85
Technica Corp.	Substantial	$76,133,994.52
Epsilon Systems Solutions, Inc.	Substantial	$76,513,524.43
Telos Corp.	Substantial	$79,821,882.41
SMS Data Products Group, Inc.	Substantial	$80,877,407.44
The Centech Group, Inc.	Substantial	$89,880,054.66
D&S Consultants, Inc.	Satisfactory	$95,937,030.59
Intelligent Decisions, Inc.	Satisfactory	$103,932,597.56
Sumaria Systems, Inc.	Substantial	$104,187,337.52
BTAS, Inc.	Substantial	$108,985,760.86
MicroTech	Substantial	$120,893,859.66
CDO Tech., Inc.	Satisfactory	$121,931,363.13
Indus Corp.	Substantial	$130,136,809.59

Source Selection Decision Document (SSDD), at 6-7.[6]

On the basis of these evaluation results, the agency awarded contracts to the 12 firms with substantial performance confidence ratings. These protests followed.

The protesters have raised numerous allegations which we have carefully considered. We sustain the protests of Abacus and D&S for the reasons discussed below, and deny or dismiss their remaining allegations whether or not we specifically address them. IDI and CDO raised different allegations, and we deny their protests for the reasons discussed below. At the outset, in reviewing protests challenging an agency’s evaluation of proposals, our Office does not independently evaluate proposals; rather, we review the agency’s evaluation to ensure that it is consistent with the terms of the solicitation and applicable statutes and regulations. Exelis Sys. Corp., B‑407111.5 et al., May 20, 2013, 2013 CPD ¶ 123 at 6.

EXCLUDED PAST PERFORMANCE EXAMPLES

IDI, CDO and Abacus all raise challenges to the Air Force’s decision to exclude from consideration certain of their past performance examples during the performance confidence evaluation. As discussed below, ultimately we recommend in this decision that the agency perform a new performance confidence evaluation. We address each of the protesters’ challenged exclusions because the agency will need to know whether each example should, or should not, be considered during the recommended reevaluation.

IDI

The record shows that the agency excluded from consideration one of IDI’s past performance examples--the so-called BSM-E contract--because the agency did not receive a PPQ for that contract and, thus, was not able to verify IDI’s performance. IDI argues that the agency erred in not expressly advising it of the fact that it had not received the PPQ, and failed to use any other avenues to obtain additional information (for example, the CPARS or PPIRS systems).

We find no merit to IDI’s protest. The record shows that the agency’s decision not to evaluate this example was principally attributable to IDI’s actions. In this regard, IDI’s initial proposal included a PPIS for the BSM-E contract and described it as a stand-alone contract under which IDI was a prime contractor to the Defense Logistics Agency (DLA). Agency Report (AR), exh. 3, 3.INT.016_V3-PCAG-INT_A6_INT_INTVolIIIPP, at 7.[7] However, the PPIS also specifically referred to the fact that there were two contracts performed during the period, one described as the “previous contract” and one described as a “follow-on contract.” Id. During discussions, the agency advised IDI that it had not evaluated this effort because IDI had not identified a single contract. The agency expressly stated that IDI could only include information on its PPIS for either the original contract or the follow-on contract. AR, exh. 8.INT.003_Q&A-INT, at 1.

In response, IDI revised its PPIS for this reference. In the revised PPIS, IDI changed the period of performance for this requirement (along with its dollar value); represented that it was a stand-alone subcontract (and, thus, not a prime contract with the government, as initially represented); and identified its prime contractor as Regent Systems, Inc. AR, exh. 3, 3.INT.018_V3-PCAG-INT_FPR_INT_INTVolIIIPP, at 5. However, the revised PPIS also listed IDI’s own employee as the program manager/site manager, listed Defense Information Systems Agency (DISA) employees as the contracting officer and administrative contracting officer respectively, and provided no contact information for anyone at Regent Systems, Inc., the firm’s prime contractor. Id.

The contracting officer explains that the agency did not receive a PPQ for the revised past performance example included in IDI’s FPR. Contracting Officer Statement of Facts (COSF), at 18-20. She further explains that the agency tried on two separate occasions to reach out to IDI’s listed point of contact (the DISA contracting officer listed in the PPIS), but received no response. Id.

The RFP specifically stated that “[t]he Government will not use in its assessment a past/present contract where performance cannot be verified through the customer of that contract.” RFP at 227. As noted above, the RFP also advised offerors that, in situations where the offeror was a subcontractor, the entity awarding the subcontract was required to complete the PPQ. Id. at 215. Here, because the Air Force did not receive a PPQ for IDI’s revised past performance example, and because IDI incorrectly identified the point of contact in its revised PPIS, the agency had no way to verify IDI’s performance for this example, despite its best efforts to do so.[8] Consequently, and in accordance with the terms of the RFP, the agency properly excluded this example during its performance confidence assessment.[9]

IDI argues that the Air Force should have advised it that it had not received a PPQ for its revised reference.[10] In support of its position, IDI directs our attention to a question and answer in RFP amendment No. 6, which stated that the agency would advise offerors that it had not received a PPQ for a given contract. AR, exh. 2, 2.077_ NOSB Amend 0006 QandA, Question No. 81, at 34. However, that question and answer only obligated the agency to advise offerors when a PPQ was not received in connection with a past performance example included in its original proposal. Id. Here, as noted, IDI changed its PPIS for this reference between its original and final proposals. Consequently, the agency was under no obligation to advise IDI that it had not received a PPQ for this example.

Next, IDI maintains that the agency should have advised it that its PPIS point of contact was erroneous and afforded the firm an opportunity to revise its PPIS to provide another contact. According to IDI, this would have amounted to a clarification of its proposal.

We disagree. As discussed above, the RFP expressly provided that the agency would only consider a past performance example where it could be verified by the customer. The RFP also expressly advised that, in the case of a subcontract, the entity awarding the subcontract was required to complete a PPQ for the example. Because IDI’s customer did not return a PPQ, the agency was placed in the position of attempting independently to verify IDI’s performance using information in IDI’s proposal. The proposal did not merely identify an incorrect point of contact, it identified the wrong organization to contact. As a consequence, the agency was unable to perform an independent verification. It follows that the information in question--the correct organization to contact to verify IDI’s performance--was a material element of the IDI proposal insofar as this past performance example was concerned. For the agency to have inquired of IDI concerning this question would have constituted discussions because it would have allowed IDI to make a material revision to its proposal. Federal Acquisition Regulation (FAR) § 15.306(d). The agency was under no obligation to reopen discussions with IDI after the submission of FPRs in order to allow it to revise its proposal.

In the final analysis, IDI failed to include the correct information in its PPIS. Since an agency’s evaluation is dependent on the information included in a proposal, it is the offeror’s responsibility to submit an adequately written proposal; this includes adequate information relating the offeror’s past performance. See Moura’s Cleaning Serv., Inc., B‑402741.4, Sept. 7, 2010, 2010 CPD ¶ 210 at 3. An offeror failing to submit an adequately written proposal runs the risk that its proposal will be evaluated unfavorably. Id.

Finally, IDI maintains that the agency’s overall performance confidence evaluation of its proposal included several factual inaccuracies and generally was not consistent with the requirements of the RFP. Because we recommend below that the agency perform a new performance confidence evaluation, we view these allegations as academic, since the agency necessarily will reach new evaluation results. We therefore dismiss this aspect of IDI’s protest.[11]

CDO

CDO challenges the agency’s decision not to consider two of its past performance examples.[12] The first of these examples is referred to by CDO as citation No. 6. The agency excluded citation No. 6 from consideration because CDO listed two different contracts under this past performance example, an original contract and a follow-on bridge contract. AR, exh. 3, CDO.015_V3 - PCAG - CDO_Initial_CDO_cdo_NetOpsSB_Proposal_Volume_III_PastPerformance_final, at 23 (CDO’s original past performance proposal). During discussions, the agency advised CDO that it had not evaluated citation No. 6 because CDO listed more than one contract in its PPIS. Notwithstanding this specific advice, CDO included both contract references in its FPR. AR, exh. 3, CDO.014_V3 - PCAG - CDO_FPR_CDO_CDO_VOLIII_PP, at 23 (CDO’s FPR). The agency therefore excluded citation No. 6 from consideration.

CDO maintains that the original contract and bridge contract should have been considered essentially as one contract because it was for an ongoing effort. However, this circumstance is no different than any other situation in which a contractor has performed the same ongoing requirement under more than one contract. As noted above, the RFP specifically provided that the agency would only evaluate a past performance example that included either a single, stand-alone contract, or a single task order under an overarching larger IDIQ contract. RFP at 214. Because CDO included more than one contract for citation No. 6, the agency properly excluded this citation during its evaluation.

CDO alleges that it was not afforded meaningful discussions in connection with citation No. 6 because the agency did not expressly advise it that it considered the original and bridge contract to be separate contracts. However, the agency advised CDO that citation No. 6 was not evaluated because it identified more than one contract. This was adequate to lead CDO into the area of the agency’s concern, and the agency was under no obligation to be more specific. CEdge Software Consultants LLC, B‑408203, July 19, 2013, 2013 CPD ¶ 177 at 7 (all-encompassing discussions are not required, nor is the agency obligated to “spoon-feed” an offeror as to each and every item that could be revised to improve its proposal).

CDO also contends that the agency improperly excluded from consideration what it refers to as citation No. 2. The record shows that CDO included this contract in its initial proposal, AR, exh. 3, CDO.015_V3 - PCAG -CDO_Initial_CDO_cdo_NetOpsSB_Proposal_Volume_III_Past Performance_final, at 8, but the agency excluded it from consideration because this was an IDIQ contract and CDO failed to identify a specific task order number for evaluation purposes. The record shows that CDO specifically was advised of this fact during discussions. AR, exh. 6, CDO.0002_ENs, at 20. The record also shows that CDO was advised prior to formal discussions that the agency had not received a PPQ for this contract. AR, exh. 4, CDO.001_CDO_Missing PPQs. CDO revised its proposal to include a specific task order number for this contract reference, but the agency still did not receive a PPQ for this effort. The Air Force therefore excluded this past performance example from consideration.[13]

CDO argues that the agency failed to engage in meaningful discussions with the firm because it was not advised of the missing PPQ. According to CDO, when the agency advised it during formal discussions that this contract was not evaluated because it was lacking a task order number, CDO believed that the agency’s earlier, informal query concerning the lack of a PPQ had been satisfied. CDO therefore maintains that it was misled during formal discussions into believing that it did not need to further address the lack of a PPQ for this effort.

We find no merit to this aspect of CDO’s protest. The record shows that at the time of discussions, CDO had not yet identified the specific task order under its IDIQ contract that it wanted the agency to evaluate. Thus, neither CDO, nor any potential reviewing official, knew the task order for which the PPQ was required. It follows that there would have been no logical reason for CDO to have assumed that a PPQ had been submitted. Moreover, even if a PPQ previously had been submitted, there would have been no logical reason for CDO to believe that the PPQ actually pertained to the task order ultimately identified.

In any event, once the task order had been identified by CDO, the firm was under an affirmative, continuing duty to ensure that a PPQ from the appropriate individual was sent to the agency. In this connection, the agency’s letter to CDO requesting its FPR specifically advised the protester as follows:

An Offeror who wishes to update Volume Ill-Past Performance, may revise or replace any or all of the citations.

* * * * *

When replacing a citation, a Past Performance Questionnaire (PPQ) should be requested following the procedures in paragraph L5.1.1.3 of the RFP. Questionnaires are due the same date and time as the FPR. If the PPQ is not received, the Government may not be able to validate the information provided in the citation.

AR, exh. 22, 22.013_NOSB_FPR_Letter, at 3. Since CDO effectively revised its proposal in connection with citation No. 2, the agency’s letter requesting CDO’s FPR adequately communicated the need to ensure that a PPQ was sent in for this past performance example by the time it submitted its FPR. We therefore conclude that CDO could not reasonably have been misled by the agency’s discussion question. In light of the foregoing, we deny CDO’s protest.

Abacus

Abacus asserts that the agency improperly excluded two of its past performance examples. The first of these is referred to as the Kirtland Air Force Base contract. The record shows that the agency excluded this past performance example from consideration because it concluded that this was an IDIQ contract for which Abacus had failed to identify a task order number.

Abacus maintains that the agency’s actions were improper because, in fact, this was not an IDIQ contract. According to Abacus, the agency irrationally relied on the contract number--which included the letter “D”--to conclude that this was an IDIQ contract.[14] Abacus concedes that this contract has a single, minor, IDIQ contract line item (under which unanticipated goods and services above and beyond the base contract’s requirements could be ordered), but asserts that the bulk of the work was performed as a stand-alone, fixed price contract. Abacus points out that it conveyed these circumstances to the agency in its proposal and during discussions, but that the agency continued to exclude this contract from consideration.

On the record before our Office, we find that the agency unreasonably excluded this contract from consideration during its evaluation. The record shows that Abacus specifically identified this contract as a stand-alone, fixed-price contract in its original proposal. AR, exh. 3, ABA.015_V3 - PCAG - ABA_A6_ABA_ORIGINAL PROPOSAL_ABA V3-ORIGINAL, at 10. Notwithstanding this designation, the agency concluded during its initial evaluation that this was an IDIQ contract, under which Abacus was required to identify a task order. Accordingly, the agency raised the matter with Abacus during discussions. In response, Abacus stated as follows:

Abacus Technology is fully confident that based upon the terms and conditions of the Kirtland Contract No. FA9401-05-D-0001, it is not an IDIQ as defined under Subpart 16.5 of FAR and moreover does not fall under any of the three IDIQ types set forth under 16.501-16.504 of FAR. Rather, it is a firm fixed price contract with a defined work scope pursuant to which Abacus Technology performs C4 services/operations and other telecommunications functions critical to Kirtland AFB. Abacus is required to perform discrete work activities based on specified support requirements, performance metrics, and service quantities as well as scheduled deliverables as set forth in the contract. However, two delivery orders were issued for a short period to support organizations that did not exist at time of contract award.

Accordingly, we have reviewed and updated this citation to ensure it only addresses Abacus' base level C4 support for Kirtland AFB.

AR, exh. 9, ABA.002_ENs, at 8. Consistent with its response to the agency’s discussion question, Abacus revised its proposal, adding essentially this same explanation to its PPIS for this contract. AR, exh. 3, ABA.017_V3 - PCAG - ABA_FPR_ABA_ABA_V3 FPR, at 10.

The record also shows that the agency contacted Abacus’s point of contact for this contract to inquire about whether this was an IDIQ contract. The agency’s point of contact represented as follows:

Although FA9401‐05‐D‐0001 is an IDIQ contract, there were two product orders placed against it. However, instead of issuing a task order for the C4 Services currently being provided at Kirtland, the work is being provided under the umbrella IDIQ contract as if it were a stand alone contract. The Past Performance Questionnaire (PPQ) provided is based on the C4 services only.

AR, exh. 10, ABA.004_ABA EMAIL_ABA-02 Email from PPQ Respondent, dated 20110131.[15]

In addition to this information, the record includes two PPQs submitted in connection with this contract that identify it as a stand-alone, fixed-price contract, AR, exh. 10, ABA.024_ABA ppq_02-ABA-FA940105D0001 10 Nov 2010 File 1; AR, exh. 10, ABA.025_ABA ppq_02-ABA-FA940105D0001 10 Nov 2010 File 2. The record also includes three CPARs that also identify the contract as a fixed-price contract. See generally, AR, exh. 10.

In view of the totality of the information, as well as the evidence presented above, we conclude that it was unreasonable for the agency to have excluded consideration of the Kirtland contract from consideration based on the Air Force’s conclusion that it was an IDIQ contract. Although the Air Force is technically correct that the contract number identifies this as an IDIQ contract, all of the evidence shows that both parties to the contract treated it as a stand-alone, fixed-price contract under which Abacus provided services without the issuance of discrete task orders. We therefore sustain this aspect of Abacus’s protest.

The record also shows that the agency excluded from consideration a task order performed by one of Abacus’s subcontractors; Abacus refers to this as the DGS task order. The record shows that this task order was introduced into the Abacus proposal for the first time in its FPR. AR, exh. 3, ABA.017_V3 – PCAG-ABA_FPR_ABA_ABA_V3 FPR, at 42-45. The PPIS identifies a contract number, and specifically identifies task order No. 36. A PPIS note stated:

TO [task order] 36 is the first in a series of task orders (followed by TO 43, 44, 46, and 47) that covered the period of 6/28/2009-3/30/2014, each involving the execution of the same network implementation requirements for DISA. . . . However, only the performance of TO 36 support services are described below.

Id. at 42.

The record shows that the agency did not receive a PPQ for this task order. As a result, the agency contacted one of the contacts in the PPIS.[16] The record includes an e-mail exchange between the point of contact and the agency, during which the point of contact states that he did not have a task order 36 for that contract; he asked the Air Force to provide information about the work involved. AR, exh. 10, ABA.045_RE_ Past Performance on [deleted](1). Based on the agency’s input regarding the substantive work performed, the point of contact provided a PPQ for task order 46 rather than for task order 36. Because the agency was unable to verify the subcontractor’s performance of task order 36 with the customer, the agency excluded this past performance example from its evaluation.

Abacus argues that it was unreasonable for the agency not to have considered its subcontractor’s performance of the task order. The protester principally maintains that the agency should have contacted it in order to clarify the fact that the task order at issue was task order 36 rather than task order 46.

We deny this aspect of Abacus’s protest. As an initial matter, we note that it was the offerors’ responsibility in the first instance to ensure that PPQs were sent to the agency from their customers. The record is silent regarding the steps taken by Abacus (or its subcontractor) to ensure that any effort was made to have the customer prepare and submit the PPQ for this contract. In fact, Abacus does not even represent that it or its subcontractor sent the PPQ to the customer for submission to the Air Force.

Further, the apparent confusion regarding the effort being reviewed (task order 36 rather than task order 46) was not a point of confusion on the part of Air Force personnel but, rather, arose because one of the points of contact included in the PPIS did not believe there was a task order 36. The Air Force request to that individual specifically referenced task order 36, stating: “The cited effort was contract number [deleted] Task Order 36. The period of performance was 06/28/2009 to 08/15/2010.” AR, exh. 10, ABA.002_Past Performance on [deleted] (e-mail dated May 18, 2013).

Finally, there is no reason to conclude that matters could have been resolved by the Air Force contacting Abacus after the PPQ for task order 46 was provided to the agency. Once again, there was no confusion on the part of the Air Force regarding which task order--task order 36--was being offered for evaluation. The confusion arose because the point of contact provided by the protester erroneously did not believe there was a task order 36.

We conclude that the Air Force’s exclusion of this task order from consideration is the responsibility of Abacus in the first instance, since nothing in the record shows what efforts the protester made to ensure that a PPQ was provided to the Air Force by its customer. Further, the record does not evidence confusion on the part of the Air Force regarding the task order offered for consideration. On the contrary, the error arose because of a misunderstanding on the part of Abacus’s point of contact. While the error is unfortunate and resulted in the exclusion from consideration of a past performance example that Abacus considers important, nothing in the record leads us to find the agency’s actions unreasonable.

PERFORMANCE CONFIDENCE EVALUATION

D&S argues that the agency’s performance confidence evaluation--and by extension its source selection decisions--failed to consider all of the elements the RFP stated would be evaluated. In this connection, D&S maintains that the agency’s performance confidence evaluation focused almost entirely on the question whether a particular contract was relevant under one or more of the specified areas of consideration. D&S argues that the record shows it was principally the relevancy ratings that dictated the ultimate performance confidence ratings which, in turn, drove the source selection decisions.

Specifically, D&S asserts that the agency failed meaningfully to factor in the other considerations enumerated in the RFP for evaluation during the performance confidence evaluation, including the quality ratings that were assigned to the past performance examples, whether the contracts demonstrated a “sustained track record of performance,” the magnitude of the effort being reviewed (expressed as the dollar value of the example), and whether the example was performed on a worldwide basis. The Air Force essentially disagrees with D&S’s allegation, arguing that it performed what it describes as an integrated performance confidence evaluation that took into consideration all of the RFP’s enumerated considerations.

We have carefully reviewed the record and, as set forth below, conclude that D&S is correct that the agency’s performance confidence evaluation was driven principally by considerations of relevance, and failed meaningfully to consider all of the elements enumerated in the RFP. In reviewing a protest of an agency’s evaluation of proposals, our Office will examine the record to determine whether the agency’s judgment was reasonable and consistent with the stated evaluation criteria and applicable procurement statutes and regulations. Shumaker Trucking & Excavating Contractors, Inc., B-290732, Sept. 25, 2002, 2002 CPD ¶ 169 at 3. While we will not substitute our judgment for that of the agency, we will sustain a protest where the agency’s conclusions are inconsistent with the solicitation’s evaluation criteria, undocumented, or not reasonably based. DRS ICAS, LLC, B-401852.4, B‑401852.5, Sept. 8, 2010, 2010 CPD ¶ 261 at 4-5.

The record includes consensus evaluation reports prepared by the agency’s performance confidence assessment group (PCAG) for each offeror.[17] These consensus reports are comprised of a collection of PCAG evaluation forms that were filled out for each past performance example. A review of the PCAG reports shows that the agency’s overwhelming concern related to the relevancy of the past performance examples, with little or no emphasis on the quality of performance, and virtually no consideration of the place of performance, whether the example demonstrated a sustained track record of performance, or the magnitude of the example in relation to the contracts being awarded.

Even the format of the forms highlights the agency’s focus on the relevancy of the past performance examples. Each form includes a brief listing of information at the top, including identifying information relating to the contract being reviewed, the start and finish dates of the contract, the dollar value of the contract, the place of performance, and several places to identify the contract type (for example, stand-alone or task order, prime contract or subcontract), along with information relating to what information was reviewed by the PCAG (for example, PPQs or CPAR reports). The balance of the form is divided among the areas of relevancy identified in the RFP (metadata environments, NM/ND, NM/ES, ITS, management, and cost), along with a listing of the numerous sub-elements for each area of relevance.

Turning to the substance of the forms, for each area of relevancy there was a space to enter the agency’s relevancy rating, a space to enter the agency’s quality rating and a reference to the customer’s quality rating. However, all of the narrative materials entered by the PCAG on these forms relate solely to the question of whether the performance example being reviewed was, or was not, relevant.[18] For example, with respect to the NM/ES area under review for one of D&S’s contracts, the entire PCAG report provides:

NETWORK MANAGEMENT AND ENTERPRISE SERVICES

Relevancy: SOMEWHAT RELEVANT Quality: EXCEPTIONAL

Comments:

Customer rated all relevant areas exceptional.

a. [E1] Provided services and solutions to accomplish network boundary management and control such as that needed to establish and secure a corporate intranet.

They installed and configured the Army Command and Control Protect Tools and a variety of firewalls & provides vulnerability assessments on USFK information systems and recommends countermeasures. They managed and ensured application compliance with all Information Assurance regulations to include Information Assurance & Vulnerability Assessments (IAVAs). Their infrastructure management support included coordination and interaction with TNOSC organizations as well as mission application support that rides on the infrastructure.

b. [E2] Provided services and solutions to accomplish cross-domain security.

Offeror indicated this was not required on this effort.

AR, exh. 10, DSC.001_DSC Consensus_DSC NETOPS SB PCAG CONSENSUS REPORT, at 4.

Simply stated, the record shows that, although the PCAG reports identify certain information aside from the question of relevancy in passing (for example, the PCAG reports note the quality ratings, and have a space to fill in other information such as the dollar value of the contract, and its dates and place of performance), the reports make clear that the PCAG was concerned almost entirely with the question of whether a contract was relevant to the proposed effort, and not with any of the RFP’s other considerations.

Turning to the proposal analysis report (PAR), a similar preoccupation with relevancy permeates the agency’s assignment of performance confidence ratings. Again, by way of example, the PAR for D&S assigns the firm’s proposal a satisfactory confidence rating. The PAR also identifies in a table the contracts reviewed, enumerates the number and type of relevancy ratings assigned under each area of consideration, and notes the quality rating assigned in each area (for example, in the NMES area the PAR states: “Within the Network Management/Enterprise Services criteria, the offeror had 1 Highly Relevant, 4 Somewhat Relevant and 3 Not Relevant citations with Very Good to Exceptional performance.” AR, exh. 15, 15.001_Redacted PAR-DSCI, at 121). The PAR then lists the areas of relevant performance, and lists areas where there was an absence of relevant performance. Id. at 122. The PAR then includes a summary paragraph:

Based on the information identified above, the PCAG assigned the performance confidence assessment of Satisfactory Confidence. Of the 9 citations evaluated, 9 covered more than one but less than four years. DSC demonstrated Not Relevant to Highly Relevant ratings and Very Good to Exceptional quality of work across the criteria areas. 3 of 6 criteria areas demonstrated multiple efforts with Highly Relevant or Relevant performance with worldwide support and a recent and sustained track record. One area, Information Transport Systems demonstrated one Highly Relevant and 5 Somewhat Relevant efforts. After reviewing the past performance in this area the PCAG determined DSC had demonstrated sufficient past performance in this area.

Id.

The PAR essentially is identical for all of the offerors in terms of the review performed and the depth of the agency’s critical analysis of the offerors’ performance confidence. For example, the PAR for Abacus concludes as follows: 

Based on the information identified above, the PCAG assigned the performance confidence assessment of Satisfactory Confidence. Of the 8 citations evaluated, 8 covered more than one but less than four years. ABA demonstrated Not Relevant to Highly Relevant ratings and Very Good to Exceptional quality of work across the criteria areas. 4 of 6 criteria areas demonstrated multiple efforts with Highly Relevant or Relevant performance with worldwide support and a recent and sustained track record.

AR, exh. 15, 15.001_PAR-Final Redacted_Redacted, at 109-110. Again, the agency’s overwhelming focus is on the question of relevance. While these narrative materials make passing references to the quality of the offerors’ past performance, the duration of the past performance efforts; and, where applicable, to the worldwide area of performance, there is no detailed analysis of how these other considerations were factored into the agency’s ultimate assignment of a performance confidence rating. In fact, a review of the entire record shows that, in every case where the agency assigned a substantial confidence rating rather than a satisfactory confidence rating, the sole discriminator was the number of areas where the offeror was found to have sufficient relevant experience. For example, in the SSDD, the source selection authority (SSA) states:

It is clear within the Source Selection Decision Brief that while the six Offerors who received satisfactory confidence assessment ratings demonstrated relevant past performance of the depth and breadth and scope of the work contemplated under this IDIQ contract in four of the criteria areas, the PCAG identified two areas of concern for each of these Offerors. It is also clear within the Source Selection Decision Brief that the twelve Offerors that received substantial confidence assessment ratings demonstrated relevant past performance of the depth and breadth and scope of the work contemplated under this IDIQ contract in at least five of the criteria areas.

SSDD at 7.[19]

What is lacking from the record is any narrative explanation regarding how--or whether--the RFP’s other enumerated considerations were factored into the agency’s assignment of performance confidence ratings. For example, D&S points out that some awardees received lower quality ratings than the ratings assigned to D&S, but those firms nonetheless were assigned substantial confidence ratings, apparently on the basis of having more relevant performance examples.

In sum, we conclude on this record that the agency failed meaningfully to consider not just the relevancy of the offerors’ performance examples, but also the extent to which the offerors’ performance examples demonstrated the other considerations identified in the RFP, including quality of performance, whether the offeror demonstrated a sustained track record of performance, the magnitude of the offerors’ past performance examples, and whether the examples were performed worldwide. We therefore sustain D&S’s protest on this basis.

SOURCE SELECTION DECISION

D&S and Abacus both argue that the agency’s source selection decision was flawed because it essentially relied entirely on the performance confidence ratings, without regard to price. D&S couches its allegation in terms of the agency failing to consider price in making its award decisions, and also in terms of the agency not giving adequate consideration to evaluation considerations other than relevance. Abacus asserts that the agency failed to make the price-performance tradeoffs required by the RFP. Both firms essentially take issue with the agency’s decision to award contracts only to those firms receiving substantial confidence ratings.

We sustain this aspect of the protests. In this connection, FAR § 15.308 specifically requires that the source selection decision be based on a comparative assessment of proposals against all source selection criteria in the solicitation. It further requires that the source selection decision be documented, and the documentation include the rationale for any business judgments and tradeoffs made or relied on by the SSA, including benefits associated with additional costs. FAR § 15.308; see also, ACCESS Sys., Inc., B-400623.3, Mar. 4, 2009, 2009 CPD ¶ 56 at 7.

The record shows that the SSA essentially made a “class” division between all offerors receiving satisfactory confidence ratings, and all offerors receiving substantial confidence ratings. Based on that division, the SSA made award only to those offerors with substantial confidence ratings, regardless of their TEP in relation to offerors with satisfactory confidence ratings. The SSA states:

I have a high expectation that Offerors with Substantial confidence will successfully perform the required effort. While some of the Offerors with Satisfactory confidence ratings may appear to provide a price discount as compared to some of the Offerors rated Substantial, it is my judgment that awarding a contract to an offeror with Satisfactory confidence introduces performance risk in the areas of concern identified by the PCAG which does not justify awarding to those Offerors. Additionally, since the pool of Offerors with whom I have Substantial confidence exceeds the target number of awardees, award to an Offeror with Satisfactory confidence is not warranted. The Offerors with Satisfactory confidence failed to provide evidence consistent with the RFP to convince the PCAG, the SSAC and me of their ability to perform work of the depth and breadth and scope necessary in at least two of the criteria areas. [20]

* * * * *

It is my judgment that awarding a contract to any offeror with Satisfactory confidence could impact mission capability and increase contract costs while also creating additional management and oversight costs and requirements for the decentralized awarding office as well as the Business Enterprise Systems NETCENTS-2 Team.

SSDD at 11.

The SSA ignored significant price differences among the offerors in refusing to make award to offerors with satisfactory confidence ratings. This point is most dramatically demonstrated by the price disparity between Abacus and the awardees; Abacus’s TEP is almost half of the TEP of the highest-priced awardee, and is below the TEPs of the 10 of the 12 awardees. The SSA offers no explanation for why it is worth making award to these firms offering such significant price premiums in comparison to Abacus. The absence of specific performance/price tradeoff explanations for each successful and unsuccessful offeror pairing does not withstand logical scrutiny, and is inconsistent with the mandate of the FAR that awards be based on a comparative assessment of proposals that documents the agency’s rationale for any business judgments and tradeoffs made by the SSA. FAR § 15.308.

We also agree with D&S that the source selection decision does not explain why significant differences among the offerors in terms of the other RFP considerations (quality of performance, sustained track record of contract performance, similarity in terms of magnitude, and worldwide performance) are not material to the selection decisions. For example, D&S notes that it received higher quality ratings than some awardees, but there is no explanation in the SSDD of why these comparative quality advantages enjoyed by D&S did not overcome its relative lack of relevant performance examples. Here, too, the absence of performance/price tradeoff explanations for each successful and unsuccessful offeror pairing also fails to meet the requirements of the FAR that an agency’s source selection be based on a comparative assessment of proposals against all source selection criteria indentified in the RFP.

RECOMMENDATION

We recommend that the agency perform a new performance confidence evaluation that is consistent with the discussion above. At the conclusion of that reevaluation, we recommend that the agency make a new source selection decision. Should the agency conclude that one or more of the original awardees are no longer in line for award, we recommend that the agency terminate their contracts for the convenience of the government. Correspondingly, we recommend that the agency make award to any concern that it identifies as in line for award, if otherwise proper. Finally, we recommend that the Air Force reimburse Abacus and D&S the reasonable costs of filing and pursuing their respective protests, including reasonable attorneys’ fees. 4 C.F.R. § 21.8(d)(1) (2014). The protesters’ certified claims for costs, detailing the time expanded and costs incurred, must be submitted to the agency within 60 days after receipt of this decision. 4 C.F.R. § 21.8(f).

The protests are sustained in part, denied in part, and dismissed in part.

Susan A. Poling
General Counsel