PricewaterhouseCoopers LLP

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: PricewaterhouseCoopers LLP

File: B-409537; B-409537.2

Date: June 4, 2014

DIGEST

Protester’s contention that source selection authority’s best value tradeoff decision improperly abandoned the solicitation’s stated evaluation scheme is sustained where the record as a whole, and the testimony provided at a hearing, shows that the source selection authority selected the lower-priced quotation after concluding that the evaluated technical merit of the protester’s quotation, and the technical weaknesses of the awardee’s quotation, were offset by the awardee’s past performance--even though both quotations were rated the same for past performance and past performance provided no basis for distinguishing between the quotations.

PricewaterhouseCoopers LLP (PwC), of McLean, Virginia, protests the issuance of a task order by the National Aeronautics and Space Administration (NASA) to CliftonLarsonAllen LLP (CLA), of Calverton, Maryland, under CLA’s General Services Administration Federal Supply Schedule (FSS) contract pursuant to request for quotations (RFQ) No. NNH13461926Q for financial statement audit services in support of NASA’s Office of Inspector General. PwC challenges the agency’s selection decision.

We sustain the protest.

BACKGROUND

The RFQ, issued pursuant to Federal Acquisition Regulation (FAR) Subpart 8.4 to holders of FSS contracts for financial and business solutions services (Schedule 520), provided for the issuance of a fixed-price task order for financial statement audit services for a base year with four 1-year options; the RFQ also provided that the task order would include five 1-year options for Federal Information Security Management Act (FISMA) evaluation services.[1] A detailed statement of work was provided that described the required financial statement audit services and FISMA evaluation services.

Vendors were informed that the task order would be issued on a best-value basis, considering the following three factors in descending order of importance: technical; past performance; and price.[2] RFQ exhib. 1, Instructions and Evaluation Criteria, at 13. The non-price factors, when combined, were significantly more important that price. Id. With respect to the technical factor, vendors were informed that the agency would evaluate, among other things, the adequacy of the vendor’s “approach to documenting systems and internal controls and effective procedures, including consideration of risk and materiality, to determine the extent of audit testing.” Id. at 13-14. The RFQ also stated that NASA would evaluate the vendor’s approach

to audit of essential areas such as Government-owned, contractor-held property; environmental liability estimates; and the SAP [system applications and products] financial management systems; as well as their methods and tools for conducting network vulnerability assessments and penetration tests for reasonableness and effectiveness.

Id. at 14.

Detailed instructions were also provided for the preparation of quotations under each factor. Specifically, vendors were informed that “[i]t is NASA’s intent . . . to solicit information that will demonstrate the [vendor]’s competence to successfully complete the requirements specified in the [SOW] . . . .” Id. at 1. Accordingly, the RFQ instructed vendors that their quotations were to: demonstrate understanding of the overall and specific requirements of the proposed task order; convey the company’s ability for transforming understanding into accomplishment; and present in detail the plans and methods for doing so. Id.

NASA’s Evaluation

NASA received quotations from two vendors, incumbent PwC and CLA, which were evaluated by the agency’s source evaluation team (SET) using adjectival rating schemes set forth in the RFQ. Specifically, quotations were evaluated under the technical factor as excellent, very good, good, fair, or poor; and were evaluated under the past performance factor as very high confidence, high confidence, moderate confidence, low confidence, very low confidence, or neutral.[3]

The final evaluation ratings and prices of the PwC and CLA quotations were as follows:[4]

	PwC	CLA
Technical	Excellent	Good
Past Performance	Very High Confidence	Very High Confidence
Price	$17,426,037	$15,200,357

Agency Report (AR), Tab 15, SET Evaluation Report, Dec. 6, 2013, at 1-18; Tab 21, SET Evaluation Report, Jan. 29, 2014, at 1-7; Tab 22, Source Selection Decision, Feb. 11, 2014, at 11.

The SET’s narrative evaluation report identified strengths and weaknesses underlying the adjectival ratings assigned to the vendors’ technical quotations. PwC’s excellent rating under the technical factor reflected the SET’s judgment that the protester’s quotation had two significant strengths, two strengths, and no weaknesses.[5] AR, Tab 15, SET Evaluation Report, Dec. 6, 2013, at 1-5. In contrast, CLA’s good rating under the technical factor reflected the SET’s judgment that CLA’s quotation had one significant strength, one strength, and three weaknesses. AR, Tab 21, SET Evaluation Report, Jan. 29, 2014, at 1-7.

One of PwC’s significant strengths was its approach to auditing the essential areas of government-owned, contractor-held property; environmental liability estimates; and the SAP financial management system.[6] AR, Tab 15, SET Evaluation Report, Dec. 6, 2013, at 1. Specifically, the SET found that PwC’s quotation recognized the risks for [DELETED].[7] Id. The SET also found that PwC’s quotation provided a comprehensive summary of the key audit risks regarding environmental liabilities. Id. at 2. Further, the SET found that PwC’s quotation provided a listing of the SAP modules in use at NASA and specified for each module whether it anticipated its inclusion as in-scope for application controls review.[8] Id. The SET found the depth to which PwC had outlined its audit approach permitted the evaluators to have a comprehensive understanding of the type and extent of testing anticipated. Id. “The Offeror’s approach to auditing these specific areas, which were each identified by the government as essential areas, represents a well-designed and effective approach that greatly enhances the potential for a comprehensive and thorough audit.”[9] AR, Tab 15, SET Evaluation Report, Dec. 6, 2013, at 2.

The SET recognized as a strength PwC’s description of its methods and tools for conducting network vulnerability assessments and penetration tests. Id. at 3. The SET found that PwC’s quotation explained the difference between internal and external penetration testing, and provided examples of vulnerabilities that could exist if security controls were not in place. Id. The agency also found that PwC’s plan was tailored to identify some of the most prevalent and potentially dangerous information technology (IT) vulnerabilities. Id. Further, PwC’s quotation set forth its penetration testing methodology, consisting a six-phase approach which included a description of the procedures performed. Id. The SET found that PwC’s detailed and comprehensive depiction of its proposed methods and tools for conducting network vulnerability assessments and penetration tests represented a well-designed plan which the government considered to be reasonable and effective, thereby enhancing the potential for successful performance. Id.

The SET also recognized as a strength PwC’s understanding of current accounting, IT security, and auditing issues that may affect the work effort. Id. at 5. The SET found that PwC identified several federal accounting, auditing, and IT issues that it was monitoring that had the potential to impact NASA’s audit efforts. “Not only did the Offeror list the issues, but also elaborated on what each of the issues are and in some cases indicated how the issues may impact the audit effort.” Id. Further, the SET found that PwC outlined the various tools used to maintain and update its understanding of the current accounting, IT security, and auditing environment within the vendor’s federal services practice. Id. The SET found PwC’s quotation demonstrated a solid understanding of pending changes in the federal accounting, auditing, and IT environment, including FISMA requirements. Id. The agency evaluators concluded that PwC’s discussion of its information dissemination tools increased the government’s confidence that personnel assigned to the NASA engagement “would be kept abreast of changes in the accounting, auditing, and IT security environment, which should result in quality work that is performed in accordance with the applicable standards.” Id.

With respect to CLA’s quotation, the SET noted, in addition to the one significant strength that was common to both vendors, that CLA’s quotation offered a strength for its audit approach for environmental liability estimates. AR, Tab 21, SET Evaluation Report, Jan. 29, 2014, at 2. The SET also noted, however, that CLA’s quotation had three weaknesses. The first concerned CLA’s proposed methods and tools for conducting network vulnerability assessments and penetration tests. Id. at 3. Specifically, the SET found that none of the tools specified by CLA were effective with respect to detecting vulnerabilities in internet (e.g., web-based) applications. Id. The SET also expressed concern that CLA’s vulnerability assessment and penetration testing procedures did not include procedures or processes to discover and assess vulnerabilities in NASA web-based applications, stating:

Because NASA has over 1,500 publicly-accessible websites, the [vendor’s] failure to describe how it would assess IT security posture for this significant population of at-risk IT assets increases the chance that the [vendor’s] vulnerability and penetration testing procedures would fail to accurately and completely identify the presence of high-risk vulnerabilities in a significant and high-risk population of the Agency’s IT assets.

Id. In this regard, the SET noted that, although CLA stated that it planned to conduct internal and external penetration tests, the approach outlined by the vendor for the testing phase did not provide details on internal testing. Id. The SET concluded that the tools proposed and level of detail contained in CLA’s penetration testing approach diminished the government’s ability to fully determine the reasonableness and effectiveness of CLA’s methodology, which increased the risk of unsuccessful contract performance. Id. at 3-4.

The second weakness assessed in CLA’s quotation concerned its approach and methodology for auditing material account balances for NASA. Id. at 5. Specifically, the SET noted that CLA failed to specify the timing of when the operating effectiveness of the relevant internal controls would be tested, stating that “[w]ithout specificity regarding the timing of the internal controls testing, for all audit areas . . . , the government is unable to sufficiently evaluate the reasonableness of the [vendor’s] planned approach and methodology.” Id. The SET concluded that the timing of the testing regarding the effectiveness of internal controls would impact the vendor’s ability to meet the schedule for deliverables related to the financial statement audit and FISMA efforts, which also increased the risk of unsuccessful contract performance. Id.

The third weakness assessed in CLA’s quotation was based on two evaluated shortcomings: (1) CLA’s approach for documenting systems and internal controls and effective procedures; and (2) its ability to adequately adapt to and meet changing requirements. Id. at 6. Specifically, the SET found that CLA had not stated how its methodology and related software program were effective for documenting IT systems, procedures, and controls and for adequately assessing IT security risks and developing recommendations for mitigating such risks. Id. “This omission by the [vendor] is considered a weakness as IT security risk identification and mitigation is the overarching objective of a FISMA evaluation.” Id. The SET also noted that CLA’s quotation did not provide any examples of how the vendor had adapted to and met contract deliverables when experiencing changes to FISMA requirements. Id. “[T]he ability to adapt to and meet contract deliverables in the face of changing FISMA requirements is essential.” Id. The SET concluded that CLA’s inadequate description of its approach for documenting systems and internal controls and effective procedures for the FISMA portion of the work decreased the government’s confidence in the adequacy of the vendor’s approach. Id. at 7. Further, the evaluators found CLA’s omission of examples of how the vendor had successfully adapted to changing FISMA requirements increased the risk that should FISMA requirements change during the course of the reporting period, the vendor may be unable to adequately adapt its program to address those changes while at the same time meeting contract deliverables and FISMA milestones. Id.

With respect to past performance, the SET found that both PwC’s and CLA’s past performance merited equal “very high” confidence ratings. In this regard, the SET found PwC’s past performance to be very highly relevant and of “exceptional merit.” AR, Tab 15, SET Evaluation Report, Dec. 6, 2013, at 7. Similarly, the evaluators found CLA’s past performance to be very highly relevant and with “very effective performance.” Id. at 17-18.

Agency Selection Decision

The SET’s evaluation results were presented to the agency’s source selection authority (SSA). The SSA did not independently read and evaluate the quotations, but accepted the SET’s evaluation findings.[10] Tr. at 45, 47. The SSA recognized that the vendors had received the same past performance rating (very high confidence), and concluded that “Past Performance was not a discriminator.” AR, Tab 22, Source Selection Decision, Feb. 11, 2014, at 14. The SSA also recognized that CLA’s price was approximately $2.2 million, or 13 percent lower than PwC’s. Id. The SSA then examined whether PwC’s higher technical rating warranted the price premium associated with its quotation.

In this regard, the SSA examined the three weaknesses assessed in CLA’s quotation. Id. at 8-9. Although he agreed with the SET’s underlying evaluation findings, the SSA disagreed with the SET’s assessments regarding whether the weaknesses would affect CLA’s ability to perform. Specifically, the SSA found that:

these Weaknesses could be successfully addressed through proper Government oversight during contract performance. . . .[11] In order to address the first Weakness . . ., the Government would need to work collaboratively with CLA to determine which tools to use for conducting network vulnerability assessments and penetration tests. In order to address the second Weakness . . . , the Government would need to coordinate with CLA to determine when testing of internal controls would occur. In order to address the third Weakness . . . , the Government would need to work with CLA to determine the specific approach to be used in order to adapt to and meet changing requirements and high-risk areas. If properly addressed (and I have confidence they could be), these Weaknesses should not increase the risk of unsuccessful contract performance.[12]

Id. at 13. Moreover, the SSA was of the opinion that CLA’s technical weaknesses were mitigated by the vendor’s past performance evaluation, which the SSA found demonstrated that CLA could actually carry out these types of audit services successfully.[13] Tr. at 53, 62-63.

The SSA also considered PwC’s evaluated technical superiority, noting in particular PwC’s significant strength for its approach to auditing several essential areas (government-owned, contractor-held property; environmental liability estimates; SAP financial management system; and its approach and methodology for auditing material account balances).[14] The SSA concluded that PwC’s significant strength was “offset as a discriminator” by CLA’s strength with respect to CLA’s approach to auditing environmental liability estimates. Id. at 14-15. The SSA stated that “[i]n comparing PwC’s Significant Strength with CLA’s Strength, I decided both offerors displayed overall competence in performing the contract work.” Id. at 15. More specifically, the SSA testified that PwC’s significant strength could be offset by CLA’s strength, because of CLA’s record of past performance. Tr. at 129, 131.

The SSA also considered two additional technical strengths in PwC’s quotation: the first one for a well-developed and thorough technical approach and presentation of the methods and tools to be used for conducting network vulnerability assessments and penetration tests, and the second one for the vendor’s detailed understanding of the current account, IT security, and auditing issues that may affect the work effort. AR, Tab 22, Source Selection Decision, Feb. 11, 2014, at 15. The SSA concluded that neither of PwC’s strengths provided any meaningful benefit that CLA would not also provide:

PwC’s first strength “enhanced the potential for successful performance;” however, both [vendors] had Very High Level of Confidence in Past Performance. The shared ratings in Past Performance satisfied me that CLA also enjoyed a high potential for successful performance of the contract. PwC’s second strength indicated it would provide quality work in accordance with the applicable standards. Similarly, the SET’s findings for CLA did not suggest in any way that this [vendor] would fail to meet the applicable standards set forth in the contract.

Id. Although the SSA acknowledged that the two strengths in PwC’s quotation were both in areas where the SET had assessed weaknesses in CLA’s quotation, and that a weakness could in fact suggest a vendor would fail to meet applicable standards, see Tr. at 144, 148-50, the SSA concluded that he was satisfied based upon CLA’s past performance record that CLA could satisfactorily perform the task order. Id. at 147.

In the end, the SSA concluded that CLA’s quotation represented the best value to the government: “[t]he benefits associated with the technical advantages in PwC’s proposal do not justify paying a $2.2 million premium.” AR, Tab 22, Source Selection Decision, Feb. 11, 2014, at 15.

DISCUSSION

PwC challenges the agency’s selection decision, arguing that the SSA in his price/ technical tradeoff unreasonably ignored both PwC’s technical strengths and CLA’s technical weaknesses, thereby essentially finding the vendors’ quotations to be technically equal when they were not.[15] PwC contends that had the agency reasonably considered the firms’ respective technical merit, it (and not CLA) would have been selected to receive the task order.

In reviewing protests of awards in a FSS competition, we do not reevaluate quotations but examine the record to determine whether the evaluation and source selection decision are reasonable and consistent with the solicitation’s evaluation criteria and applicable procurement laws and regulations. Savvee Consulting, Inc., B-408416.3, Mar. 5, 2014, 2014 CPD ¶ 92 at 4; IBM Global Bus. Serv. - U.S. Fed., B-409029, B-409029.2, Jan 27, 2014, 2014 CPD ¶ 43 at 4. In this regard, FAR Subpart 8.4 requires that agencies make their award decisions in accordance with the solicitation’s stated evaluated criteria. FAR § 8.405-2(c)(3)(iii)(C) (the ordering activity contracting officer shall ensure . . . award is made in accordance with the evaluation criteria in the RFQ).

Where, as here, a procurement provides for issuance of a task order on a “best value” basis, it is the function of the SSA to perform a price/technical tradeoff, that is, to determine whether one quotation’s technical superiority is worth its higher price. General Dynamics Info. Tech., Inc., B-406030, B-406030.3, Jan. 25, 2012, 2012 CPD ¶ 55 at 6; InnovaTech, Inc., B-402415, Apr. 8, 2010, 2010 CPD ¶ 94 at 6. While an agency has broad discretion in making a tradeoff between price and nonprice factors, an award decision in favor of a lower-rated, lower-priced quotation must acknowledge and document any significant advantages of the higher-priced, higher-rated quotation, and explain why they are not worth the price premium. See NOVA Corp., B-408046, B-408046.2, June 4, 2013, 2013 CPD ¶ 127 at 5. For example, in Blue Rock Structures, Inc., B-293134, Feb. 6, 2004, 2004 CPD ¶ 63, our Office sustained a challenge to an agency’s selection of a lower-rated, lower-priced proposal, where the SSA failed to acknowledge the evaluated advantages of the higher-rated proposal, and to furnish an explanation as to why the protester’s higher-rated proposal’s advantages were not worth price premium. In contrast, our Office held in Phoenix Group of Virginia, Inc., B-407852, Mar. 12, 2013, 2013 CPD ¶ 80, that the selection of a lower-rated, lower-priced proposal was unobjectionable where the record showed that the SSA considered evaluated differences in quotations, documented her deliberations and rationale, and concluded that a slight technical advantage was not worth a substantial price premium.

Here, after reviewing the agency’s report and the SSA’s selection decision, our Office had concerns about the adequacy of the SSA’s price/technical tradeoff determination and conducted a hearing to obtain the SSA’s further explanation. We find from our review of this record, as explained below, that the SSA’s tradeoff determination was not consistent with the stated evaluation criteria. Rather, the SSA discounted PwC’s technical superiority based upon CLA’s past performance (which the record does not show to be a discriminator) and based upon the SSA’s view that CLA’s weaknesses would be negated by contract oversight.

As noted above, the SSA accepted the SET’s underlying findings regarding PwC’s technical strengths and CLA’s technical weaknesses. The SSA concluded, however, that CLA’s weaknesses could be effectively mitigated by government oversight, and were not a cause for concern because CLA’s record of past performance indicated that the firm could satisfactorily perform the task order. Likewise, the SSA found that PwC’s significant strength--to the extent not offset by CLA’s smaller-in-scope strength--was offset by CLA’s record of past performance. Again, the SSA found that PwC’s other two strengths were also offset by CLA’s record of past performance. Put simply, the testimony provided in the hearing before our Office showed that the SSA essentially used CLA’s past performance as a replacement, or substitute, for CLA’s evaluated technical inferiority:

Q: Why -- on a number of occasions now, [SSA], you seem to be relying upon CLA’s past performance to either make up for weaknesses in their proposal or to negate the strengths in PwC’s technical proposal.

A: And I certainly don’t mean to do that. But PwC had a very solid technical proposal. And -- you know, CLA had a very good one too. But then when I looked at [CLA’s]--any of the weaknesses identified, when I looked at the past performances, they had done all of this comparable work in all of these [audit] areas, they just did not put it in the -- in the technical proposal section.

Tr. at 131-32.

The record reflects, however, that CLA did not have stronger past performance than PwC, such that the SSA could reasonably make tradeoffs between one vendor’s technical superiority and another vendor’s stronger past performance. Rather, PwC and CLA were found to have essentially equal past performance.[16] In fact, the SSA expressly found that past performance was not a discriminator between the PwC and CLA quotations. AR, Tab 22, Source Selection Decision, Feb. 11, 2014, at 14.

The SSA, however, treated CLA’s past performance as if it were superior to PwC’s, and used it as the reason to offset PwC’s technical advantages. In this regard, the SSA repeatedly testified that in making his determination, both CLA’s weaknesses and PwC’s strengths were offset by CLA’s past performance (“when you look at the past performance, that would just offset some of [PwC’s] strengths”). Tr. at 139; see also Tr. at 62-63, 129, 147. We fail to see how CLA’s past performance, which was equal to PwC’s and was not a discriminator between the two vendors, could reasonably be considered to offset PwC’s technical advantages.[17]

The agency argues that the SSA, within the broad discretion afforded him, properly selected CLA’s lower-technically rated, lower-priced quotation after reasonably deciding that the price premium involved in selecting PwC’s higher-rated, higher-priced quotation was not justified. NASA cites to the SSA’s testimony, concluding that “I just made the determination that for the best value to the government, that the $2 million price differential was not worth the technical advantage. Tr. at 54. Taken as a whole, however, the record reflects that the SSA did more than simply trade technical merit for price; instead the SSA improperly magnified the importance of past performance and used it to conclude that the identified technical differences were of no consequence. Put simply, the SSA first concluded that there is no difference between the vendors’ past performance, but then relied upon CLA’s past performance as the reason for offsetting PwC’s technical superiority, before conducting the final price/technical tradeoff.

In sum, considering the broad discretion accorded selection officials to make price/technical tradeoffs, NASA might well decide that the technical merit of the PwC’s quotation is not worth the added cost--recognizing that past performance for these companies provides no basis to distinguish one from the other. Such a judgment is governed only by tests of rationality and consistency with the solicitation’s evaluation criteria. See Buckley & Kaldenbach, Inc., B-298572, Oct. 4, 2006, 2006 CPD ¶ 138 at 3. On this record, we cannot conclude NASA met this test.

RECOMMENDATION

We recommend that NASA make a new source selection decision that is consistent with the solicitation and adequately documents the comparative merits of vendors’ quotations and the agency’s rationale for award. If CLA’s quotation is not found to reflect the best value to the government, we recommend that the agency terminate the task order issued to CLA and issue the task order to PwC. We also recommend that the agency reimburse PwC’s costs of filing and pursuing the protest, including reasonable attorneys' fees. 4 C.F.R. § 21.8(d)(1) (2014). In accordance with section 21.8(f) of our Regulations, PwC’s claim for such costs, detailing the time expended and the costs incurred, must be submitted directly to the agency within 60 days after receipt of this decision.

The protest is sustained.

Susan A. Poling
General Counsel