Shaw-Parsons Infrastructure Recovery Consultants, LLC; Vanguard Recovery Assistance, Joint Venture

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Shaw-Parsons Infrastructure Recovery Consultants, LLC; Vanguard Recovery Assistance, Joint Venture

File: B-401679.4; B-401679.5; B-401679.6; B-401679.7

Date: March 10, 2010

DIGEST

1. Protest is sustained where agency failed to consider past performance questionnaires it received regarding the protester's performance on its past performance references. Such information was "too close at hand" for the agency to ignore.

2. Protest arguing that performance of awardees on their incumbent contracts was "too close at hand" for the agency to have ignored is denied where the record indicates that the agency did not in fact maintain any information regarding the performance of the incumbent contractors.

Shaw-Parsons Infrastructure Recovery Consultants, LLC, of Alexandria, Virginia, and Vanguard Recovery Assistance, Joint Venture, of Washington, D.C., protest the award of architect/engineering (A/E) services contracts to Fluor Enterprises, Inc., AECOM Services, Inc., Nationwide Infrastructure Support Technical Assistance Consultants (NISTAC), and CH2MHill-CDM PA-TAC Recovery Services (CCPRS), pursuant to solicitation No. HSFEHQ-09-R-0411, issued by the Department of Homeland Security, Federal Emergency Management Agency (FEMA), to support the Public Assistance (PA) program under the Robert T. Stafford Disaster Relief and Emergency Assistance Act, 42 U.S.C. sections 5121-5206. Both Shaw-Parsons and Vanguard challenge the agency's technical evaluation. Among other things, they argue that the agency failed to reasonably consider past performance information that was in its possession.

We sustain the protest filed by Shaw-Parsons and deny the protest filed by Vanguard.

BACKGROUND

This procurement was conducted pursuant to the Brooks Act, 40 U.S.C. sections 1101 et seq. (Supp. V 2005) and its implementing regulations, Federal Acquisition Regulation (FAR) subpart 36.6. In accordance with those regulations, on February 19, 2009, the agency issued a Sources Sought Notice (SSN) on the FedBizOpps website, identifying its need for A/E, consultant, and other professional services in support of the agency's PA program. The SSN indicated that FEMA needed firms capable of supporting the PA program throughout the 50 states, the District of Columbia, Puerto Rico, the U.S. Virgin Islands, Guam, American Samoa, and the Commonwealth of the Northern Mariana Islands. FEMA also sought firms to "augment" its capacity to respond to "natural catastrophes, with emphasis on riverine and coastal flooding, tornadoes, hurricanes, typhoons, earthquakes, and tsunamis, or regardless of cause, any fire, flood, or explosion." SSN at 1. Through this SSN, FEMA announced its intent to negotiate and award up to four indefinite-delivery/indefinite-quantity (ID/IQ) contracts with a 1-year base period and four 1'year option periods for performance of this work. FEMA estimates the combined value of the contracts to be approximately $2 billion. According to the SSN, orders would be issued on a time and materials or fixed-price basis. The contemplated contracts are in fact the third iteration in a series of public assistance and technical assistance contracts (PA TAC III), which have been used to implement FEMA's PA program. They are to serve as the successors to the PA TAC I and, most recently, the PA TAC II contracts.

Interested firms were invited to submit a completed standard form (SF) 330 (A/E qualifications statement) detailing their qualifications to provide the services. Firms were advised that their qualifications would be evaluated under the following five factors: (1) specialized experience and technical competence; (2) capacity to accomplish work within the required time; (3) professional qualifications; (4) past performance; and (5) location in the general geographic area of the project. The SSN, which was amended several times, specified that factors 1, 2, and 3 were of equal importance and more important than factors 4 and 5. Evaluation factor 5 was identified as more important than factor 4. In addition, factor 1 consisted of the following three subfactors: subfactor 1A, experience developing reliable cost estimates for multi-million dollar construction projects and/or infrastructure repair projects; subfactor 1B, experience in evaluating projects for compliance with environmental regulations and preparing environmental documents; and subfactor 1C, experience in staffing at the levels in evaluation factor 2.

As it relates to the protest, under factor 2, the SSN required firms to "demonstrate the ability" to identify, within 48 hours of receiving notice from the government, approximately 290 technical specialists who could perform an 8-month assignment, with 200 of those specialists available for an additional 4-month period. SSN at 3. Under the past performance factor, the SSN provided that FEMA would evaluate firms' past performance in performing contracts of "similar size, type, and scope . . . in terms of project management, accuracy of costs estimates, cost control, quality control, completion of projects within budget, and compliance with performance schedules." Id. Firms were required to provide "references for at least five contracts within the past three (3) years with names, affiliations, and telephone numbers, with a narrative discussion." Id. The SSN also advised that FEMA "reserves the right to use information outside of the response in evaluating past performance, including agency knowledge of the firm[']s performance." Id. at 4.

FEMA received completed SF 330s from nine firms, including Shaw'Parsons, Vanguard, Fluor, AECOM, NISTAC, and CCPRS. As it relates to the protest, Fluor, AECOM, and NISTAC were the incumbent PA TAC II contractors.[1] On April 20, FEMA instructed the firms to have at least five of their past performance references complete a "past performance questionnaire" (PPQ) form which FEMA provided to the firms. The PPQ asked references to rate firms' performance as "superior," "acceptable," or "unacceptable" with respect to four performance elements: (1) quality of product or services; (2) cost control; (3) timeliness of performance; and (4) business relations. These forms also provided for narrative discussions of the firms' performance ratings. The firms were ultimately provided 3 days to accomplish this task.

After evaluating the SF 330s, the Source Selection Evaluation Board (SEB) determined that 7 of the 9 firms should move forward for oral presentations.[2] Shaw'Parsons Agency Report (AR), Ex. 14, SEB Report Phase I, at 11; Vanguard AR, Ex. 11, SEB Report, at 11. Shaw-Parsons, Vanguard, Fluor, AECOM, NISTAC, and CCPRS were included in this group, and oral presentations were held in late April.

Upon completion of the oral presentations, the SEB reevaluated the firms and prepared its consensus evaluation report, which included a technical ranking of the seven firms. On May 6, the SEB transmitted its report to the source selection authority (SSA). That same day, the contracting officer asked the seven short-listed firms to submit cost proposals in preparation for negotiations pursuant to FAR sect. 36.606. Shaw-Parsons AR at 3; Vanguard AR at 2. Based on the SEB's rankings, the contracting officer conducted negotiations with the top four firms, which included Shaw-Parsons. After reaching agreement on contract terms with those firms, on June 15, the agency announced its decision to make award to the following firms: Fluor, AECOM, NISTAC, and Shaw-Parsons.

Two of the unsuccessful firms filed agency-level protests.[3] In response to one of the protests, FEMA took corrective action, which included the issuance of an addendum to the SSN, which identified changes with respect to evaluation factor 1, and allowed the seven short-listed firms to submit revised SF 330s addressing the change. Specifically, FEMA revised subfactor 1A to read as follows:

(A) The firm is required to identify completed projects from the past five (5) years that demonstrate its experience developing reliable cost estimates for a variety of major multi-million dollar construction projects and/or repair of damaged infrastructure systems (e.g., buildings, roads, schools, hospitals, and power and water systems, etc.).

In its project examples, the firm must demonstrate its experience, methodology, and tools to estimate the total cost of projects (e.g., labor, materials, and equipment), and its use of forward pricing models for multi-year projects. The firm must also demonstrate its experience in developing and utilizing quality control measures to ensure the accuracy of its cost estimates.

The firms are required to provide a detailed explanation of the reasons for any variances on the identified completed projects that exceed plus or minus 10 percent between the estimated costs in the proposal solicitation and the actual costs of the completed project.

SSN Addendum, at 2.

In addition, the addendum established that subfactor 1A was "significantly more important" than either subfactors 1B or 1C, that it was more important than subfactors 1B and 1C combined, and that subfactors 1B and 1C were of equal importance. The SSN addendum further advised firms that "[n]o other factors shall be revised in your SF 330. Id. Only Evaluation Factor 1(A) will be re-evaluated. No other factors will be adjusted from the original evaluation, however, they will be considered for the final evaluation." Id.

After receiving the firms' revised submissions, the SEB conducted a reevaluation of subfactor 1A and prepared a revised consensus report with a new ranking of the firms. Based on the rescoring, the short-listed firms were rated as follows:

Factor	Fluor	AECOM	CCPRS	NISTAC	Shaw	Vanguard	Firm A
1. Specialized Experience and Technical Competence	S*	A	S	S	S	S	A
Subfactor 1A. Experience Developing Reliable Cost Estimates	S	A	S	S	S	S	A
Subfactor 1B. Experience in Evaluating Projects for Compliance with Environmental Regulations	S	S	S	A	A	A	S
Subfactor 1C. Experience Staffing at Levels required in Factor 2	A	S	A	A	A	A	A
2. Capacity to Accomplish Work within required time	S	S	S	A	A	S	A
3. Professional Qualifications	S	S	A	S	S	A	A
4. Past Performance	A	S	S	A	A	A	A
5. Location in Geographic Area	S	S	S	S	S	S	S

Shaw-Parsons AR, Ex. 15, Revised Consensus Report, at 17; Vanguard AR, Ex. 13, at 17.

Based on the revised ratings, the SEB determined that Fluor was the highest-rated firm, with AECOM and CCPRS tied for second. Because the outcome of the competition would not have been affected, the SEB decided not to determine which of these firms held the second and third positions. The next three firms, NISTAC, Shaw-Parsons, and Vanguard were tied for the fourth position. In order to break the tie, the SEB considered the differences between these firms under factor 1A and determined that NISTAC, Shaw-Parsons, and Vanguard were ranked, respectively, fourth, fifth, and sixth. Id. The SEB recommended that the four most highly rated firms be considered for award.

After reviewing the SEB's revised report and recommendation, the SSA made her selection decision, which she documented in a memorandum dated October 6, 2009. Shaw-Parsons AR, Ex. 16, Source Selection Decision Document (SSDD); Vanguard AR, Ex. 14, SSDD. In this decision, the SSA independently compared and contrasted the evaluation findings under each factor for the three firms tied for the fourth position (NISTAC, Shaw-Parsons, and Offeror A). Id. at 4-5. Based on this assessment, the SSA identified NISTAC for the fourth award position, concluding that it was more highly technically qualified than Shaw-Parsons and Vanguard. Id. As part of this assessment, the SSA concluded that there was nothing to distinguish these firms under the past performance factor. Upon learning of the agency's revised selection decision, Shaw-Parsons and Vanguard filed these protests.

DISCUSSION

Shaw-Parsons' Protest

Shaw-Parsons' protest raises numerous challenges to the agency's evaluation and selection decision. Among other things, Shaw-Parsons argues that FEMA failed to properly evaluate its past performance by not considering questionnaires submitted by its references. As detailed below, we conclude that FEMA's past performance evaluation was fundamentally flawed. Although we do not specifically address all of Shaw-Parsons' remaining issues and arguments, we have fully considered all of them and find they provide no basis on which to sustain the protest; we discuss some illustrative examples below.[4]

Consideration of Past Performance Questionnaires

Shaw-Parsons argues that FEMA's past performance evaluation was fundamentally flawed because it failed to consider the PPQs it received regarding Shaw-Parsons' performance, as well as those of the other firms, and instead relied solely upon information contained in the firms' SF 330 submissions. We agree.

In reviewing an agency's evaluation under the past performance factor, the critical questions are whether the evaluation was conducted fairly, reasonably, and in accordance with the stated evaluation terms, and whether it was based on relevant information sufficient to make a reasonable determination of the firm's overall past performance rating. University Research Co., LLC, B-294358.6, B'294358.7, Apr. 20, 2005, 2005 CPD para. 83 at 16. An agency's past performance evaluation is unreasonable where the agency fails to give meaningful consideration to all the relevant past performance information it possesses. DRS C3 Sys., LLC, B-310825, B-310825.2, Feb. 26, 2008, 2008 CPD para. 103 at 22.

Here, the SSN required firms to provide references for at least five contracts performed within the last 3 years. FEMA specifically provided firms with the PPQs, which they were to have completed by their past performance references and returned to FEMA. In its evaluation of Shaw-Parsons' past performance information, FEMA determined that Shaw-Parsons had "[p]rovided requested information on past performance on 5 contracts of similar size, type, and scope with Government agencies and private industry in terms of project management, accuracy of costs estimates, cost control, quality control, completion of projects within budget, and compliance with performance schedules." Shaw-Parsons AR, Ex. 17, SEB Evaluation of Shaw-Parsons, at 23. The record also reflects that five of Shaw-Parsons' past performance references provided FEMA with completed PPQs. Shaw-Parsons' PPQs reflect that it was rated "superior" under each performance element (quality of product or service, cost control, timeliness of performance, and business relations) for all five contracts, except for one contract where it received a rating of "acceptable" under the cost control element. This translates to ratings of "superior" on 19 out of 20 possible past performance ratings in Shaw-Parsons' PPQs.

Further, the PPQs contain narrative comments regarding the quality of Shaw-Parsons' performance. For example, on a U.S. Postal Service (USPS) contract with a value of nearly $500 million, the reference provides narrative comments for each performance element. Specifically, with respect to the "quality of product or service" performance element, the reference commented that Shaw-Parsons "has provided SUPERIOR services and products . . . evidenced by the fact that the USPS has awarded Parsons nearly 17,000 individual task orders! . . . Parsons has consistently met and exceeded the expectations of the USPS in the performance of large projects and programs such as the Leased Space Accessibility Program, where Parsons manages the work at over 12,000 facilities in nearly every state of the Union." Shaw-Parsons' USPS Contract PPQ. Under timeliness of performance, the USPS reference indicated that "Parsons' performance in terms of meeting the project schedules has been outstanding" and noted two prime examples in this regard. On a $250 million FEMA contract for "individual assistance" and "technical assistance" (ITAC), the reference, in justifying the "superior" ratings for Shaw-Parsons, noted that "Shaw has provided superior support by having highly qualified staff and resources onsite in less than 24 hours during critical need, and between 48-72 hours for routine requirements. Far exceeds their competitors." Shaw-Parsons' FEMA IATC PPQ. The other PPQs provide comments with specific examples in justification of their "superior" ratings regarding the quality of Shaw-Parsons' performance.

FEMA, however, did not consider Shaw-Parsons' PPQs as part of its past performance evaluation. In fact, FEMA did not consider the PPQs for any firm. Rather, FEMA performed its assessment of the quality of the firms' past performance based entirely on the information provided by the firms in their SF 330s.[5] As a consequence, FEMA rated Shaw-Parsons' past performance "acceptable," finding, in part, that the information contained in Shaw-Parsons' SF 330 was insufficient to adequately judge the quality of its past performance. Shaw-Parsons AR, Ex. 17, SEB Evaluation of Shaw-Parsons, at 23. We conclude that FEMA's failure to consider Shaw-Parsons' PPQs in assessing the quality of Shaw-Parsons' past performance was improper.

While there is no legal requirement that an agency consider all past performance references, we have held that some information is simply "too close at hand to require offerors to shoulder the inequities that spring from an agency's failure to obtain and consider information." SCIENTECH, Inc., B-277805.2, Jan. 20, 1998, 98-1 CPD para. 33 at 5 (citing Int'l Bus. Sys., Inc., B'275554, Mar. 3, 1997, 97-1 CPD para. 114 at 5'6). Specifically, we consider PPQs in an agency's possession to be past performance information too close at hand to ignore. Intercontinental Constr. Contracting, Inc.--Costs, B-400729.3, Mar. 4, 2009, 2009 CPD para. 44 at 2. Consideration of the PPQs was particularly material in this case since FEMA had not obtained any other third-party assessments of the quality of the firms' past performance. Common sense dictates that an offeror's self-assessment regarding the quality of its past performance, as reflected in its SF 330, is, by its nature, of less value as compared to the disinterested assessments provided by third parties.[6] As noted above, a critical consideration in our review of an agency's past performance evaluation is whether it is based on relevant information sufficient to make a reasonable determination of the firm's overall past performance rating. University Research Co., LLC, supra at 16. By ignoring the PPQs it had at hand, the agency here failed to satisfy this standard in its evaluation of past performance.

FEMA maintains that its decision not to consider the PPQs was reasonable because the factors for evaluation under the PPQs did not match the factors for evaluation under the SSN. FEMA also asserts that such consideration would have been unfair given that the agency did not receive PPQs for some references for some firms and some of the PPQ responses addressed firms' performance under individual task orders, rather than the firms' overall contract performance.[7] Based on these inconsistencies, the contracting officer determined that the SEB "would not further consider [the PPQs] in the evaluation." SEB Chairperson, Decl., Jan. 22, 2010, at 3. The agency's arguments are unpersuasive.

As an initial matter, the record shows that FEMA prepared the PPQs and requested that firms distribute them to their references. It therefore strains credulity for FEMA to now claim that the PPQs were so fundamentally inadequate as to render them per se unusable. Second, the inconsistencies between the past performance elements as set forth in the PPQs and those identified in the SSN do not justify ignoring the only third-party assessments of the quality of the firms' past performance obtained by FEMA. Specifically, the past performance elements identified for evaluation in the PPQs were "cost control," "quality of product or service," "timeliness of performance," and "business relations." While not identical, these topics closely parallel and appear to be directly relevant to the past performance evaluation categories identified in the SSN, which included "accuracy of cost estimates," "cost control," "completion of projects within budget," "quality control," "compliance with performance schedules," and "project management." SSN at 3.

In addition, to the extent some firms' references did not return, or did not properly prepare, individual PPQs, these failings cannot justify the agency's decision to ignore what are otherwise relevant PPQs with detailed information documenting the quality of the firms' past performance. Such a conclusion would be at odds with the very nature of the "close at hand" principle, which reflects the duty of an agency to consider relevant information in its possession notwithstanding whether it was actually submitted by an offeror or whether the agency has sought similar information for other offerors. Moreover, the agency's "fairness" concerns weigh in favor of giving consideration to those firms whose references responded appropriately with relevant past performance information, as opposed to denying these firms the benefit of this information because some of the references for the other firms failed to do the same.[8]

FEMA also asserts that its decision to evaluate past performance based solely on the information provided by the firms in their SF 330s was consistent with the SSN and that sufficient information existed in the firms' SF 330s to reasonably assess the quality of their past performance. Regarding the former point, FEMA notes that it was not required to consider the PPQs because under the terms of the SSN, FEMA merely reserved the right to use information outside of the response in evaluating past performance, including agency knowledge of the firm's performance. As discussed above, however, the PPQs in this case were simply too close at hand for FEMA to have ignored. In other words, regardless of what discretion the solicitation may have afforded FEMA in seeking out additional information, once it had the PPQs, it could not simply ignore them.[9]

Regarding the latter point, the SEB Chairperson asserted that the "SEB believed that each of the firms provided sufficient information in the SF 330 to allow the SEB to conduct this qualitative assessment." Decl. of SEB Chairperson, Jan. 22, 2010, at 3. This argument, however, is inconsistent with FEMA's fundamental concern regarding the inadequacy of the past performance information contained in Shaw-Parsons' SF 330. FEMA cannot on the one hand claim that it was justified in disregarding the PPQs because the SF 330s provided a sufficient basis to evaluate firms' past performance, and then, on the other hand, assert that Shaw-Parsons' "acceptable" past performance rating was justified based on a lack of information in its SF 330.

Other Evaluation Challenges

As noted above, Shaw-Parsons also challenges other aspects of FEMA's evaluation. We have reviewed these allegations and conclude that they are without merit. For example, Shaw-Parsons argues that its "acceptable" rating under subfactor 1B, environmental compliance experience, was unreasonable because it was based on the erroneous determination that it did not demonstrate "extensive" experience in this area. Shaw-Parsons AR, Ex. 17, SEB Evaluation of Shaw-Parsons, at 13. According to Shaw-Parsons, this determination was unjustified where the company demonstrated its experience in having completed thousands of National Environmental Policy Act (NEPA) documents. As explained by FEMA, Shaw'Parsons had prepared [DELETED] environmental assessments and [DELETED] environmental impact statements; in FEMA's view, this reflected only "moderate" experience and the agency found Shaw-Parsons' general reference to having "completed" thousands of NEPA documents as lacking sufficient specificity to convey the requisite level of environmental compliance experience in producing completed environmental assessments and environmental impact statements as required under subfactor 1B.[10] While the protester may ultimately disagree with the agency's assessment in this regard, such disagreement does not demonstrate that the agency's evaluation was unreasonable. See The OMO Group, Inc., B-294328, Oct. 19, 2004, 2004 CPD para. 212 at 5.

With respect to factor 2, Shaw-Parsons takes issue with FEMA's finding that Shaw'Parsons did not demonstrate its ability to staff at the requisite levels "for the time period specified." Shaw-Parsons AR, Ex. 17, SEB Evaluation of Shaw-Parsons, at 18. According to Shaw-Parsons, this finding is unwarranted in light of the statement in its SF 330 that it "has the capacity to mobilize specialists within 48 hours and to provide support for as long as necessary." Shaw-Parsons' SF 330 at 47. In addition, Shaw-Parsons argues that the agency's finding is inconsistent with the strengths the agency attributed to its submission, which included the ability of Shaw'Parsons to maintain staff through the life of the contract and specifically cited its [DELETED], as well as its strategies for [DELETED]. Shaw-Parsons AR, Ex. 17, SEB Evaluation of Shaw-Parsons, at 18.

Contrary to the protester's contentions, FEMA's findings are reasonably based and are not internally inconsistent. In this regard, the weakness about which the protester complains stemmed from FEMA's concern that Shaw-Parsons' SF 330 did not address its ability to maintain personnel for the specific 8-month and 4-month assignment periods identified under factor 2 since Shaw-Parsons did not specify the length of deployments for its personnel. Shaw-Parsons AR, Ex. 17, SEB Evaluation of Shaw'Parsons, at 18. The general assurance provided by Shaw-Parsons that it can "provide support for as long as necessary" does not provide any insight regarding its ability to deploy staff for the specific periods required under factor 2, and thus it does not address the agency's concern, which was reasonably related to the specified requirements. Shaw-Parsons' SF 330 at 47. Moreover, there is no inconsistency arising from the strength identified by FEMA since the strength relates to Shaw-Parsons' ability to provide staffing "through the life of the contract," not the ability of Shaw-Parsons to provide staff for the specific 8-month and 4-month deployment periods, which formed the basis of the agency's concern.

Vanguard's Protest

Throughout its protest, Vanguard principally asserts that the agency's evaluation under subfactor 1A and its past performance evaluation were flawed because FEMA failed to consider allegedly negative information relating to the reliability of the awardees' cost estimating in connection with their performance as the incumbent PA TAC II contractors. In this regard, Vanguard asserts that such information was "too close at hand" for FEMA to have ignored. The record, however, does not support Vanguard's assertions in this regard.[11]

As a preliminary matter, we find one aspect of Vanguard's protest untimely. Specifically, Vanguard alleges that the agency's past performance evaluation was flawed because FEMA failed to consider the PPQs. This is the same argument discussed above in connection with the protest filed by Shaw-Parsons. Unlike the protest filed by Shaw-Parsons, however, Vanguard's protest in this regard is untimely because it learned this basis for protest in its June 12 debriefing,[12] but did not protest the matter within 10 days of its debriefing, as required under our Bid Protest Regulations. See 4 C.F.R. sect. 21.2(a)(2) (2009).[13]

Vanguard does, however, timely protest FEMA's failure to consider information other than the PPQs which it alleges was too close at hand to ignore. Specifically, Vanguard argues that FEMA failed to consider past performance information regarding the cost estimating abilities of the incumbent contractors in connection with their performance under the PA TAC II incumbent contracts. In support of its contentions, Vanguard cites two GAO reports,[14] both of which identify concerns regarding the reliability of FEMA's cost estimating and cost controls following major disasters under the PA program.[15] Vanguard further asserts that SEB members had direct knowledge of the incumbent contractors' poor cost estimating performance and that FEMA failed to reasonably consider information known by an advisor to the SEB, who was the contracting officer's technical representative (COTR) for the incumbent PA TAC II contracts.[16]

According to Vanguard, the awardees' alleged poor cost estimating performance under their incumbent PA TAC contracts should have been considered by FEMA in its evaluation under subfactor 1A, which required firms to demonstrate their experience with "developing reliable cost estimates for a variety of multi-million dollar construction projects and/or repair damaged infrastructure systems," as well as under the past performance factor. SSN Addendum, at 2. The agency and intervenors, however, argue that Vanguard's protest, as it relates to subfactor 1A, is misguided because it confuses the concepts of experience and past performance. We agree.

Generally, an agency's evaluation under an experience factor is distinct from its evaluation of an offeror's past performance. Specifically, the former focuses on the degree to which an offeror has actually performed similar work, whereas the latter focuses on the quality of the work. See Commercial Window Shield, B-400154, July 2, 2008, 2008 CPD para. 134 at 3.

Here, subfactor 1A required firms to demonstrate, and FEMA evaluate, the depth of the firms' cost estimating "experience" to include their experience using "tools to estimate the total cost of projects . . . use of forward pricing models for multi-year projects . . . [and] experience in developing and utilizing quality control measures to ensure the accuracy of [their] cost estimates." SSN, Addendum, at 2. According to Vanguard, by including a reference to "reliable" cost estimates, and requiring firms to explain cost variances on their identified projects in excess of 10 percent, FEMA was required to qualitatively assess the firms' actual cost estimating performance. Vanguard, however, over-reads these references, which merely served to define the scope of the experience that the agency would consider and provided a context for assessing the experience identified by the firms. They did not, as Vanguard suggests, transform what was intended as an assessment of experience into an overall qualitative assessment of a firm's cost estimating abilities. Rather, such qualitative assessments were properly considered under the past performance factor, which specifically informed firms of FEMA's intention to evaluate firms' performance with respect to the "accuracy of costs estimates" and "cost control." SSN, at 3. Moreover, Vanguard's interpretation of the evaluation criteria would effectively require FEMA to perform the identical evaluation under what are otherwise two separate and distinct evaluation factors. This is not a reasonable interpretation of the evaluation criteria established by the SSN.

Turning to the question of whether FEMA unreasonably ignored information regarding the awardees' alleged poor cost estimating performance under the incumbent PA TAC II contracts in its evaluation of their past performance, the record does not support this aspect of Vanguard's protest.[17] Regarding the GAO reports, to the extent the members of the SEB team were even aware of these reports and the findings contained therein, the reports do not identify problems with any particular contractor's performance under their PA TAC II contracts. Rather, the concerns raised generally address problems with FEMA's cost estimating activities. While the PA TAC II contractors may work closely with FEMA at times in developing its cost estimates, the reports simply do not provide any direct indication that the problems associated with FEMA's cost estimating issues were due to poor performance by the PA TAC contractors.

Moreover, the record does not support Vanguard's assertions regarding the SEB's knowledge of the incumbent contractors' performance under the PA TAC II contracts. In this regard, declarations from the SEB team state that they did not have knowledge of the incumbent firms' overall performance on the PA TAC contracts. For example, the SEB chairperson represented to our Office that "the SEB did not have past performance information or knowledge of the [incumbent] firms' performance on the [PA TAC] contracts other than those presented by each firm in its SF 330 and the questionnaires." SEB Chair Decl., Jan. 22, 2010, at 4.[18]

In our view, it would be reasonable to expect, as Vanguard asserts, that the individual serving as Chief of the PA TAC Management Branch of the PA Division and PA TAC II COTR, who had been identified by the incumbent PA TAC contractors as the point of contact for their PA TAC II contracts, see AECOM SF 330 at 21-22; Fluor SF 330 at 19, 21, 23, 25, 26, 56; NISTAC SF 330 at 21, 29, would possess information regarding the overall quality of the PA TAC II contractors' performance; however, she attests to having no such knowledge.[19] Specifically, she states, "There is no central repository for past performance assessments for the three (3) PA TAC II awards" and "there is no past performance information on PA TAC II." Suppl. Decl. of PA TAC II COTR, Feb. 25, 2010. Instead, even though the PA TAC II contracts had combined values of approximately $2 billion, the PA TAC II COTR explains that FEMA failed to incorporate any mechanism for measuring performance under these performance based contracts, and that "past performance information was not collected for the [PA TAC II contracts]."[20] Decl. of PA TAC II COTR, Jan. 22, 2010. While this lack of oversight and accountability is troubling, and a matter that should be rectified in connection with FEMA's award and oversight of the PA TAC III contracts, we cannot attribute knowledge to the agency evaluation team that it did not possess. Thus, there is no basis for concluding that FEMA ignored relevant information demonstrating the awardees' poor performance under the incumbent contracts, as Vanguard has argued.

RECOMMENDATION

We recommend that the agency, consistent with our decision, reevaluate the short-listed firms' SF 330s giving reasonable consideration to the PPQs it received for the firms as the information relates to the quality of these firms' past performance. In addition, we recommend that Shaw-Parsons be reimbursed its costs of filing and pursuing its protest, including reasonable attorneys' fees. 4 C.F.R. sect. 21.8(d)(1). Shaw-Parsons should submit its certified claim, detailing the time expended and costs incurred, directly to the contracting agency within 60 days of receiving this decision. 4 C.F.R. sect. 21.8(f)(1).

Shaw-Parsons' protest is sustained; Vanguard's protest is denied.

Lynn H. Gibson
Acting General Counsel