ASRC Research & Technology Solutions, LLC

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: ASRC Research & Technology Solutions, LLC

File: B-400217; B-400217.2

Date: August 21, 2008

DIGEST

1. Protest is sustained where agency unreasonably concluded that the protester's proposed plan for capturing incumbent personnel presented significant technical risk. The agency's technical evaluation in this regard stemmed from its determination, in the context of the agency's cost evaluation, that the protester's weighted average direct labor rates were too low to successfully capture incumbent employees. This conclusion, however, was based upon the premise that the agency's own unweighted average direct labor rates were a better reflection of the cost of the incumbent workforce, a premise not supported by the record.

2. Agency's evaluation of the awardee's past performance was unreasonable since the agency did not consider the substantial difference in size between the awardee's past performance references and the size of the contemplated contract, as required by the solicitation.

ASRC Research & Technology Solutions, LLC (ARTS) protests the award of a contract to SP Systems, Inc. under request for proposals (RFP) No. NNG07197688R, issued by the National Aeronautics and Space Administration (NASA), for programmatic support services within the Flight Project Directorate at Goddard Space Flight Center in Greenbelt, Maryland. ARTS argues that NASA's cost and technical evaluations were flawed.

We sustain the protest.

BACKGROUND

The Flight Project Directorate is responsible for planning and managing space flight projects at the Goddard Space Flight Center. These responsibilities include managing spacecraft flight hardware systems, ground systems, launch vehicles, and research instrumentation payloads. RFP attach. A, at 1. NASA currently receives contractor services in support of these responsibilities under its Program Analysis and Control (PAAC) II contract. The RFP was issued as a follow-on to PAAC II, and sought support services in the areas of planning and scheduling, configuration management, information technology, documentation/library, general business, and general accounting functions. The contemplated award under the RFP has been referred to as the PAAC III contract.

The RFP, issued on October 12, 2007 as a competitive 8(a) set-aside,[1] contemplates the award of a cost-plus-award-fee indefinite'delivery/indefinite-quantity (ID/IQ) contract, with a 1-year base period of performance plus four 1-year options. Award was to be made to the offeror submitting the proposal representing the best value to the government, considering three evaluation factors: mission suitability, past performance, and cost.[2] The RFP provided that the mission suitability factor would be weighted and scored based on a 1,000 point scale considering the following subfactors: understanding the requirements of the statement of work (SOW) (275 points); technical approach to representative task orders (RTO)[3] (275 points); management plan (400 points); and safety and health (50 points). RFP at 116.

As it relates to the protest, with respect to the management plan subfactor, the RFP advised that NASA would evaluate an offeror's staffing and recruiting plan and generally provided that –[t]he plan for any incumbent capture and the basis for this rate will be evaluated for reasonableness and consistency with other parts of the proposal.— RFP at 517-18. With respect to the cost factor, the RFP indicated that the agency would evaluate proposals for reasonableness and cost realism. For the purpose of evaluating cost, NASA sought to capture offerors' costs for the RTOs, as well as their total contract costs through a cost model.[4] In completing the cost model, offerors were required to propose, among other things, their direct and indirect rates, and specifically their direct labor rates for 18 specified non-management labor categories. These 18 labor categories were composed of six primary labor categories (Scheduling Specialist, Configuration Management Specialist, Documentation Specialist, Information Technology Specialist, Project Support Specialist, and Accounting Specialist) with three sub-categories under each category (Junior, Intermediate, and Senior). RFP, exh. 1A.

Under the RFP, offerors were allowed to propose labor categories which varied from those mandated by the cost model. RFP at 95. Thus, offerors' cost proposals could include sub-level labor categories under the 18 identified labor categories.[5] Where they did so, offerors were required to identify the percentage of effort for each sub-level labor category in the cost model, which was then used to calculate a –weighted— average labor rate for each of the 18 labor categories.

NASA also provided offerors, in a reading library, an average labor rate for the 18 labor categories. Regarding these –library rates,— the RFP's instructions stated that they –reflect the incumbent's current direct labor categories and average (prime and major subcontractors) unburdened direct labor hourly rates for those categories being used under the incumbent contract for PAAC II services. These Average Labor Rates . . . are provided for information purposes only.— RFP at 89-90 (emphasis in RFP). In response to questions from offerors, NASA further explained that the rates reflected a –'straight' average as opposed to a 'weighted average'—--that the rates –did not take into account the number of individuals in an specific [labor] category.— Agency Report, Tab 6, Responses to Industry Questions, at 00395, 00400. The RFP's instructions also stated that –when proposing to capture incumbent personnel, Offerors shall clearly explain variances from their proposed . . . Direct Labor Rates [for the 18 labor categories] and those of the current incumbent's unburdened rates.— RFP at 90.

While mission suitability and cost were separate factors, because –the proposed cost of the work (and rates proposed) may be a significant indicator of an Offeror's understanding and ability to perform the [work],— the RFP provided for adjusting mission suitability point scores to account for a lack of cost realism. RFP at 116. In this regard, the RFP included a table detailing point deductions to an offeror's mission suitability score based on the difference between the offeror's proposed costs for the RTOs and the most probable RTO costs as calculated by the agency (e.g., a difference of between 10 percent and 14.99 percent between an offeror's proposed cost and the probable cost would result in a deduction of 50 points from the offeror's mission suitability score). RFP at 117. The RFP specified that –[t]his adjustment is in addition to any finding(s) already reflected under Mission Suitability concerning the inadequacy of resources, cost or otherwise, prior to the cost realism adjustment.— Id.

For the purpose of evaluating past performance, the RFP instructed offerors to provide information for all relevant contracts and subcontracts in excess of $2 million that the offeror and/or subcontractor were currently performing or had completed within the past 3 years. The RFP also directed offerors (and their significant proposed subcontractors) to provide questionnaires to references in order to establish their record of past performance.

In evaluating offerors' past performance information, the RFP indicated that the agency would consider two components: relevance and performance. Regarding relevance, the RFP stated that NASA would –consider the degree of similarity in size, content, and complexity— between an offeror's past performance information and the solicitation requirements. RFP at 119. Under this scheme, past performance would be evaluated as –highly relevant,— –very relevant,— –relevant,— –somewhat relevant,— or not relevant. RFP at 119-20. The RFP advised that the agency would utilize the following adjectival rating scheme for the purpose of rating past performance: excellent, very good, good, fair, poor, and neutral. In order to be rated –excellent,— an offeror's past performance would have to be deemed to be –highly relevant.—

NASA received four proposals by the November 20 closing date. The agency established a source evaluation board (SEB) and its findings are summarized below:

Offeror	Mission Suitability	Mission Suitability Adjectival Rating	Proposed Cost Plus Award Fee	Recommended Cost Plus Award Fee	Past Performance
SP Systems	914	Excellent	$191,086,870.00	$192,448,817.00	Excellent
A	592	Good	$187,014,782.00	$189,426,766.00	Excellent
ARTS	864	Very Good	$176,914,763.19	$189,331,294.37	Excellent
B	382	Fair	$191,339,121.20	$196,763,915.36	Very Good

AR, Tab 37, SEB Report, at 2.

In its evaluation of SP Systems' proposal under the mission suitability factor, NASA identified 9 significant strengths, 12 strengths, 4 weaknesses, and no significant weaknesses or deficiencies. In evaluating ARTS' proposal under this factor, NASA identified 10 significant strengths, 9 strengths, 3 weaknesses, and 1 significant weakness (under the management plan subfactor). The significant weakness attributed to ARTS was a consequence of NASA's finding that ARTS' labor rates for their senior level personnel in the various labor categories of the cost model (i.e., accounting, configuration management, information technology, project support, and scheduling) were below the library rates. AR, Tab 37, SEB Report, at 65. In the view of the SEB this created a –significant risk— regarding ARTS' ability to achieve its proposed 98 percent incumbent capture rate. Id.According to the SEB, failing to capture the incumbents –will lead to an increased risk of unsuccessful contract performance.— Id.

Based upon the results of the SEB's evaluation, the SSA determined that the proposal submitted by SP Systems represented the best value to the government. In sum, the SSA found that SP Systems' overall –excellent— mission suitability proposal coupled with their –excellent— past performance outweighed the –slight— probable cost advantage maintained by ARTS. AR, Tab 29, SSA Decision, at 17. After receiving its debriefing, ARTS filed this protest.

DISCUSSION

In its protest, ARTS principally challenges NASA's evaluation of its cost proposal and the significant weakness attributed to its proposal under the management plan subfactor. Regarding these issues, ARTS complains that the agency unduly relied upon and mechanically applied the library rates in its cost realism and technical evaluations.[6] As discussed more fully below, we conclude that the agency's cost realism evaluation was reasonable to the extent the agency utilized the library rates solely as –plug numbers— in an attempt to normalize the basis for the cost evaluation among offerors who proposed to capture the incumbent workforce. Because the library rates may not in fact reflect the actual cost of the incumbent workforce, however, we sustain the protest to the extent that the agency relied on the difference between the library rates and the labor rates proposed by ARTS to conclude that ARTS' proposed plan for capturing incumbent personnel presented significant technical risk. ARTS also challenges the agency's past performance evaluation of the awardee.[7] Specifically, the protester argues that in assessing the relevance of SP Systems' past performance information, the agency failed to evaluate the degree to which the size of SP Systems' contracts was similar to that of the PAAC III requirement as required by the RFP. We sustain the protest in this regard as well.

Cost Realism and Mission Suitability

The agency's cost evaluation and the propriety of the significant weakness identified in ARTS' proposal under the management plan subfactor are directly related, and we therefore address these matters together. As it relates to these issues, and as noted above, the agency provided offerors with the PAAC II average labor rates, the –library rates,— for the 18 direct labor categories in the cost model. These rates were as follows:

	Accounting Specialist	CM	Documentation Specialist	MIS Specialist	Project Support Specialist	Scheduling Specialist
Junior Level	$16.46	$18.65	$14.74	$20.47	$19.49	$21.49
Mid Level	$17.25	$24.57	$21.31	$26.65	$22.41	$29.83
Senior Level	$24.24	$37.38	$29.16	$38.79	$29.91	$52.17

AR, Tab 8, Procurement Reading Library Materials, at 00712.

In their proposals, and as reflected in their respective cost models, both ARTS and SP Systems proposed sub-levels under the junior, intermediate, and senior sub-categories, in several instances. Specifically, while ARTS proposed only one junior-level and one intermediate-level position (which it called level I and II, respectively), it proposed to sub-divide the senior-level position into three sub-levels (which it called levels III, IV, and V). In a similar fashion, SP Systems' cost model was based on additional sub-levels, such that each sub-category (i.e., junior, intermediate, and senior) was composed of no fewer than three sub-levels. As explained above, ARTS and SP Systems had to identify in the cost model the percentage of effort for each sub-level, which was then used to calculate a –weighted— average labor rate. RFP, exhs. 2A and 2B.

For example, with respect to the accounting specialist labor category, the ARTS cost model provided the following information:

Direct Labor Category	Labor Rate	Portion %	Weighted Average Labor Rate
Level I - Accounting Specialist Junior	[deleted]	[deleted]	$16.46
Level II - Accounting Specialist Intermediate	[deleted]	[deleted]	$17.25
Level III	[deleted]	[deleted]	$22.97
Level IV	[deleted]	[deleted]
Level V	[deleted]	[deleted]
Accounting Specialist Senior			$22.97

ARTS Cost Proposal, exh. 2A.

For the senior-level labor category, ARTS' weighted average labor rate of $22.97, reflected in the above table, was less than the $24.24 library rate for this same labor category. This was true for each of ARTS' senior-level labor categories in the cost model, which reflected differences ranging from $1.27 up to $2.74 per labor category.

With respect to the Accounting Specialist labor category, SP Systems' cost model reflected the following:

Direct Labor Category	Labor Rate	Portion %	Weighted Average Labor Rate
Junior Acct. -- A	[deleted]	[deleted]
Junior Acct. -- B	[deleted]	[deleted]
Junior Acct. -- C	[deleted]	[deleted]
Accounting Specialist Junior	[deleted]	[deleted]	$16.46
Intermediate Acct. -- A	[deleted]	[deleted]
Intermediate Acct. -- B	[deleted]	[deleted]
Intermediate Acct. -- C	[deleted]	[deleted]
Accounting Specialist Intermediate	[deleted]	[deleted]	$17.25
Chief Acct. Support Specialist	[deleted]	[deleted]
Principal Acct. Support Specialist	[deleted]	[deleted]
Senior Acct. Support Specialist -- A	[deleted]	[deleted]
Senior Acct. Support Specialist -- B	[deleted]	[deleted]
Accounting Specialist Senior			$24.24

SP Systems Cost Proposal, exh. 2A.

Unlike the cost proposal submitted by ARTS, SP Systems' weighted labor rate for the senior-level positions exactly matched the library rates for all six labor categories.

With respect to both the cost evaluation and the significant weakness identified in ARTS' proposal, a key issue is the rates being paid to the incumbent workforce and how ARTS' proposed labor rates compared to those rates. Because both ARTS and SP Systems proposed to retain a very high proportion of the incumbent workforce, the agency was justifiably focused on how the offerors' proposed rates compared to those of the incumbent. This affected the agency's cost-realism adjustment to proposed costs, since the agency rightly assumed that, absent some valid explanation, an offeror proposing to retain a very high proportion of the incumbent workforce would need to pay at least equal to the incumbent workforce's rates. Because this is a cost-reimbursement contract, a cost realism analysis was required to determine the extent to which each offeror's proposed costs represent the offeror's likely costs in performing the contract under the offeror's technical approach, assuming reasonable economy and efficiency. See Federal Acquisition Regulation (FAR) sections 15.305(a)(1), 15.404'1(d)(1). The proposed labor rates also affected the agency's evaluation of proposals under the management plan subfactor for offerors proposing to retain the incumbent workforce, since proposing labor rates lower than the incumbents' could reasonably be found to represent a management plan weakness.

Unfortunately, NASA's solicitation did not disclose the incumbent workforce's actual labor rates to offerors, at least not in a meaningful way, and those conducting the evaluation may not have had access to the actual rates.[8] The library rates are unweighted averages, which do not reflect the distribution of actual labor rates among the sub-levels within the junior/intermediate/senior levels that the incumbent uses.[9] This problem with the RFP was not protested, however, so that the agency was free to choose any reasonable method, within the context of the RFP, to assess the evaluated cost of each proposal and to evaluate the cost-related technical factors, such as the management plan subfactor.

With respect to the cost-realism analysis, the agency, as noted above, took the reasonable view that it could adjust proposed costs up to reflect the rates paid to the incumbent workforce, if the offeror proposed to retain the great majority of the incumbent workforce, as both ARTS and SP Systems did. The problem in making that adjustment, however, was that those conducting the evaluation apparently did not have access to the incumbent's actual weighted rates. Instead, the agency relied on the unweighted library rates and treated them as reflecting the rates paid the incumbent workforce.

While it would clearly have been preferable to use the incumbent workforce's weighted rates in calculating offerors' evaluated costs, we believe that it was adequate, as a legal matter, that the offerors were treated equally, through the agency's use of the library rates as a –plug number.— This methodology of treating the incumbent workforce cost as, in effect, a normalized cost was reasonable, since the cost of the incumbent workforce would not have been unique to the particular approach of any individual offeror (nor has there been a suggestion to the contrary), and offerors such as ARTS were not in a position to know the actual cost of the incumbent workforce. In other words, one would reasonably expect that the direct labor cost of the incumbent workforce should be the same among all offerors. Absent persuasive explanation for any deviation (which ARTS did not offer here), a reasonably derived estimate of direct, unburdened labor rates for comparable labor categories can provide an objective standard against which the realism of proposals can be measured. United Int'l Eng'g et al., B'245448.3 et al., Jan. 29, 1992, 92-1 CPD para. 122 at 11. As a consequence, there is no basis for our Office to question the agency's upward adjustment of ARTS' cost, as part of the cost-realism analysis, to account for the cost of the incumbent workforce.

We do not, however, find support in the record for the determination that ARTS' proposed rates were inadequate to retain the incumbent workforce. For this reason, we find problematic both the input that NASA received from the Defense Contract Audit Agency (DCAA) and NASA's assignment of a significant weakness to ARTS' proposal under the management plan subfactor.

Regarding the DCAA input, with respect to ARTS' proposed labor rates for labor categories at the senior level, DCAA noted that ARTS proposed to capture 98 percent of the incumbent workforce and determined that –the method used by [ARTS] to compute the proposed senior level direct labor rates resulted in a potential understatement of direct labor rates. [ARTS] used a weighted average which resulted in a rate lower than the straight average rate.— AR, Tab 33, DCAA Audit of ARTS Cost Proposal, at 5. DCAA then calculated an unweighted average rate with respect to the rates proposed by ARTS under the senior level labor categories and determined that the unweighted average was identical to the library rates.[10] Without further elaboration, DCAA determined that the library rates were a more reliable and reasonable basis for the proposed senior level rates.[11] Id.As explained above, there is no way to tell, from the unweighted library rates, how much the incumbent workforce is being paid. Indeed, DCAA's analysis demonstrates this. As DCAA discovered, ARTS' proposed rates, when averaged without weighting, are precisely the same as the unweighted library rates, so that it is theoretically possible that ARTS' proposed labor rates are in fact identical to those actually being paid by the incumbent. In any event, because the record does not establish a connection between the library rates and the incumbent's actual, weighted rates, we see no basis in the record to support DCAA's analysis.

More importantly, we find no reasonable basis for the agency's assignment of a significant weakness to ARTS' proposal under the management plan subfactor. [12] Absent more information, there simply is no way for the agency to determine whether the library rates or the weighted averages proposed by any offeror are closer to the incumbent's direct labor cost. In addition, if one compares the weighted rates proposed by ARTS and SP Systems, it is not possible in many instances to determine whether one firm will be more or less likely to attract the incumbent workforce in any given labor category, since the firms proposed different high and low rates within a labor category, in some instances different numbers of levels of sub-categories, and different percentages of effort for each sub-level. For any given labor category, it may be that, as compared to the incumbent workforce, one offeror's rates are high, the other's are low; neither the evaluators nor the offerors had any basis to know.

ARTS, like the other offerors, proposed to perform the PAAC III contract utilizing the existing incumbent workforce. However, as explained above, offerors such as ARTS did not have access to the actual labor rates that the incumbent was paying its workforce. Nonetheless, offerors proposing to use the incumbent workforce, including ARTS, had to account for the cost of this workforce in their proposals. ARTS attempted to do this by including a blanket statement committing ARTS to paying incumbent employees their current salaries, at a minimum, and providing labor rates based in part upon outside salary survey information. While the rates proposed by ARTS, when averaged on a weighted basis, were lower than the non-weighted library rates, they were identical to the library rates, when averaged on a straight line basis, as the library rates themselves had been calculated.

As relevant here, the agency could only assign ARTS' proposal a significant weakness under the management plan subfactor based on a determination that ARTS is unlikely to be able to retain the incumbent workforce with its proposed labor rates.[13] There is simply no basis in the record for that. ARTS' proposed rates may be lower than those paid to the incumbent workforce--but they may be higher than the incumbent's rates. Indeed, as noted above, they may be identical to the rates of the incumbent. Yet, under the management plan subfactor, NASA questioned the ability of ARTS to achieve its proposed 98 percent incumbent capture given its –unreasonably low— labor rates for the senior-level positions as compared to the library rates. AR, Tab 37, SEB Report, at 65. Given the meaninglessness of the library rates as a criterion for retaining the incumbent workforce, the conclusion drawn by the agency in assessing ARTS' ability to retain that workforce was unreasonable, especially where ARTS committed to paying incumbents their current wages, at a minimum. Accordingly, we conclude that the agency's evaluation in this regard was unreasonable.

While our findings regarding the technical evaluation and the cost-realism adjustment may appear inconsistent with regard to the treatment of the library rates, we believe that they are consistent. In the cost-realism analysis, we found that the agency could reasonably use a –plug number— for labor rates for all offerors that proposed to retain the incumbent workforce. We found use of the library rates acceptable, under the circumstances--not because they reflected the incumbent's rates, but simply because they were a constant used equally for all offerors. From that standpoint, the agency could just as well have used ARTS' labor rates, or SP Systems', as the plug numbers. With regard to the technical evaluation evaluation, however, the agency was finding that ARTS had proposed rates so much lower than the incumbent's as to present a significant management plan weakness, and that finding could not be supported without evidence that the library rates were closer than ARTS' rates to the incumbent's rates--and the record provides no basis for that finding. [14]

Past Performance

ARTS also challenges the agency's evaluation of SP Systems' past performance, arguing that the agency failed to evaluate the degree to which the size of SP Systems' contracts was similar to the PAAC III requirement as required by the RFP. The PAAC III procurement was valued at nearly $200 million and is expected to involve more than 270 personnel. In its evaluation of SP Systems' past performance, NASA considered six past performance contract references for SP Systems and its major subcontractor. AR, Tab 37, SEB Report, at 125-28. Of these, five were valued between $2 million and $3.5 million, with between 5 and 12 employees, and the sixth had a dollar value of approximately $30 million, with 67 employees. AR, Tab 23, SP Systems' Past Performance Information, at 02168. In its evaluation of ARTS' past performance, NASA considered the contracts performed by ARTS team, which included, among others, a contract with value of $600 million, one with a value of $250 million, one valued at more than $100 million, as well as a contract with a value of $43.6 million. AR, Tab 37, SEB Report, at 137-43. Most of these contracts involved performance with more than 200 employees. AR, Tab 14, ARTS' Past Performance Information, at 01146. Both offerors' proposals were rated as –excellent— under the past performance factor, which required a determination by NASA that their past performance information was –highly relevant.—

As a general matter, the evaluation of an offeror's past performance is a matter within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based past performance ratings. However, we will question an agency's evaluation conclusions where they are unreasonable or undocumented. Clean Harbors Envtl. Servs., Inc., B-296176.2, Dec. 9, 2005, 2005 CPD para. 222 at 3; OSI Collection Servs., Inc., B-286597, B-286597.2, Jan. 17, 2001, 2001 CPD para. 18 at 6. The critical question is whether the evaluation was conducted fairly, reasonably, and in accordance with the solicitation's evaluation scheme. Clean Harbors Envtl. Servs., Inc., supra. The agency's past performance evaluation here did not meet this standard.

The record reflects that, in evaluating offerors' past performance, NASA utilized, in essence, a –pass/fail— criterion with respect to its consideration of the relative size of offerors' past performance references. In responding to ARTS' allegations, NASA essentially argues that in considering relevance, offerors' prior contracts were deemed to be relevant by the terms of the RFP if they met the $2 million minimum threshold established by the RFP. [15] Agency Supplemental Report at 31.

The fundamental premise of NASA's argument, however, is flawed. By the terms of the RFP, the evaluation of relevance with respect to size was not merely a –pass— or –fail— determination (either over or under the $2 million threshold). Rather, the solicitation specified that NASA would consider the –degree— to which the size of an offeror's past performance references (in addition to scope and complexity) are similar to the size of the PAAC III requirements. Thus, consistent with the solicitation language, NASA had to consider the relative size of offerors' past performance references in weighing their past performance ratings and assessing whether the references were highly relevant, very relevant, relevant, somewhat relevant, or simply not relevant at all. In this context, the $2 million minimum set forth in the solicitation cannot be considered anything more than a floor, establishing the minimum dollar amount that NASA would consider for the purpose of evaluating relevance. Since there is nothing in the record to indicate that NASA engaged in the type of analysis required by the solicitation, we conclude that its determination that SP Systems' past performance was –highly relevant,— particularly given that SP Systems' references were, in most respects, small fractions of the size of the contemplated PAAC III contract, was unreasonable. Sytronics, Inc., B-297346, Dec. 29, 2005, 2006 CPD para. 15 at 6-7.

RECOMMENDATION

We recommend that NASA reevaluate proposals by removing the –significant weakness— improperly assigned to ARTS under the mission suitability factor and re-adjusting ARTS' point scores and ratings accordingly, and reevaluating the relevance of SP Systems' past performance. Based upon this reevaluation the agency should make a new source selection decision. If, after the new evaluation, NASA determines that another firm's proposal represents the best value to the government, the agency should terminate SP Systems' contract and make a new award. We also recommend that the agency reimburse the protester for the reasonable costs of filing and pursuing its protests, including reasonable attorneys' fees. Bid Protest Regulations, 4 C.F.R. sect. 21.8(d)(1). The protester's certified claims for costs, detailing time expended and costs incurred, must be submitted directly to the agency within 60 days of receiving this decision. 4 C.F.R. sect. 21.8(f)(1).

The protest is sustained.

Gary L. Kepplinger
General Counsel