Matter of: Advanced Communication Systems, Inc. File: B-283650; B-283650.2; B-283650.3 Date: December 16, 1999

Matter of: Advanced Communication Systems, Inc. File: B-283650; B-283650.2; B-283650.3 Date: December 16, 1999

DIGEST

Attorneys

DECISION

Advanced Communication Systems, Inc. (ACS) protests the award of contracts to the team led by Booz-Allen & Hamilton, Inc. under request for proposals (RFP) No. N00039-99-R-3205(Q), issued by the Space and Naval Warfare Systems Command (SPAWAR), Department of the Navy, for support services for systems located at fixed sites and mobile platforms. /1/ ACS challenges the selection decision on numerous grounds, primarily arguing that SPAWAR's cost realism analysis of Booz-Allen's proposal was flawed and that SPAWAR misevaluated ACS's technical proposal. /2/

We deny the protest.

BACKGROUND

The RFP, issued on April 16, 1999, contemplated the award of up to five cost reimbursement, level-of-effort type contracts, with fixed and award fee provisions, for a base period with up to four 1-year option periods. RFP Sec. B. The RFP estimated the level of effort at 1.2 million direct labor hours per year. Id. Sec. L-2(h). Offerors were required to submit a written proposal in two separate volumes and make an oral presentation. Id. Sec. L Para. 2.4, at 53. Volume I of the written proposal was to consist of (1) corporate experience and past performance information; (2) personnel resumes; (3) small, small disadvantaged, and woman-owned small business participation; and (4) briefing charts for the oral presentation. The oral presentation was to address the offeror's technical and management approach; sample tasks; and small, small disadvantaged, and women-owned small business participation. Volume II was the cost proposal. Id. The RFP required that the functional areas of the statement of work (SOW) be accomplished by a team of contractors; each competing team was to describe in its proposal how the SOW would be assigned among the various team members. Offerors were also instructed to include a matrix in their proposals cross-referencing each numbered SOW paragraph to a specific team member. /3/

The RFP stated that the agency would evaluate each team's proposal in the following two areas: (1) team competence/composition and (2) cost. Within the first area, the RFP listed the following factors of equal importance: technical and management approach; sample tasks; small business participation; corporate experience and past performance; and personnel resumes. Id. Sec. M-1. The RFP stated that the technical evaluation factors would be considered significantly more important than cost, and that the agency would select the responsible team whose proposal conformed to the RFP's requirements and represented the best value to the government. Id. The RFP explained that after the agency selected a team based on the results of the evaluation, the government would award a separate contract to each member of the selected team; the SOWs of each of those separate contracts would be consistent with how the team divided the work in its proposal. Id. Sec. L.4.0.

Three teams, including teams led by ACS and Booz-Allen, submitted proposals by the time set on May 24 for receipt of initial proposals. The teams provided their responses to the technical and management approach, sample tasks, and small business participation factors in oral presentations to the technical evaluation board (TEB) and the contracting officer (CO); the agency also conducted a question and answer session with each team during the oral presentations. Following the oral presentations, the TEB evaluated proposals from each team by assigning adjectival ratings of either marginal, average, good, or excellent under each evaluation factor, and an overall proposal rating. A cost evaluation board (CEB) reviewed cost proposals and developed a most probable cost (MPC) for each team based on Defense Contract Audit Agency (DCAA) audits and recommendations and information obtained from the Department of Labor. The following chart shows the results of the TEB's and CEB's evaluations for the teams led by the protester and the awardee:

Evaluation Factor Booz-Allen ACS

Tech./Mgmt. Approach Good Average

Sample Tasks Excellent Average

Small Business Particip. Excellent Excellent

Corp. Exp/Past Perf. Good Good

Personnel Resumes Excellent Average

Overall Rating Excellent Good

MPC $257,161,635 $326,929,388

Agency Report (AR), Oct. 12, 1999, at 5, 9.

Based on the results of the evaluation, the TEB recommended to a source selection advisory council (SSAC) that the Booz-Allen team be selected for award. AR, exh. 50, MM, PMTO Brief to SSAC, Aug. 25, 1999, at 40. The SSAC reviewed the TEB's and CEB's findings, concurred with the TEB's conclusions, and recommended to the source selection authority (SSA) that the Booz-Allen team be selected for award. Id. at 1. The SSA reviewed the TEB's and CEB's findings and agreed with the SSAC's recommendation. Id., NN, Source Selection for the PMTO Contract, Aug. 27, 1999. On September 2, SPAWAR awarded five separate contracts, one to each member of the Booz-Allen team. AR, Oct 12, 1999, at 9. This protest followed a debriefing by the agency.

PROTESTER'S CONTENTIONS

ACS contends that the awards were improper on numerous grounds, primarily arguing that the agency's cost realism analysis of Booz-Allen's proposal was flawed. /4/ The protester specifically maintains that SPAWAR failed to properly consider Booz-Allen's allegedly unreasonable low hourly rates for certain labor categories, and that it failed to evaluate Booz-Allen's compensation structure. /5/ ACS further contends that Booz-Allen engaged in a prohibited "bait-and-switch" tactic. ACS further argues that SPAWAR misevaluated its team's technical proposal, especially under the corporate experience and past performance, and personnel resumes evaluation factors.

DISCUSSION

Cost Realism Analysis

ACS contends that SPAWAR failed to perform an adequate cost realism analysis of Booz-Allen's proposal. Specifically, ACS argues that SPAWAR failed to consider an in-house estimate of $400 million developed for the procurement, in light of Booz-Allen's lower direct hourly rates for certain labor categories and total MPC. In this regard, ACS argues that the agency failed to consider risks associated with the awardee's cost proposal because Booz-Allen will be unable to hire and retain its employees at the proposed rates. /6/

When an agency evaluates proposals for the award of a cost reimbursement contract, an offeror's proposed estimated costs of contract performance and proposed fees are not considered controlling since the offeror's estimated costs may not provide valid indications of the final actual costs that the government is required, within certain limits, to pay. See ManTech Envtl. Tech., Inc., B-271002 et al., June 3, 1996, 96-1 CPD Para. 272 at 8. Accordingly, a cost realism analysis must be performed when a cost-reimbursement contract is contemplated. Federal Acquisition Regulation (FAR) 15.404-1(d)(2). A cost realism analysis is the process of independently reviewing and evaluating specific elements of each offeror's proposed cost estimate to determine whether the estimated proposed cost elements are realistic for the work to be performed; reflect a clear understanding of the requirements; and are consistent with the unique methods of performance and materials described in the offeror's technical proposal. FAR Sec. 15.404-1(d)(1). The requirement to conduct a cost realism analysis of proposals for a cost-reimbursement contract does not require the agency to conduct an in-depth cost analysis, see FAR Sec. 15.404-1(c), nor the verification of each and every item in the conduct of such analysis. Rather, the evaluation of competing cost proposals requires the exercise of informed judgment by the contracting agency involved, since it is in the best position to assess the realism of cost and technical approaches and must bear the difficulties or additional expenses resulting from a defective cost realism analysis. Because the contracting agency is in the best position to make this cost realism determination, our review is limited to determining whether the agency's cost realism analysis is reasonably based and not arbitrary. The Warner/Osborn/ G&T Joint Venture, B-256641.2, Aug. 23, 1994, 94-2 CPD Para. 76 at 5. As explained in greater detail below, ACS's contention that the agency's cost realism analysis was flawed is without merit.

The RFP instructed offerors how to prepare their cost proposals. Specifically, section L-2 listed 33 labor categories and a corresponding number of hours for each, ranging from 2,000 to 140,000 estimated hours per year. RFP Sec. L-2(h), at 47-48. Offerors were required to propose a direct labor rate for each labor category at the number of hours specified for each category. In other words, offerors could not propose different labor categories or a different mix of hours from that stated in the RFP. The agency explains that the purpose of this requirement was to facilitate a cost comparison among the competing teams. AR at 6. All offerors, including subcontractors, were also required to submit current, actual, unloaded direct labor rates for all proposed key personnel. RFP Sec. L-2(a).

Regarding the cost evaluation, section M-3 of the RFP provided as follows:

The cost proposal must demonstrate realism and reasonableness. As part of the evaluation, the Government will assess the [MPC] of the Offeror's proposed approach. The realism evaluation will assess the compatibility of the overall proposed costs with the scope of effort to be performed. The evaluation will determine if the effort described in the Offeror's proposal as linked to the SOW is appropriately reflected in the Cost Proposal. The Offeror's costs, including direct and indirect rates, will be reviewed and DCAA will be contacted to determine if the rates proposed are current. In the event that DCAA can provide no information with regard to the offeror's current rates, the offeror may be requested to provide documentation that will verify that the rates proposed equal the offeror's actual current rates for personnel, fringe benefits, overhead, [general and administrative], and/or any other applicable indirect rates. Proposed costs may be adjusted for purposes of evaluation, based upon the results of the cost realism evaluation. This evaluation may include consideration of actual salaries being paid for similar work under other contracts, salaries being paid for comparable civil service employees, DCAA audit information and evaluation of compensation for professional employees. When this cost realism is performed, the resulting estimate or MPC shall be used in the evaluation of cost reasonableness. A cost proposal which is determined to be unrealistic, will be assessed as having high performance risk and the offeror's overall proposal may be downgraded.

The record shows that SPAWAR evaluated proposed costs for realism and reasonableness, and that the agency's analysis involved several steps. A cost realism analysis was performed on the costs proposed by each team member, except for those subcontractors that submitted a cost proposal valued at less than $500,000, or those subcontractors that submitted a proposal based on a fixed price. AR, exh. 52, SS, Business Clearance Memorandum, Aug. 30, 1999, at 10. Those proposals valued at less than $500,000 were accepted as proposed, since the agency considered that any adjustments would have been negligible in relation to the total estimated cost for each team. /7/

The CEB contacted DCAA and requested a rate review for all of Booz-Allen's team members, and for [DELETED] of the [DELETED] proposed subcontractors. If DCAA did not have any current rate information available for a particular firm, SPAWAR requested that an audit be conducted. Specifically, the agency requested DCAA to verify the direct labor rates, direct labor escalation, and indirect rates, and that the contractor had an adequate accounting system to perform a cost-type contract. The CEB also requested that DCAA verify the actual, current annual salaries for the proposed key personnel.

DCAA provided rate information for 16 of the 17 key personnel Booz-Allen proposed. /8/ AR, exh. 47, II. The CEB found that Booz-Allen submitted actual labor rates for the 17 key personnel in accordance with the RFP's instructions. DCAA verified that these direct rates were current as of April 1999, and took no exception to proposed indirect rates (fringe benefits, overhead, general & administrative, and cost of money). AR exh. 47, II, section 1, Fax from DCAA to SPAWAR, June 2, 1999.

The CEB further found that Booz-Allen proposed all key personnel at "100 percent participation" during the entire 5-year expected life of the PMTO contracts using their actual salaries. AR, exh. 48, JJ, Tab 2.

The agency explains that in those instances for non-key personnel where DCAA was unable to verify or make a recommendation regarding Booz-Allen's proposed labor rates, the CEB compared the proposed labor rate to labor rates published by BLS for the San Diego, California area for 1997. AR, Oct. 12, 1999, at 13 and exh. 52, SS, attach. 3. If the proposed labor rate was equal to or greater than the minimum rate of the corresponding BLS wage range, the CEB accepted the proposed labor rate for the purposes of cost realism. On the other hand, if the proposed labor rate was less than the minimum rate of the corresponding BLS wage range, the CEB adjusted the proposed rate upwards to the minimum of the range. /9/

ACS's contention that the agency did not consider any performance risks associated with Booz-Allen's cost proposal is without merit. After evaluating Booz-Allen's proposal for cost realism, and taking into account DCAA's recommendations, the CEB adjusted Booz-Allen's proposed costs upward from $251,785,403 to $257,161,635, or approximately two percent (ACS's costs were increased by less than one percent). The TEB considered the cost realism adjustment to Booz-Allen's proposal to be immaterial and concluded that it did not represent a performance risk. AR, exh. 52, SS, Business Clearance Memorandum, Aug. 30, 1999, at 11. The CEB found that the difference between Booz-Allen's proposed direct labor rates and the DCAA-recommended rates would result in a change of less than one percent to the total direct labor cost. Accordingly, the CEB concluded that for purposes of the cost realism evaluation, Booz-Allen's direct labor rates would be accepted as proposed. AR, exh. 48, JJ, Tab 2.

We find reasonable the methodology used by the agency to analyze Booz-Allen's costs for realism. Such an analysis involves predictions of future costs, and is thus by its nature imprecise. Accordingly, we think any methodology employed by an agency need only provide some measure of confidence that the costs proposed are realistic in view of other cost information reasonably available to the agency from its own and outside sources. See Radian, Inc., B-256313.2, B-256313.4, June 27, 1994, 94-2 CPD Para. 104 at 7. Here, the agency requested rate checks from DCAA, obtained specific information for virtually all labor categories, and evaluated proposals consistent with this information from DCAA. In this regard, an agency reasonably may use rate checks from DCAA in connection with a cost realism analysis. See Systems Research Corp., B-237008, Jan. 25, 1990, 90-1 CPD Para. 106 at 5.

Further, the protester's suggestion that the agency failed to consider risks associated with Booz-Allen's compensation structure because the protester believes that Booz-Allen will be unable to hire and retain employees in the San Diego area at the rates proposed, is similarly lacking in merit. As already explained, SPAWAR obtained DCAA's verification that Booz-Allen's proposed rates for key personnel were the current rates being paid by the firm, its team members and proposed subcontractors, and the CEB concluded that those rates were realistic. In the absence of evidence showing that the rates Booz-Allen proposed were unrealistically low, the agency could properly rely upon DCAA's advice in performing its cost realism analysis. Systems Integration & Research, Inc.; Presearch Inc., B-279759.2, B-279759.3, Feb. 16, 1999, 99-1 CPD Para. 54 at 12. With respect to non-key personnel, as discussed above, the CEB compared the proposed costs with the corresponding BLS rates and reasonably concluded that Booz-Allen's rates were sufficiently high to hire and retain its personnel. Under these circumstances, there is no basis to conclude that the agency failed to consider risks associated with Booz-Allen's proposed rates.

ACS next complains that SPAWAR did not compare Booz-Allen's proposed costs with the significantly higher government estimate of the work. SPAWAR responds that the figure ACS refers to was not an "independent government estimate" (IGE), as ACS contends, but a preliminary estimate based on information obtained in the early stages of the PMTO procurement, developed solely for planning and budgeting purposes, before SPAWAR had definitized the number of labor hours, labor categories, or labor mix described in the RFP. AR, Nov. 8, 1999, at 10-12 and exh. 56, Tab JJJ, Declaration of SPAWAR Supervisory Procurement CO, Nov. 8, 1999, Paras. 9-10. In any event, even assuming that the $400 million estimate could be considered SPAWAR's IGE, there is no requirement that the agency compare an offeror's proposed costs with the government estimate. See ELS, Inc., B-283236, B-283236.2, Oct. 25, 1999, 99-2 CPD Para. ___ at 10; EDAW, Inc., B-272884, Nov. 1, 1996, 96-2 CPD Para. 213 at 5-6; Energy and Envtl. Servs. Corp., B-258139.4, May 15, 1995, 95-2 CPD Para. 32 at 4. Rather, the agency may employ various types of analyses in determining what, in the agency's view, it would realistically cost the offeror to perform given the offeror's technical approach. Here, as described above, the agency performed a detailed and reasonable cost realism evaluation of Booz-Allen's costs. The fact that Booz-Allen's proposed costs were less than SPAWAR's preliminary estimate does not establish that Booz-Allen's costs were unrealistic. See, e.g., Department of State--Recon., B-243974.4, May 18, 1992, 92-1 CPD Para. 447 at 7; United Eng'rs & Constructors Inc., Stearns-Roger Div., B-240691, B-240691.2, Dec. 14, 1990, 90-2 CPD Para. 490 at 6. In fact, the record shows that all of the offerors' costs, including ACS's, were less than that estimate even after adjustments for cost realism purposes. In sum, we find that ACS's arguments do not provide us with any basis to question the agency's evaluation of Booz-Allen's cost proposal.

"Bait and Switch"

ACS contends that Booz-Allen engaged in an impermissible "bait and switch" tactic because it appeared that it intended to substitute key personnel after award. In support of this allegation, ACS states that within a few days after award, a "headhunter" representing Booz-Allen solicited ACS employees at the protester's facilities. ACS states that it immediately reported the incident to SPAWAR.

To establish an improper "bait and switch," a protester must show that the firm either knowingly or negligently made a misrepresentation regarding employees that it does not expect to furnish during contract performance, that the misrepresentation was relied upon in the evaluation, and that this had a material impact on the evaluation results. USATREX Int'l, Inc., B-275592, B-275592.2, March 6, 1997, 98-1 CPD Para. 99 at 9-10. There is no evidence of a prohibited bait and switch here.

The agency states that it immediately conducted an investigation into ACS's allegations and found no basis for concluding that Booz-Allen knowingly or negligently misrepresented its intent to furnish the employees specified in its proposal. During its investigation into this allegation, SPAWAR learned that the alleged "headhunter" was neither a personnel recruiter, nor an employee of Booz-Allen or of any other awardee of a PMTO prime contract. SPAWAR reports that the individual was merely an engineer employed by a subcontractor to one of the members of Booz-Allen's team. SPAWAR's investigation further revealed that, while the individual in question did contact several ACS employees concerning potential opportunities with her employer, this was done on her own volition, without Booz-Allen's knowledge or consent. The individual has provided a signed statement further affirming that she has "never acted on behalf of [Booz-Allen] in a business development or personnel recruiting capacity." AR, exh. 53, WW, Statement of [DELETED], Sept. 10, 1999. Finally, as a result of SPAWAR's investigation into ACS's allegation, Booz-Allen confirmed that all of Booz-Allen's proposed key personnel are currently employed either by Booz-Allen or subcontractors of its team members. AR, exh. 53, XX. In this regard, the record contains signed statements in which all proposed key employee attest to their continued availability and dedication to performing the contract. Under these circumstances, there simply is no basis upon which to conclude that Booz-Allen made any misrepresentations regarding employees that it did not expect to furnish during contract performance. As such, we conclude that the record does not support ACS's allegation that Booz-Allen engaged in an improper "bait and switch" tactic in this procurement.

Second Supplemental Protest

In a second supplemental protest, ACS alleges that SPAWAR did not conduct or document the required risk assessment of Booz-Allen's technical and management approach. In this connection, ACS asserts that the PMTO solicitation required SPAWAR to perform more than a risk assessment limited to costs, arguing that SPAWAR did not follow the "multi-step" risk assessments established by the solicitation associated with non-cost factors. ACS Second Supplemental Protest, Oct. 25, 1999, at 4.

Where a protester files supplemental protest grounds, each new ground must independently satisfy the timeliness requirement of our Bid Protest Regulations, which do not contemplate the piecemeal presentation or development of protest issues. QualMed, Inc., B-257184.2, Jan. 27, 1995, 95-1 CPD Para. 94 at 12-13. This includes the identification of "examples" of flaws in the evaluation generally alleged in the initial protest. Id. Such new issues must be filed within 10 calendar days after the protester knew or should have known the basis for its protest. Bid Protest Regulations, 4 C.F.R. Sec. 21.2(a)(2). The fact that an initial protest includes a general allegation of impropriety provides no support for the timeliness of more specific allegations. GE Gov't Servs., B-235101, Aug. 11, 1989, 89-2 CPD Para. 128 at 4.

ACS argues that its second supplemental protest, filed October 25, is timely because it was filed within 10 days of when ACS learned of the inadequate documentation surrounding SPAWAR's risk assessment. ACS Comments, Nov. 17, 1999, at 29-30. ACS further argues that its initial protest allegations "clearly encompass SPAWAR's flawed risk assessment as required by the PMTO solicitation." Id. at 30. The protester then points out that its initial protest referenced various solicitation provisions that allegedly required SPAWAR to conduct a multi-step risk assessment, thus arguing that its second supplemental protest is "in addition to, and not in substitution for ACS's" initial protest. Id. at 31. ACS asserts that its initial protest was broadly worded to subsume an assessment of technical risk.

In its initial protest, ACS titled the first general ground of protest in broad terms, "SPAWAR Relaxed The Solicitation's Risk Assessment Evaluation Requirements for [Booz-Allen's] Benefit." ACS Protest, Sept. 13, 1999, at 9. ACS then quoted several sections of the RFP which allegedly required the agency to conduct risk assessments. For instance, ACS stated that "[u]nder Corporate Experience and Past Performance, the Solicitation stated that the performance risk assesses the probability of the Offeror successfully accomplishing the proposed effort based on the Offeror's demonstrated past and present performance.'" Id. Except for quoting from the RFP, however, ACS did not articulate any specific challenge to SPAWAR's risk assessment of Booz-Allen's technical proposal; nor did the firm provide any factual basis in support of its general allegation. Instead, ACS quoted several other RFP sections regarding cost risk assessments. ACS then stated that it proposed [DELETED] id. at 10, and that Booz-Allen did not propose actual direct labor rates in its proposal. These were the only specific allegations ACS made in its initial protest regarding SPAWAR's risk assessment of Booz-Allen's proposal. Because of the general nature of ACS's initial protest ground, SPAWAR was unable to respond, except for addressing the cost realism analysis, and did not address any risk assessment of Booz-Allen's technical proposal.

ACS then supplemented its initial protest on October 25, arguing that SPAWAR did not conduct the required risk assessment of Booz-Allen's technical and management approach. ACS stated that it first learned of the grounds for this protest from reviewing SPAWAR's document production of October 13, 21, and 22, in response to its initial protest which, according to ACS, failed to show "how SPAWAR analyzed the performance and other risks inherent in the proposal submitted by Team [Booz-Allen]." Second Supplemental Protest, Oct.25, 1999, at 4.

In our view, ACS's general allegation in its initial protest--that SPAWAR relaxed the RFP's risk assessment evaluation for Booz-Allen's benefit, without more--simply cannot reasonably be read to encompass a challenge to the risk assessment of Booz-Allen's technical proposal. As already explained, this portion of the protest consisted primarily of various quotations from the RFP, mainly having to do with the assessment of risks associated with costs, and assertions by ACS that it proposed [DELETED] while Booz-Allen proposed hypothetical rates. In its comments on the agency reports responding to ACS's initial and first supplemental protests, ACS stated that "SPAWAR confirmed at ACS' debriefing that SPAWAR improperly evaluated Team [Booz-Allen's] technical proposal." ACS Comments, Oct. 28, 1999, at 7. In support of its position that its second supplemental protest is timely, ACS repeats this point in its comments on the agency report responding to the second supplemental protest--that its "initial protest attacked SPAWAR's acknowledged failure to conduct the required RFP 'multi-step' risk assessment (based on what ACS was told by SPAWAR at the debriefing regarding SPAWAR's failure to conduct the risk assessments required by the solicitation)." ACS Comments, Nov. 17, 1999, at 29. The protester's own statements thus show that it learned of the basis of its second supplemental protest from the information it received at its September 9, 1999 debriefing. ACS was required, therefore, to provide the factual and legal basis for this allegation in its initial protest.

The fact that the title of the first general allegation in ACS's initial protest was broadly worded to subsume an assessment of technical risk, as ACS contends, is not sufficient. As already explained, ACS provided no reference to Booz-Allen's technical proposal nor explained the factual basis for its allegation in the initial protest. Including such a broadly stated allegation in an initial protest does not permit the protester to later present any specific, and otherwise untimely, argument having some relevance to that initial general allegation. See Global Eng'g and Constr. Joint Venture, B-275999.4, B-275999.5, Oct. 6, 1997, 97-2 CPD Para. 125 at 4; GE Gov't Servs., supra at 5. This timeliness rule is clearly applicable where, as here, the protester asserts that SPAWAR provided it with the information that formed the basis for this protest ground at its debriefing. An offeror who receives specific information in its debriefing, but ignores it when drafting its initial protest, does so at its peril. Cornet, Inc., Datacomm Management Servs., Inc., B-270330, B-270330.2, Feb. 28, 1996, 96-1 CPD Para. 189 at 4. Thus, if ACS concluded from its debriefing that SPAWAR failed to conduct the allegedly required "multi-step" risk assessment of Booz-Allen's technical proposal, to be timely, ACS was required to provide the legal and factual basis for this protest ground, at the latest, within 10 days of its September 9 debriefing, or by September 19. Since ACS did not raise this issue with any specificity until it filed its second supplemental protest on October 25, 1999, 47 days after its debriefing, this issue is untimely and will not be considered. 4 C.F.R. 21.2(a)(2); Ralph G. Moore & Assocs., B-270686, B-270686.2, Feb. 28, 1996, 96-1 CPD Para. 118 at 2-3 n.2, recon. denied, B-270686.3, June 5, 1996, 96-1 CPD Para. 268.

Evaluation of ACS's Technical Proposal

ACS contends that SPAWAR improperly assigned its proposal an overall rating of "good," rather than "excellent," primarily due to the agency's unreasonable evaluation under two factors--corporate experience and past performance, and personnel resumes. ACS maintains that its proposal was entitled to a rating of "excellent" under the corporate experience and past performance factor because more than half of the references consulted rated its team's past performance as [DELETED]. The protester also notes that ACS itself held three of the previous service contracts that were consolidated under the PMTO RFP. With respect to the evaluation of its proposal under the personnel resumes factor, the protester argues that the resumes ACS included in its proposal for key personnel exceeded SPAWAR's evaluation criteria, thus its proposal should have received a rating of "excellent" under this factor.

Our Office will not engage in an independent evaluation of proposals nor make an independent determination of their relative merits. Litton Sys., Inc., B-239123, Aug. 7, 1990, 90-2 CPD Para. 114 at 9. Rather, we review the agency's evaluation only to ensure that it was reasonable and consistent with applicable statutes and regulations as well as with the terms of the solicitation. Sensis Corp., B-265790.2, Jan. 17, 1996, 96-1 CPD Para. 77 at 6. A protester's mere disagreement with the agency's conclusions does not render the evaluation unreasonable. ESCO, Inc., B-225565, Apr. 29, 1987, 87-1 CPD Para. 450 at 7. Based on our review of the record, we conclude that the evaluation of ACS's proposal is reasonably supported.

Corporate Experience and Past Performance

The RFP required offerors to demonstrate their recent experience with work of a nature, scope, magnitude, complexity, and difficulty similar to that contemplated under the solicitation. RFP Sec. L Para. 4.4, at 61. The RFP explained that the objective of the evaluation in this area was to determine the degree to which the offeror has previously encountered the kinds of work, uncertainties, challenges, and risks that they are likely to encounter under the contemplated contract. Id. The performance evaluation was to determine the government's level of confidence in the offeror's ability to perform based on that offeror's work record. To assist the agency in the evaluation of past performance, offerors were required to submit completed Past Performance Data Sheets provided as attachments to the RFP to describe in detail the past performance of up to three relevant contracts (ongoing or completed within the past 3 years) referenced for each team member. Id.

The criteria the TEB applied for assigning proposal ratings of either "excellent" or "good" in this area stated as follows:

EXCELLENT: Outstanding in all respects, consistently met and frequently exceeded customer expectations. All team members have a wide-ranging breadth and depth of experience in performing similar efforts for comparable customers. The team has the highest probability of meeting the Government's requirements.

GOOD: There are a few problem areas reported by current or former customers; however, they were reported as corrected promptly and/or to the customer's satisfaction. Many of the team members have performed similar efforts for comparable customers. The team shows a high probability of meeting the Government's requirements.

AR, exh. 52, LL, TEB Summary Report for PMTO Contract, Aug. 24, 1999, at 4.

The TEB assigned an adjectival rating of "good" to ACS's proposal under the corporate experience and past performance evaluation factor. The TEB evaluated all of the contracts provided by ACS, including its team members and subcontractors, based upon their similarity to the five functional areas contemplated by the RFP. Id. at 16. Based on its evaluation, the TEB found that all of ACS's team members had at least [DELETED] experience within all of the five functional areas identified in the RFP.

The responses the TEB obtained to the past performance questionnaires /10/ on ACS indicate that while they generally were positive, the TEB could reasonably conclude that the team was not "[o]utstanding in all respects," and that it failed to "consistently [meet] and frequently exceeded customer expectations," as required to earn an "excellent" rating. For instance, on four of the six questions, respondents rated the team as "excellent" on [DELETED] percent of the responses; of the two remaining items (quality and business relations), [DELETED] percent of the responses were "excellent" while [DELETED] percent were only "good." In view of the criteria described above, the responses the agency obtained to the performance questionnaires reasonably support the agency's evaluation in this area.

With respect to the similarity of the referenced contracts to the work contemplated by the RFP, the TEB found that the contracts of the ACS team were rated "highly similar" in all five areas; the composite of all responses was [DELETED] percent "excellent" and [DELETED] percent "good." When viewed as a whole, however, these ratings reflect the respondents' assessment that [DELETED]. In addition, the agency explains that the three contracts ACS referenced for itself did not require the scope of management similar to its responsibility under the PMTO RFP. In this connection, Booz-Allen points out that under the PMTO contract, ACS would be responsible for managing [DELETED] other contractors, including small businesses which comprise [DELETED] percent of the total contract value. Booz-Allen Comments, Oct. 28, 1999, at 42. Booz-Allen also points out, and the record shows, that the three contracts ACS identified as references for itself were valued at considerably less than the value of the portion of the contract ACS assigned itself under the PMTO RFP. Contrary to ACS's suggestion, performance on three smaller contracts with significantly different management responsibilities and lesser in scope, is not a definitive indicator of a firm's corporate ability to perform one contract, larger in complexity, scope and magnitude as that contemplated under the PMTO RFP. See, e.g., Chem-Services of Ind., Inc., B-253905, Oct. 28, 1993, 93-2 CPD Para. 262 at 4. In sum, applying the evaluation criteria the evaluators used to rate this area, we think that the TEB's rating of "good" is reasonably supported.

Personnel Resumes

Offerors were required to submit resumes for 17 key personnel comprising five labor categories (program manager, deputy program manager, senior project manager, project manager, and project administrator) and several non-key personnel. RFP Sec. L, Para. 4.5, at 62. The RFP required that the resumes specifically identify the education and pertinent experience of the proposed personnel, demonstrating that they were capable of performing the area of the SOW to which they were assigned. The SOW specifically stated that all personnel must have experience with Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance (C4ISR) systems, or other experience in Navy systems acquisition and life cycle support. RFP attach. 2, SOW, Sec. 3.0.2. In addition, the SOW stated that the Program Manager and Project Manager were to have a minimum of 15 years experience with complex Navy C4ISR programs, at least 5 of which were in positions having authority over the management, formulation, design, test, and evaluation of complex C4ISR programs. Id. Sec. 3.0.2.c. The SOW further stated that the Program Manager and the Project Manager should have a thorough knowledge of the Department of Defense (DOD)/Navy budgeting and acquisition process. Id. The RFP stated that substituted experience should be recent and clearly relevant to the work proposed, and that SPAWAR would not infer or presume qualifications. RFP Sec. L, at 53-54.

ACS contends that the TEB unreasonably evaluated its personnel resumes because they exceeded the RFP's requirements. ACS also argues that the TEB unreasonably downgraded its proposal in this area because the TEB improperly equated the requirement for key personnel to have a thorough knowledge of DOD acquisition with having actual experience.

The TEB assigned ACS's proposal an adjectival rating of "average" under the personnel resumes factor. The TEB identified no strengths in the proposal in this area and several weaknesses. For example, the TEB found that, contrary to the RFP requirements, the resumes consistently indicated that ACS's proposed personnel [DELETED]. Of particular concern to the TEB was the [DELETED] demonstrated for the proposed program manager and the deputy program manager for operations, both of which were identified as key positions in the RFP. AR, exh. 50, LL, TEB Summary Report for PMTO Contract, Aug. 24, 1999, at 17. The TEB further found that, with one exception, notwithstanding the RFP's specific instructions, ACS's proposed senior project managers' resumes [DELETED]. Id. In addition, one project manager's resume indicated [DELETED] background, but little in the way of [DELETED]. The TEB found that another project manager's experience was not tied to SPAWAR or to C4ISR, and that individual's resume also [DELETED]. Since the SOW required that all personnel have experience in Navy systems acquisition, ACS's argument that the TEB unreasonably evaluated these individuals' resumes is without merit.

With respect to project administrator positions, which were also designated as key personnel, RFP Sec. L-2(e), the solicitation required that these individuals have a bachelor's degree from an accredited college or university, and expressed a preference for degrees in computer science, engineering, mathematics, or business administration. Alternatively, 4 years of equivalent, recent experience in clearly relevant work could be substituted. RFP attach. 2, SOW, Sec. 3.0.2.e. Notwithstanding these specific requirements, the TEB found that [DELETED]. Given the RFP's specific instructions and requirements concerning key personnel, we have no basis to object to the rating assigned ACS's proposal in this area.

Finally, the TEB found sufficient errors in the resumes in ACS's proposal--specifically regarding dates of employment for key personnel--to cause the evaluators concern as to the ability of the team to provide quality deliverables. In this regard, offerors have the burden of submitting adequately written proposals, and an offeror's disagreement with the agency's judgment is not sufficient to establish that the agency acted unreasonably. Caldwell Consulting Assocs., B-242767, B-242767.2, June 5, 1991, 91-1 CPD Para. 530 at 6. In sum, the record shows that ACS's proposal earned ratings of "average" under three evaluation factors; and ratings of "good" and "excellent" under the remaining two factors. Based on these ratings, which are reasonably supported, the TEB's assigning an overall rating of "good" to ACS's proposal is unobjectionable.

The protest is denied.

Comptroller General of the United States

1. The agency explains that the RFP consolidates seventeen support services contracts, referring to this concept as the Program Management Team Omnibus (PMTO) RFP. The PMTO RFP called for services in the following five functional areas: project management, engineering services, system test and evaluation, site/platform installation support, and integrated logistics support. RFP attach. 2, Para. 1.0.

2. In this decision, unless indicated otherwise, we use the names ACS and Booz-Allen to refer to the teams headed by those companies.

3. With respect to costs, the RFP instructed offerors to submit one cost proposal, which was to contain the rates proposed for the base year and each option period for each team member and for all subcontractors. Id. Sec. L.5.5, at 64.

4. Analogizing to several decisions of our Office where we have stated that a joint venture, not any individual firm, is the appropriate "interested party" to protest the contracting agency's action, see, e.g., Comark Bldg. Sys., Inc., B-259515, Apr. 10, 1995, 95-1 CPD Para. 188 at 2 n.1; H.J. Group Ventures, Inc. B-246139, Feb. 19, 1992, 92-1 CPD Para. 203 at 1 n.1; Robert R. Nathan Assocs., Inc., B-230707, June 28, 1988, 88-1 CPD Para. 615 at 1 n.1., SPAWAR and the intervenor requested that we dismiss the protest, arguing that ACS is not an interested party because it did not file the protest on behalf of "Team ACS." We are not persuaded by these arguments. There was no requirement for team members to create a joint venture in order to compete, and ACS is an actual offeror whose direct economic interest was affected by not being awarded a PMTO contract. See 4 C.F.R. Sec. 21.0(a) (1999). In any event, ACS has provided signed statements from each of its team members agreeing that ACS should pursue the protest on their behalf. ACS Comments to SPAWAR's Reply to ACS's Opposition to SPAWAR's Request for Dismissal, Oct. 5, 1999, attach. 2. Based on the agency's arguments, however, ACS supplemented its protest, arguing that the agency misled it during the procurement by not announcing that SPAWAR was seeking proposals from joint ventures, and alleging that the teaming approach contemplated by the RFP was improper. SPAWAR's analogy to joint venture cases in support of its request for dismissal provides no basis to conclude that SPAWAR was seeking proposals from joint ventures and there is no other evidence in the record to support ACS's contention. To the extent that ACS challenges the RFP's teaming approach, this issue, raised for the first time in its first supplemental protest, filed on September 28, 1999, well after closing, is untimely and will not be considered. 4 C.F.R. Sec. 21.2(a)(1).

5. In its initial protest, ACS also argued that the agency improperly applied unstated evaluation criteria, and that SPAWAR should have conducted discussions. Following receipt of the agency report, ACS withdrew these allegations. ACS Comments, Oct. 28, 1999 at 5 n.2.

6. In its initial protest, ACS alleged that SPAWAR's cost realism analysis of Booz-Allen's proposal was flawed because SPAWAR applied an improper escalation rate, "mismapped" some of the labor categories, and failed to take into account that Booz-Allen proposed uncompensated overtime. SPAWAR responded to these allegations and ACS did not rebut the agency's response. Accordingly, we consider these issues abandoned. Appalachian Council, Inc., B-256179, May 20, 1994, 94-1 CPD Para. 319 at 8 n.8.

7. The agency reports that due to the size of the procurement and the teaming approach, the CEB performed a total of 131 separate cost realism analyses. AR, Oct. 12, 1999, at 6.

8. For the one key person for whom DCAA could not provide rate information, the CEB relied on Department of Labor, Bureau of Labor Statistics (BLS) information to assess the realism of that individual's proposed labor rate. The CEB's comparison shows that the proposed labor rate was higher than that of a similar BLS labor rate. The CEB thus determined that individual's rate was realistic and made no adjustments to Booz-Allen's proposal with respect to this individual.

9. The CEB's use of the BLS data was a reasonable method of assessing the realism of Booz-Allen's labor rates and it was consistent with the RFP's stated evaluation scheme. By comparing the proposed labor rates of non-key personnel to the BLS rates for each category, the agency concluded that Booz-Allen's proposed rates were reasonable, realistic, and sufficiently high to hire and retain the necessary personnel in the San Diego area. See, e.g., ENMAX Corp., B-281965, May 12, 1999, 99-1 CPD Para. 102 at 10.

10. The questionnaire contained six items concerning the following areas: quality, timeliness, experience and technical capability of personnel, cost control, business relations, and overall customer satisfaction. For each item, respondents could assign adjectival ratings of either marginal, average, good, excellent or "N/A" for not applicable.