Matter of: Future-Tec Management Systems, Inc.; Computer & Hi-Tech Management, Inc. File: B-283793.5; B-283793.6 Date: March 20, 2000

Matter of: Future-Tec Management Systems, Inc.; Computer & Hi-Tech Management, Inc. File: B-283793.5; B-283793.6 Date: March 20, 2000

DIGEST

Attorneys

DECISION

Future-Tec Management Systems, Inc. and Computer & Hi-Tech Management, Inc. (CHM) protest the Department of the Navy's award of a contract to Systems Engineering and Security, Inc. (SES) under request for proposals (RFP) No. 00140-99-R-G413, issued by the Navy Fleet and Industrial Supply Center (FISC)-Norfolk Detachment Philadelphia. Both Future-Tec and CHM protest that the Navy's evaluation of technical proposals and the resulting source selection were improperly performed, and allege that SES proposed key personnel that it knew or should have known would not work full time on the project, as required by the RFP. In addition, CHM protests that the Navy's cost realism analysis was improper, and resulted in an unreasonable and arbitrary increase in CHM's evaluated proposed costs.

We sustain the protests.

The RFP was issued on June 25, 1999, by the FISC office on behalf of the Navy Reserve Information Systems Office and the Systems Executive Office for Manpower and Personnel, both located in New Orleans, Louisiana. The RFP explained that these offices have been tasked to develop world class software products and to market their capability to new customers, and sought offers for a broad range of automated information system support and infrastructure services for these offices.

The RFP, issued as a competitive procurement set aside for small disadvantaged businesses under the Small Business Administration's 8(a) program, contemplated the award of a cost-plus-fixed-fee (CPFF), indefinite-delivery/indefinite-quantity contract, under which services would be ordered by delivery or task orders. The RFP provided that the government intended to make an award to the offeror whose proposal presented the best value to the government, and stated that a proposal's technical merit would be considered more important than cost. The solicitation listed the following technical evaluation factors in descending order of importance, stating that the first two would be considered equal in weight: corporate experience, past performance, technical approach, management plan, and personnel resources. RFP Sec. M.

Offerors were instructed to prepare their technical proposals in two volumes, with the first volume to include written descriptions of the offeror's past performance and its personnel resources, and the second to include transparencies and slides that would be used during oral presentations, covering technical approach, management approach, and corporate experience. RFP Sec. L.I.

With respect to past performance, the RFP instructed offerors as follows (in relevant part):

[D]escribe . . . past performance on directly related or similar contracts . . . held within the last five (5) years which are of similar scope, magnitude and complexity to that which is detailed in the RFP. Offerors who describe similar contracts shall provide a detailed explanation demonstrating the similarity of the contracts to the requirements of the RFP. In determining the rating for the past performance evaluation factor, the Government will give greater consideration to the contracts which are most relevant to the RFP.

RFP Sec. L.III.A.1, at 51.

The personnel resources factor listed key personnel resumes and staffing as subfactors and stated that they were of equal importance. The RFP listed the 11 key personnel full-time equivalent positions for which the offerors were required to submit resumes. RFP Sec. L.III.A.2, at 52. It also provided that resumes for key personnel must indicate whether the proposed employee was currently employed by the offeror, and required offerors to certify that proposed key personnel had confirmed their availability for contract performance. Id.

The second volume was to consist of slides that the offeror would use in an oral presentation addressing its technical approach, management plan, and corporate experience. The RFP offered some guidance for preparing proposals in each of these areas and cautioned offerors to provide sufficient detail in their proposals. Regarding corporate experience, offerors were instructed to "demonstrate either similar or directly related work experience of similar scope, magnitude and complexity to the [statement of work]. The offeror should address the following: history, organization, qualifications and work experience within the last five (5) years as they relate to the requirements of the [statement of work]." RFP Sec. L.III.B.3, at 52.

Offerors were required to provide a detailed price/cost breakdown as the third volume to their proposals. The RFP stated that costs would be evaluated for cost realism. RFP Sec. M.3, at 56.

Five offerors submitted proposals by the August 12, 1999 closing date. A five-member technical evaluation committee (TEC) reviewed the initial proposals and prepared a written technical evaluation. The TEC scored the technical proposals under the following adjectival rating scale: highly acceptable (HA); acceptable (A); unacceptable, but with the capacity of being made acceptable as a result of discussions (U(a)); and unacceptable with no reasonable chance of being selected for award (U(b)). The TEC's rating summary for the SES, CHM and Future-Tec proposals was as follows:

Offeror Past. Corp. Exp. Tech. App. Mgt. Plan Pers. Perf. Res.

SES HA HA HA A U(a)

CHM HA A A A U(a)

Future-Tec HA A A A U(a)

Agency Report, Tab 16, Technical Evaluation of Proposals, at 15.

The technical evaluation for each of these offerors began with the following summary:

[The offeror's] overall rating is Unacceptable (a) due to a rating of Unacceptable (a) in the Personnel Resources factor.

Id. at 1, 4, 9.

Although the evaluation report provides a separate evaluation for each offeror's proposal, the comments for the three proposals are often identical for CHM and Future-Tec and provide little or no detail or explanation of any differences between the different proposals. For example, the following comments are all that appears in the report for the following evaluation factors for each of these two proposals:

Corporate Experience: [The offeror] demonstrated corporate experience on contracts of similar scope, magnitude and complexity with the past 5 years to that detailed in the RFP. This factor is rated acceptable.

Past Performance: [The offeror] described its past performance on contracts of similar scope, magnitude and complexity to that detailed in the RFP. The [contracting officer or contract performance report on a cited contract] rated [the offeror] exceptional in all evaluation areas. This history of quality past performance mitigates the risk to the Government on this effort in the areas of cost, schedule and performance in meeting the requirements of this RFP. This factor is highly acceptable.

Technical Approach: [The offeror] has submitted a technical approach which demonstrated an understanding of the [statement of work]. This factor is rated acceptable.

Management Plan: [The offeror] has submitted a management approach which demonstrated an understanding of the [statement of work]. This factor is rated acceptable.

Personnel Resources: This factor is rated as unacceptable (a).

Staffing: [The offeror's] staffing plan meets the requirements of the RFP. This subfactor is rated acceptable.

Id.

For key personnel resumes, the report listed the resumes each offeror had submitted, noting which ones failed to demonstrate compliance with requirements in the RFP. Id. at 2-3, 5-6, 10-11.

The evaluation for SES was identical to the others for the factors of past performance, management plan, personnel resources, and for the key personnel resumes subfactor. With respect to corporate experience, the TEC made the same initial statement--i.e., that SES demonstrated experience on contracts of similar scope, magnitude and complexity--but then continued as follows:

Further, SES's education profile is noteworthy, demonstrating that [deleted]% of employees have [deleted] advanced degrees. SES also identified work experience of an exceptional magnitude and diversity, for example, contract number [deleted] (a $73 million dollar contract for multi-disciplinary IT services with a staffing level of [deleted] FTE). This varied experience reduces risk to the Government in the execution of this RFP. This factor is rated highly acceptable.

Id. at 9.

With respect to staffing, the report stated as follows:

This proposal outlines a detailed and thorough transition plan that amounts to a blueprint for the whole transition process. This plan addressed a comprehensive approach to staff development, recruiting, retention, and transition of incumbent personnel. This staffing plan ensures a low-risk, seamless transfer of control. SES staffing plan is rated highly acceptable.

Id. at 10.

Based on the TEC's evaluation and her own review of the proposals, the contracting officer determined that only the Future-Tec, CHM and SES proposals would be included in the competitive range. The competitive range offerors were permitted to make oral presentations and to answer questions posed by the TEC. At a hearing held by our Office to complete the record in this protest, the contracting officer stated that no written record was made of the oral presentations or the ensuing discussions. Hearing Transcript (Tr.) at 84. The contracting officer conducted written technical discussions, notifying the three competitive range offerors of the deficiencies found in each of their initial proposals in the area of key personnel and requesting revised technical proposals.

The TEC reviewed the revised proposals and prepared a technical evaluation. The new report differed from the initial report only in the evaluation of key personnel resumes, for which each proposal was now rated acceptable. Overall, SES was rated highly acceptable, while CHM and Future-Tec were rated acceptable. SES was ranked first, and CHM and Future-Tec were rated technically equal in second place. Agency Report, Tab 24, Revised Technical Evaluation, at 8. The three offerors were informed that their proposals were now considered technically acceptable, and offerors were apprised of the areas in their cost proposals that, based on a cost realism analysis performed by the contracting officer and contract negotiator, were considered to be understated. Offerors were invited to submit best and final offers (BAFO). The contracting officer reminded offerors that revised proposals had to include a certification that proposed employees had been contacted after the date of the letter requesting final proposal revisions, and that all proposed employees had confirmed that they were available for contract performance. Agency Report, Tabs 25, 26-27, Requests for Final Proposal Revisions, at 2.

SES and CHM submitted revised cost proposals, while Future-Tec resubmitted its costs as initially proposed. In its BAFO, CHM slightly [deleted] its overhead rate and its fee. It also proposed a [deleted] in order to reduce the risk of [deleted]. Agency Report, Tab 29, CHM BAFO, at 2.

In its BAFO, CHM proposed a total CPFF of $[deleted]. For reasons that will be discussed below, the Navy adjusted this figure to $[deleted] based on its cost realism analysis. SES's costs were also adjusted, and were evaluated at $83,666,971.37. /1/ Future-Tec's costs remained unadjusted at $102,293,791.65. Agency Report, Tab 33, Post-Negotiation Business Clearance Memorandum, at 23-24.

The contracting officer, acting as the source selection authority, analyzed the technical and cost ratings of the three proposals and selected SES for award. In her source selection decision memorandum, the contracting officer summarized the procurement process (as described above) and essentially repeated the comments in the TEC's evaluation report in a narrative paragraph discussing each technical proposal. /2/ Regarding the source selection, the contracting officer noted that Future-Tec's proposal was considered in third place, based on its combined technical and cost rating. Comparing the remaining two proposals, the memorandum notes that SES's proposal was rated more highly on technical merit and was priced [deleted] percent higher than CHM's, and refers to a cost/technical tradeoff analysis for which the only documentation is the contracting officer's conclusory statement in her source selection memorandum that "the substantial technical superiority of the SES proposal outweighs any benefit that would be gained from CHM's lower cost proposal." Agency Report, Tab 31, Source Selection Decision Memorandum, at 6. Comparing the two proposals, the contracting officer notes that both offerors were rated "highly acceptable" under past performance, based on their performance under prior contracts, but that under corporate experience, SES was rated highly acceptable while CHM was rated only acceptable. The contracting officer repeated the TEC report's reference to SES's "work experience of exceptional magnitude and diversity," adding that "[n]o other proposal demonstrated any experience of comparable magnitude, and the Navy places great value on the magnitude as well as the nature of this experience." She again noted that SES's proposal was rated highly acceptable in technical approach, repeating the TEC's comment that SES "demonstrated in more than sufficient detail a technical approach that will successfully accomplish the [statement of work]," and offering as the sole example to support this rating "SES's exceptional demonstration of how their risk mitigation techniques and methods had been successfully used on prior contracts and how those techniques would be applied in the instant requirement." Id.

The contracting officer awarded the contract to SES on September 23. Future-Tec requested a debriefing the following day. The debriefing was held on September 28, and this protest followed on September 29. CHM was debriefed on September 29 and filed its protest in our Office on October 1. On October 1, the head of the contracting activity determined to override the stay of performance that was required by the Competition in Contracting Act of 1984, finding that continued performance of the contract was in the best interests of the government. See 31 U.S.C. Sec. 3553(d)(3)(C)(i)(I) (1994).

THE TECHNICAL EVALUATION

Both Future-Tec and CHM protest that its own proposal merited a higher technical rating than it received under the Navy's evaluation, and that SES's proposal, in contrast, was rated too highly.

In reviewing an agency's evaluation of proposals and source selection decision, our review is confined to a determination of whether the agency acted reasonably and consistent with the stated evaluation factors and applicable procurement statutes and regulations. Main Bldg. Maintenance, Inc., B-260945.4, Sept. 29, 1995, 95-2 CPD Para. 214 at 4. An agency's evaluation of proposals and source selection decision must be documented in sufficient detail to allow for the review of the merits of a protest. Southwest Marine, Inc.; American Sys. Eng'g Corp., B-265865.3, B-265865.4, Jan. 23, 1996, 96-1 CPD Para. 56 at 10. An agency that fails to adequately document its evaluation of proposals and source selection decision in sufficient detail to show that it is not arbitrary bears the risk that our Office will not conclude that the agency had a reasonable basis for its determinations. Engineering and Computation, Inc., B-261658, Oct. 16, 1995, 95-2 CPD Para. 176 at 3; U.S. Defense Sys., Inc., B-245563,

Jan. 17, 1992, 92-1 CPD Para. 89 at 3. That is not to say that our Office, in determining the reasonableness of an agency's evaluation and award decision, limits its review to the contemporaneous evaluation and source selection documentation. Rather, we will consider, in addition to the contemporaneous documentation, information provided to our Office for consideration during the protest, including the parties' arguments and explanations and testimony elicited at a hearing. Southwest Marine, Inc.; American Sys. Eng'g Corp., supra. However, in considering the entire record, we accord greater weight to contemporaneous evaluation and source selection material rather than judgments made in response to protest contentions. Boeing Sikorsky Aircraft Support, B-277263.2, B-277263.3, Sept. 29, 1997, 97-2 CPD Para. 91 at 15. Where post-protest explanations simply fill in previously unrecorded details of contemporaneous conclusions, we will generally consider them in our review of the rationality of selection decisions, so long as those explanations are credible and consistent with the contemporaneous record. NWT Inc.; PharmaChem Labs., Inc., B-280988, B-280988.2, Dec. 17, 1998, 98-2 CPD Para. 158 at 16.

Documentation of the Evaluation

Our review of the record confirms that the Navy did not adequately document its evaluation of proposals and that the documentation and further explanation offered during the course of the protest, including the two-part hearing that was conducted by our Office, fail to demonstrate that the evaluation and source selection were reasonable and supported by the facts.

All of the parties agree that the procurement at issue is for a complex and expensive project. The Navy acknowledges in its written post-hearing comments the complexity and magnitude of this procurement, asserting that the agency "dedicated significant resources to the evaluation process." /3/ Agency Post-Hearing Comments at 18. However, in spite of this complexity, the Navy's entire technical evaluation record consists of only three documents: the TEC's extremely brief and conclusory evaluation of initial proposals, the TEC's revised technical evaluation (which is unchanged from the initial evaluation except in the area of key personnel resumes), and the contracting officer's source selection decision memorandum, which generally adopts the TEC's findings and ratings with little further explanation or amplification, including only a single paragraph narrative covering the evaluation of each of the competitive range offerors.

When questioned at the hearing about certain aspects of the technical evaluation, the contracting officer repeatedly referred to an "overall analysis" in which she "looked at each company, I looked at them in total," Tr. at 116-17; "I looked at . . . the big picture, and from that I developed an overall rating plan." Tr. at 117. These statements, coupled with the absence of any specific or detailed documentation of the evaluation process, suggest the application of an imprecise methodology, and leave us to speculate about what considerations actually went into the evaluation and source selection. There simply is no reasoned discussion of the relative strengths and weaknesses of the three proposals under each of the evaluation factors, no evidence that the different factors were weighted differently under the evaluation, and no evidence to show how information provided in the oral presentations was considered. Moreover, the contracting officer's recollections during the hearing about specific aspects of the evaluation were at times inconsistent, as could be expected in so complex a procurement, and exacerbate the problems arising from the lack of adequate contemporaneous documentation. /4/ In addition, she frequently referred to information that was allegedly considered under the evaluation but was never documented. For example, when asked why SES's proposal was rated highly acceptable under the technical approach evaluation factor, while the other two proposals were rated only acceptable, the contracting officer stated that SES "went very much into detail" to explain their technical approach during their oral presentation and that "[i]t was very clear that they offered a technical approach that was far superior to what would be called an acceptable approach . . . ." However, she verified that although the TEC members discussed their ratings after the oral presentations, no notes were made of the oral discussions, nor was the evaluation of the oral presentations documented in any way. Tr. at 190.

In addition, the contracting officer frequently relied on the expertise of the TEC members without necessarily understanding their conclusions, and--in the absence of any documentation of their evaluation--there is no way to ascertain the reasonableness of those conclusions. For example, after testifying that SES's proposal demonstrated its ability to perform the statement of work in a manner that was "[v]ery clear, very easy to read, very easy to understand," Tr. at 191, and after being questioned about the flowcharts that were cited in the source selection decision memorandum as demonstrating SES's proven technical approach, the contracting officer basically conceded that she did not personally understand the flowcharts and could not explain their significance. Tr. at 193-96. Rather, she stated: "I think if you're technical, you do understand what it means . . . if I can remember, one of the comments that was made to me from the technical people was, They mapped it out so well anybody could pick this up and know how to do the job.' That was their concept." Tr. at 194. After being questioned repeatedly about what the flowcharts specifically demonstrated, or how, the contracting officer testified that she did not have an answer. Tr. at 198. Instead of providing any rationale for the agency's technical evaluation, the contracting officer repeated the conclusions the TEC had reached:

They presented a technical approach that was far beyond just demonstrating an ability to perform the work. They flowcharted it, they showed the risks, they showed the mitigation to the risk. They presented to me and to the team a technical approach far superior than what was expected as an acceptable. It was in excess of an acceptable. It showed that the risk would be reduced to practically nothing, and that's what highly acceptable is.

Tr. at 199.

The contracting officer recognized at the hearing that the source selection decision was poorly documented, and attempted to provide additional support. Where post-protest explanations provide sufficient, consistent detail by which the rationality of an evaluation decision can be judged, it is possible to conclude that the agency had a reasonable basis for the decision. See Quality Elevator Co., Inc., B-276750, July 23, 1997, 97-2 CPD Para. 28 at 3-4. Here, however, the explanations provide no such reasonable basis.

EVALUATION OF CORPORATE EXPERIENCE

Future-Tec and CHM each allege that their own proposals should have received a higher score under corporate experience than they received, and that SES's rating for this factor was unreasonably inflated. We agree.

Future-Tec's Corporate Experience

To demonstrate its experience, Future-Tec listed seven contracts in its proposal, six of which were for the same Navy offices as the current requirement, five of which listed the same contracting officer's representative as the one overseeing the current effort. Agency Report, Tab 14, Future-Tec's proposal, at 7-11. Future-Tec also referred to similar technical support services it had provided to four other federal agencies. Id. at 6. Future-Tec discussed all 20 of the tasks required under the RFP's statement of work, noting areas where the firm had designed, developed and implemented processes that are currently being used.

One of the factual premises underlying Future-Tec's allegations is that, as the incumbent contractor under the predecessor contracts for these services, Future-Tec's own experience (for which it received high performance ratings) was most relevant and should have been rated more highly than any other offeror's under the corporate experience factor. The Navy disputes Future-Tec's characterization of itself as the "incumbent," however, insisting that Future Tec's prior contracts involved only "[a] portion of the effort anticipated by [this] RFP" and that "significantly more than half of the anticipated effort had not been included" in Future-Tec's earlier contracts. Agency Report at 3. In contrast, the Navy contends that a contract that the [deleted] awarded to SES was "the only contract cited by any offeror that was actually comparable to the instant requirement in scope, magnitude and complexity." Agency Report at 8.

The agency's position in this regard is not supported by the record. Regarding the size of the prior contract and the award at issue here, which is currently being performed as a result of the override of the stay of performance ("the current contract"), the contracting officer acknowledged at the hearing that Future-Tec and its subcontractor, TAI, had performed work valued at approximately $10 million during the last year of the previous contracts. /5/ Tr. at 77. At the hearing, the contracting officer was unable to support the Navy's position that the current contract is substantially larger than Future-Tec's earlier contracts. Indeed, she acknowledged that, based on orders that have actually been placed, the current contract is "running very roughly around the same size" as the prior contracts. Tr. at 110. She stated that while additional work is anticipated, it has not been funded yet; there had not been a "drastic change from where Future-Tec and [its subcontractor] left off on September 30 to where SES picked up;" the requirement was only expected to increase in size over the 5-year course of performance. Id. Regarding the scope of the work under the prior and current contracts, she acknowledged that under the prior contracts, Future-Tec and TAI had performed every one of the 20 tasks called for in the current statement of work. Tr. at 154. Regarding staffing levels, she acknowledged that the current contract is being performed essentially by the employees who were performing Future Tec's prior contract (including its subcontractor's staff), whom she estimated to number around 170. Tr. at 162-63. In addition, while she testified that she gave less credit to a firm that had performed a number of separate contracts than to a firm that had performed one larger-volume contract because under smaller contracts "[y]ou don't have one person managing the full contract," Tr. at 92, she acknowledged that the separate Future-Tec contracts that were equivalent to the work being consolidated under the current contract had all been managed by a single project manager. Tr. at 176.

In spite of the evidence in the record showing that Future-Tec and TAI were performing essentially the same services as would be required under the current contract, the contracting officer insisted that Future-Tec "never ha[s] managed a contract anywhere near the size of this," Tr. at 92, and, based on this conclusion, rated Future-Tec's proposal as merely acceptable under corporate experience.

CHM's Corporate Experience

CHM listed 13 contracts in its proposal, several of which were multi-year, multimillion dollar contracts with values such as $[deleted] million, $[deleted] million, $[deleted] million, and $[deleted] million, and listed revenue of $[deleted] million for 1998 and a projected revenue of $[deleted] million for 1999. It provided a chart showing the 20 tasks that would be required under the RFP and indicating which of them had been performed under the previous contracts listed in the proposal. Agency Report, Tab 11, CHM Technical Proposal, at 7, 9-10. CHM provided a description of each listed contract, including a separate chart showing which tasks had been performed, and standard information such as contract name, number, agency, period of performance and dollar value, as well as a narrative description of the work performed. Although the contracting officer stated at the hearing that "CHM and its subcontractors have many more contracts than Future-Tec," Tr. at 98, and that CHM "did show that they've done all the tasks in the statement of work" and have "had many contracts" and were "certainly capable of doing the work, very, very well-liked by the people they had worked for," CHM's proposal was also rated only acceptable under corporate experience. The only explanations the contracting officer offered for this rating were that not all tasks had been performed under a single contract (an allegation that conflicts with the record); that certain tasks were not discussed in the narrative of CHM's proposal (although they were shown in the charts); and that CHM had not shown "risk mitigation that really took the risk away from me as the government" (although the evaluation identified no particular risk presented by CHM and noted that CHM's "history of quality past performance mitigates the risk to the Government on this effort in the areas of cost, schedule and performance in meeting the requirements of this RFP." Agency Report, Tab 16, Technical Evaluation, at 1.) Tr. at 89-91.

SES's Corporate Experience

The Navy's evaluation of SES's proposal for this factor provides a telling contrast. SES's proposal lists [deleted] contracts performed over approximately the past 10 years, among which only [deleted] 5-year contract with the [deleted] ("the [deleted] contract"), valued at a total of $[deleted] million, exceeded an average yearly value of $3 million. In connection with the evaluation of Future-Tec's corporate experience, the contracting officer stated that contracts "valued at less than $3,000,000 . . . were not considered to be large enough to be relevant based on the requirement for contracts to be of similar scope, magnitude and complexity." Agency Report at 18; Tr. at 141-43. Notwithstanding that statement, the contracting officer stated that she did consider contracts valued below $3 million per year when she evaluated SES's corporate experience. /6/ Tr. at 135-41. SES's proposal included a separate description of each of the listed contracts, and a chart matching the 20 required tasks with the previous contracts' requirements. The [deleted] contract was clearly the largest one listed, showing a staffing level of [deleted] full-time equivalents (FTEs), while the others ranged from [deleted] to [deleted] FTEs. The average staffing level among all of the contracts that SES listed was [deleted] FTEs. The evaluation record repeatedly reflects the Navy's conclusion that the [deleted] contract "was the only contract cited by any offeror that was actually comparable to the instant requirement in scope, magnitude and complexity." /7/ Agency Report at 8. This contract, and the TEC's reference to SES's education profile, are the only examples given in the evaluation record to explain why SES was rated highly acceptable for this factor.

In our view, these examples do not support the rating assigned. While the description of the [deleted] contract listed a 5-year period of performance, the contracting officer acknowledged that SES had only been performing under this contract for 14 months when proposals were evaluated. Tr. at 123-24. The contracting officer recognized that the [deleted] contract was an indefinite-delivery contract--i.e., that its volume was determined by the number and type of orders placed under it for specific tasks, Tr. at 132, and that the level of performance could vary wildly from year to year. Tr. at 156. She acknowledged that SES's proposal listed no task orders as having been placed under the contract, but stated that she based her evaluation on the total 5-year maximum value of the contract as listed in SES's proposal, Tr. at 179-80; she did not distinguish between tasks that SES had actually performed and tasks that might be ordered under future contract years. Tr. at 126-27. To demonstrate the actual size of the contract thus far, CHM provided records valuing the total orders placed under the [deleted] contract in 1998 at $711,000 and in 1999 at $1,488,000. /8/ CHM's Comments, Nov. 19, 199, exh. 1. The evaluation record also refers to the [deleted]-FTE staffing level for the GSA contract, demonstrating that the Navy was aware that the staffing level under that contract of "exceptional magnitude and diversity" represented only a fraction of the staffing level of approximately 170 FTEs that would be required under the current contract even at its inception, which proportional relationship it apparently chose to ignore. /9/ The chart provided in SES's proposal to match up the tasks required under the current contract with previous contracts under which they had been performed references the [deleted] contract for only 4 of the 20 tasks. Agency Report, Tab 10, SES's Proposal, at 53-54. We find no reasonable basis to support the agency's assessment that this contract demonstrated a higher level of corporate experience than the level demonstrated by any of the other offerors.

The Navy contends that the RFP was "set up so that dollar value' would be the only available statistic to measure the magnitude of a contract." Agency's Post-Hearing Comments at 16. However, the RFP nowhere states that an offeror's experience under prior contracts would be valued according to the face value of contracts it had been awarded, irrespective of the extent of performance that actually had taken place. Regarding past performance, the RFP instructed offerors to identify for each contract held within the last 5 five years the dollar value of the contract and a detailed description of the work performed (among other things), and cautioned them to provide a detailed explanation demonstrating the similarity of the contracts to the requirements in the RFP. RFP Sec. L.III.A.1, at 51. Regarding corporate experience, offerors were instructed to demonstrate either similar or directly related experience of similar scope, magnitude and complexity to the statement of work, and to address the history, organization, qualifications and work experience within the last 5 years as they relate to the requirements of the statement of work. RFP Sec. L.III.B.3, at 52.

In our view, the methodology actually used by the agency contributed to the unreasonableness of the evaluation in several ways. It illogically evaluated past experience on the basis of anticipated future performance. It distorted the value of some contracts, over-valuing a multi-year contract under which performance had only begun, while undervaluing multiple single-year contracts which, when viewed together (where several were performed at the same time or represented one year in a series of awards), would equal or exceed the value of multi-year or consolidated contracts. In addition, the methodology seems to have been unevenly applied, since CHM's proposal, which listed more multimillion dollar contracts than SES in its proposal, was nonetheless rated lower in this area.

The only other basis offered by the Navy for rating SES's proposal highly acceptable under corporate experience is that "SES's education profile was noteworthy, demonstrating that [deleted]% of the employees have advanced degrees." Agency Report, Tab 16, Technical Evaluation, at 9, and Tab 31, Source Selection Memorandum, at 3. We do not see the relevance of advanced degrees to corporate experience. Moreover, as the contracting officer acknowledged, the referenced level of education relates only to SES staff generally--not to the proposed key personnel, whose resumes would be separately evaluated--and thus is even less relevant as a measure of corporate experience for an offeror proposing to perform the contract with the incumbent staff. Tr. at 220-22.

OTHER TECHNICAL EVALUATION ALLEGATIONS

As another example of evaluation improprieties, the protesters challenge the ratings that the Navy assigned the three proposals under staffing. The Navy rated SES's proposal highly acceptable under this factor, and rated the protesters' proposals acceptable. When questioned at the hearing about how an incoming contractor whose staffing plan was based on capturing the incumbent workforce (and therefore included elements of uncertainty or risk) could merit a higher rating than the incumbent, whose staffing plan would require no transition at all, she emphasized that SES had explained in its proposal how it would "ramp up through the full amount of the contract" as the requirements increased. Tr. at 239. We do not find this analysis persuasive. First, while SES's proposal does discuss supplementing the incumbent hires "[deleted] as necessary," Agency Report, Tab 9, SES's Technical Proposal, at 61, it describes its [deleted] methods in very general terms, such as [deleted]." /10/ Id. at 63. In comparison, CHM also proposed to hire the incumbent staff, and presented a more detailed methodology for recruiting, hiring and transitioning, and included discussions of its approach for handling workload fluctuations and surges and its employee retention rate. Future-Tec, as the incumbent whose staff the other offerors were planning to capture, did not discuss the transition--because it would not have any transition--but based its staffing plan on its first-hand knowledge of the previous requirements. In these circumstances, where the incumbent has staffed the requirement to the agency's satisfaction (as shown by Future-Tec's exceptional performance ratings under its Navy contracts) and proposes to continue doing so, and where the other two offerors present very similar staffing approaches, we see no reasonable basis for the SES proposal having been rated more favorably than the others. Neither the evaluation record nor the contracting officer's explanations provide any reasonable support for the ratings in this area.

In sum, the record, including the hearing testimony in this case, offers no convincing explanation as to why the Navy concluded that SES's proposal was entitled to a higher overall technical score than the protesters' proposals, as these examples demonstrate. /11/ Since we cannot find reasonable support for the technical evaluation of the three proposals, as discussed above, we conclude that the source selection lacks a reasonable basis. We sustain both Future-Tec's and CHM's protests on this basis.

THE COST REALISM ANALYSIS

CHM alleges that the Navy's adjustment of its proposed BAFO price, which resulted in an increase of approximately $[deleted] million, was based solely on a mathematical error by the agency and therefore had no reasonable basis.

When an agency evaluates proposals for the award of a cost-reimbursement contract, an offeror's proposed estimated costs are not dispositive, because the offeror's estimated costs may not provide valid indications of the final actual costs that the government is required, within certain limits, to pay. See ManTech Envtl. Tech. Inc., B-271002 et al., June 3, 1996, 96-1 CPD Para. 272 at 8. Accordingly, a cost realism analysis must be performed by the agency whenever a cost-reimbursement contract is contemplated. Federal Acquisition Regulation (FAR) Sec. 15.404-1(d)(2). A cost realism analysis is the process of independently reviewing and evaluating specific elements of each offeror's proposed cost estimate to determine whether the estimated proposed cost elements are realistic for the work to be performed, reflect a clear understanding of the requirements, and are consistent with the unique methods of performance and materials described in the offeror's technical proposal. FAR Sec. 15.404-1(d)(1). The requirement to conduct a cost realism analysis of proposals for a cost-reimbursement contract does not require the agency to conduct an in-depth cost analysis, see FAR Sec. 15.404-1(c), or to verify each and every item in the proposals. Rather, the analysis of cost realism calls for the exercise of informed judgment by the contracting agency involved, since it is in the best position to assess the realism of proposed costs and it must bear the difficulties or additional expenses resulting from a defective cost realism analysis. Our review is limited to determining whether the agency's cost realism analysis was reasonable. The Warner/Osborn/G&T Joint Venture, B-256641.2, Aug. 23, 1994, 94-2 CPD Para. 76 at 5. Here, we conclude that the agency's cost realism analysis, and therefore its cost realism upward adjustment of approximately $[deleted] million to CHM's proposed cost, was unreasonable.

CHM's BAFO showed an overhead rate of [deleted]% and a G&A rate of [deleted]%. Agency Report, Tab 29, CHM BAFO, schedules A, B-1 through B-5. In the BAFO, CHM had offered to reduce the risk of [deleted] escalation by [deleted] composite rate (overhead and G&A)" at [deleted]%. /12/ Agency Report, Tab 29, CHM BAFO, at 2. It is these numbers that led to the cost realism adjustment at issue here.

At the hearing, the contracting officer stated that she had to "try and figure out what [a composite rate] meant, and I went so far as to look in the dictionary," Tr. at 16, and that she concluded that the "composite" rate must mean the sum of the overhead rate and the G&A rate. She therefore added [deleted]% and [deleted]% to conclude that CHM was proposing a rate of [deleted]%, which was lower than the [deleted]% [deleted]. /13/ In its cost realism analysis, the Navy indicated that the capped rate CHM had offered was inconsistent with, and higher than, the individual rates the firm had offered for overhead and G&A. Agency Report, Tab 33, Post-Negotiation Business Clearance Memorandum, at 14. Because the agency had concern that rates might escalate to the [deleted], which was higher than the [deleted]% rate that the agency believed CHM had proposed, the agency tried to compensate by increasing CHM's G&A rate. The cost realism adjustment at issue here resulted when the agency increased CHM's G&A rate by 2.782%, believing that this adjustment was "the best way to account for the capped rates . . . ." /14/ Id. The result was the cost realism increase of approximately $[deleted] million.

We find the agency's concern and the resulting adjustment unreasonable. The schedules in CHM's BAFO show the hourly direct labor rate multiplied by the [deleted]% overhead rate, with the resulting rate then multiplied by the [deleted]% G&A rate--that is, the [deleted]% rate is multiplied against a base that includes the [deleted]% overhead as well as direct costs. The BAFO also stated, in its summary of its proposed indirect rates, that the allocation base for overhead was direct labor dollars, and the allocation base for G&A was total cost input (i.e., direct labor and overhead). Agency Report, Tab 29, CHM BAFO, schedule A. In our view, it should have been clear to the agency that CHM's G&A rate of [deleted]% was multiplied times a base that included both direct costs and overhead. Accordingly, it should also have been clear that the [deleted]% composite rate simply included both component rates (that is, overhead and G&A). /15/ The [deleted] was thus not in excess of the proposed rates, and in fact reasonable review of the schedules in CHM's proposal would have readily shown the agency that the proposal already incorporated the [deleted]% composite rate. /16/

Because the agency's analysis was based on the mistaken assumption that the cap was above the proposed rate, there was no reasonable basis for the agency's cost realism concern, and the resulting adjustment to CHM's proposed cost was unjustified. The record does not demonstrate that the agency considered CHM's costs to be unrealistic based on any independent review of the reasonableness of the proposed rates themselves, or that its conclusions were substantiated through market surveys or historical cost data from similar contracts.

As a result, if CHM's proposed total cost of $[deleted] is compared to SES's adjusted cost of $[deleted], SES's cost is actually approximately [deleted] percent higher than CHM's price, rather than the [deleted] percent used by the agency in its evaluation. Accordingly, we conclude that the source selection, based on both a flawed technical evaluation and an improper cost realism adjustment for CHM, had no reasonable basis.

THE KEY PERSONNEL ALLEGATION

The protesters also allege that only 1 of the 11 key personnel identified in SES's proposal has been performing on a full-time basis under the contract and that SES essentially engaged in "bait and switch" by failing to provide all of the proposed key personnel as full-time staff.

An offeror may not propose to use specific personnel that it does not expect to use during contract performance, since to do so would have an adverse effect on the integrity of the competitive procurement system and generally provide a basis for proposal rejection. CBIS Fed. Inc., B-245844.2, Mar. 27, 1992, 92-1 CPD Para. 308 at 5. The elements of "bait and switch," rendering a contract award improper, are as follows: (1) the awardee represented in its proposal that it would rely on certain specified personnel in performing the services; (2) the agency relied on this representation in evaluating the proposal; and (3) it was foreseeable that the individuals named in the proposal would not be available to perform the contract work. Ann Riley & Assocs., Ltd.--Recon., B-271741.3, Mar. 10, 1997, 97-1 CPD Para. 122 at 2-3.

Here, the RFP specifically required offerors to certify the availability of the key personnel being proposed, even when they were currently employed by the offeror, and required offerors to certify in their BAFOs that personnel they currently employed had confirmed their availability after the date of the request for BAFOs. RFP Sec. L.III.A.2.a, at 52. SES proposed to provide all 11 of the required key personnel from its current staff, certifying that they had confirmed their availability as required. As illustrated by the initial proposal evaluation, any deficiency in any of the 11 key personnel resumes was considered by the agency as rendering the proposal unacceptable. Agency Report at 7.

The protesters essentially allege that SES knew at the time it submitted its proposal that the proposed key personnel were committed to other projects and would not be available to work full time under this contract. As evidence, they point to timesheets submitted by these employees for the first 2 months of contract performance, showing that only the project manager has worked full-time for any of the pay periods since the inception of this contract. The remaining key personnel have been working and continue to work primarily on the contracts to which SES assigned them prior to the current contract.

In response, SES states that it has not relocated its personnel for valid reasons that arose after the award, and not in furtherance of a bait and switch plan. It cites the following reasons: a directive from the Navy installation, issued after award, requiring SES to hire all incumbent staff, creating an overstaffing problem that, in turn, would exhaust available funding unless SES cut back on certain personnel hours; a decision not to relocate personnel pending the outcome of this protest; and an unresolved security issue, which resulted in two proposed employees being denied access to the facility. The Navy further argues that because SES had obtained signed statements from its proposed key personnel, their later
"unavailability" was unforeseeable. Agency's Post-Hearing Comments at
60.

The record shows that only one of the 11 proposed personnel has been
working full-time on the contract awarded to SES. Two employees,
foreign nationals, have been ineligible to work (through no fault of
SES's) based on unresolved security concerns at the Navy installation.
Two others left SES during the first week of performance and have been
replaced with substitute personnel. The remaining six persons have
not been relocated to New Orleans, instead commuting to work only
part-time on this contract, working an average of 2 to 4 days per week
on this contract. Tr. 477-78; SES Post-Hearing Comments at 14. While
we recognize that this represents fewer full-time employees working on
the contract than SES proposed, we see no evidence that SES
misrepresented the availability of key personnel in its proposal.
Absent such evidence, whether proposed employees are, in fact,
performing on a full-time basis under the contract is a matter of
contract administration not subject to our review. 4
C.F.R. Sec. 21.5(a) (1999). We therefore deny this basis of protest.

We recommend that the agency reevaluate all proposals consistent with
the RFP criteria, and reasonably document support for the evaluation
and source selection. We also recommend that the Navy correct the
error in its cost realism adjustment of CHM's proposal, and, if
necessary, reevaluate the proposals for cost realism. Upon completion
of its review, we recommend that the Navy make a new source selection
based on those evaluations. If an offeror other than SES is selected
for award as a result of the agency's reevaluation, the agency should
terminate SES's contract for the convenience of the government. We
also recommend that Future-Tec and CHM be reimbursed their costs of
filing and pursuing their protests, including reasonable attorneys'
fees. 4 C.F.R. Sec. 21.8(d)(1). The protesters should submit their
certified claims for such costs, detailing the time expended and cost
incurred, directly to the contracting agency within 60 days of
receiving this decision. 4 C.F.R. Sec. 21.8(f)(1).

The protests are sustained.

Comptroller General of the United States

1. Although this figure is listed as the "realistic" CPFF in the business clearance memorandum, the contract award value is listed on the first page of this same document as $81,290,286.00, which is the (unadjusted) cost that SES proposed.

2. The narratives for Future Tec's and CHM's proposals are identical, except for the insertion of a different contract number to reference a performance assessment report for each offeror. Agency Report, Tab 31, Source Selection Memorandum, at 4.

3. Offerors were required to submit two-volume technical proposals with hundreds of pages of detailed information and to make oral presentations to the Navy. The agency assembled its TEC of five individuals from the two requiring activities, selected for their varied areas of expertise, who traveled to Philadelphia and spent a full week evaluating the written portion of the proposals and later returned for an additional 3 days to evaluate the offerors' oral presentations. The contracting officer provided guidance to the TEC throughout the process, performing an independent concurrent evaluation of the proposals, with the assistance of four other members of her staff. Tr. at 81-84. The complexity of the material itself is illustrated by the contracting officer's response, when asked at the hearing to explain a flowchart in SES's proposal that the evaluators had described as easy to understand, that she could not explain it, stating that "it would probably take me hours to put all my thoughts together." Tr. at 187.

4. For example, while she initially stated that no credit was given for past performance of contracts valued at less than $3 million, Tr. at 143, she later said, "I guess I actually gave them credit for it, because if I didn't include those [contracts], they would have been unacceptable," Tr. at 146, demonstrating both confusion about what credit was given and a willingness to extrapolate from a sparse evaluation record what "must have been considered."

5. The RFP itself lists Future-Tec and its subcontractor as incumbents and values their predecessor contract efforts at $13,245,170. Agency Report, Tab 37, RFP amend. 002, Part II.

6. Although the contracting officer implied in testimony that she saw little value in analyzing the value of multi-year contracts based on an average yearly value, Tr. at 136-38, she in fact reduced the value Future-Tec listed for a 2-year contract by half to evaluate it on a per-year basis. Agency Report, Tab 44, Contracting Officer's Declaration, at 1.

7. Although the contracting officer asserted that the [deleted] contract was just "an example of a contract," Tr. at 206, 214, and that the Navy "looked at total corporate experience," Tr. at 214, the fact remains that the evaluation record cites only this contract, and cites it repeatedly. Moreover, the remaining contracts listed in SES's proposal are all significantly smaller, falling as they do below the $3 million evaluation threshold.

8. Neither SES nor the Navy has disputed these figures. In addition to this example, the disparity between the magnitude of SES's total corporate experience as listed in its proposal and as perceived by the contracting officer is illustrated by her statement at the hearing that "I didn't do an evaluation of how many dollars they did a year. If I were to do that and I were to add up all of SES's, I mean, they would be up in the hundreds of millions . . . ." Tr. at 214-15.

9. The contracting officer indicated that she did not consider staffing levels to be particularly relevant in determining the magnitude of a contract, even a contract for services, Tr. at 204-08; she primarily looked to the contract's total dollar value rather than other indicators of size such as staffing levels or work actually performed. Tr. at 153; Agency's Post-Hearing Comments at 16.

10. In light of the emphasis that the contracting officer placed on SES's ability to "ramp up" to full performance level (and her assertions concerning the eventual magnitude of the contract), it is noteworthy that SES expresses the expectation in its proposal "to hire no more than [deleted] % of the required staff (non-incumbent new hires)" using the hiring/recruiting process it describes. Agency Report, Tab 9, SES Technical Proposal, at 63.

11. While both protesters have challenged the technical evaluation in various other areas, we think the examples discussed are sufficiently representative of the flaws in the evaluation and will not discuss the others.

12. CHM actually listed separate composite rates for work that would be performed on-site and off-site, as required by the RFP; in the interest of brevity, we will discuss only the rates proposed for government-site work, since the analysis for the two rates is the same.

13. Although she stated at the hearing that she understood that she could have asked clarification questions without opening discussions "[I]f I needed them," Tr. at 34, she did not request any clarification from CHM on this matter because she concluded that it would not make any difference in the source selection decision whether she evaluated CHM's proposal based on the submitted price or the adjusted price. Tr. at 37-38.

14. The Navy's precise calculations that produced this adjustment figure involved rates for both on-site and off-site rates and are not relevant here.

15. The use of the composite rate provided a way to take two cumulative adjustments into account in one step. The composite rate can be calculated as follows: [deleted], which equals [deleted], or, rounded, [deleted]%.

16. For example, schedule B-1 showed a $[deleted] direct labor rate escalated by the [deleted]% overhead rate to $[deleted], which was then multiplied by the [deleted]% G&A rate to $[deleted]. The same result is reached by multiplying the $[deleted] hourly rate by [deleted]%. If, as the agency apparently believed, the proposed cost reflected a combined overhead and G&A rate of only [deleted]%, the loaded rate would have been $[deleted].