Where such staffing was not a significant factor in the listed evaluation criteria. The protester's advantage on the first sample task was reasonably found to be an advantage of incumbency which was not indicative of technical superiority. Protest is sustained where. Agency failed to evaluate cost proposals involving sample task costs in accordance with the listed solicitation evaluation criteria and thereby did not reasonably consider the impact that widely divergent sample task costs should have had on the selection decision. The protester argues that the evaluation of the awardee's technical and cost proposals lacked a rational basis and is inconsistent with the terms of the RFP. GTC is particularly concerned with the agency's assessment of GE's proposed solutions to three sample tasks required to be submitted by the RFP.
B-240736, Dec 19, 1990, 90-2 CPD 502
PROCUREMENT - Competitive Negotiation - Offers - Evaluation - Technical acceptability DIGEST: 1. Agency reasonably found competing proposals to be technically equal despite the awardee's proposing a higher number of staff hours than did the protester in its solution to a sample task problem, where such staffing was not a significant factor in the listed evaluation criteria, the difference in hours involved only one of three sample tasks, and the protester's advantage on the first sample task was reasonably found to be an advantage of incumbency which was not indicative of technical superiority. PROCUREMENT - Competitive Negotiation - Contract awards - Administrative discretion - Technical equality - Cost savings 2. Protest is sustained where, in deciding to award a time and materials contract on the basis of cost because competing proposals had been determined to be technically equivalent, agency failed to evaluate cost proposals involving sample task costs in accordance with the listed solicitation evaluation criteria and thereby did not reasonably consider the impact that widely divergent sample task costs should have had on the selection decision.
Group Technologies Corporation:
Group Technologies Corporation (GTC) protests the award of a contract to the General Electric Company (GE) under request for proposals (RFP) No. DAAB07-90-R-A022, issued by the Department of the Army for the development of software in support of cryptological activities. The protester argues that the evaluation of the awardee's technical and cost proposals lacked a rational basis and is inconsistent with the terms of the RFP. GTC is particularly concerned with the agency's assessment of GE's proposed solutions to three sample tasks required to be submitted by the RFP.
We sustain the protest based on our conclusion that the cost analysis was faulty.
The RFP was issued on January 31, 1990, contemplating a 3-year, time and materials contract. Under this type of contract, services are acquired on the basis of direct labor hours at specified fixed hourly rates with subsistence, travel, and materials paid for on a cost reimbursable basis; the services are provided in response to task orders. The task order is initiated by a task directive issued by the agency after which the contractor is to propose a task execution plan including a proposed technical approach and the resources necessary to accomplish the task, and the agency and the contractor are to then negotiate a specific approach prior to the placement of a formal task order.
Award was to be made on the basis of the proposal offering the best value to the government considering three evaluation factors listed in descending order of importance: technical, cost, /1/ and management. The technical factor was the most important, with its significance described as being greater than cost and management combined.
The technical factor was divided into two subfactors: (1) "sample tasks-- engineering approach"; and (2) "overall contractual requirements." Under the "sample task-- engineering approach" subfactor, offerors were required to submit separate task execution plans for three sample tasks of the type to be performed under a resulting contract. The first sample task involved the design, development, and coding of software for a computer known as the Data Transfer Device (DTD) to perform specified control functions required for mobile subscriber equipment (i.e., a telephone-like device used for field communications). The second and third sample tasks involved the development of key management architectures and key management plans preparatory to fielding, respectively, a Combat Net Radio and a communications encryption device known as the Enhanced KG-84.
The sample task solutions were to be assessed considering eight subelements, listed in descending order of importance:
(1) understanding the problem
(3) feasibility of approach
(4) automated tools and techniques
(7) engineering man-hours
The technical subfactor involving overall contractual requirements was less important than the sample tasks and did not require offerors to describe any specific technical efforts they proposed to employ during contract performance.
Offerors were required to submit separate cost proposals for each of the sample tasks and for the overall contract requirements. The latter principally consisted of offering wage rates on a matrix of agency estimated staff-hours by labor category. Cost was to be evaluated under two sub-factors: "cost proposal" and "cost realism." Under the first subfactor, costs were to be examined to determine an offeror's understanding of the project and its ability to perform; the subfactor further provided:
"In the evaluation for award, consideration will be given to the estimated costs proposed for both the sample tasks and the overall contractual requirements."
Under the realism subfactor-- which also assessed sample task and overall costs-- the term "realism" was defined as a correlation between the proposed technical effort and the proposed costs.
Four proposals were received by the March 21 closing date. All were regarded as acceptable under the management factor. In the technical evaluation, GTC's proposal, which received "outstanding" ratings on both technical subfactors (and all subelements within the sample task subfactor), was rated technically "outstanding" overall. GE's proposal was also rated technically "outstanding" overall and the only difference between its rating and the protester's was that GE received an "acceptable" rating in the seventh-listed subelement under the sample task subfactor, involving the number of hours proposed to perform the tasks. /2/
With respect to the number of hours and costs associated with the three sample tasks, GE proposed the following: /3/
Task No. 1 33,282 hrs. $ 1,926,153
Task No. 2 1,513 hrs. 103,956
Task No. 3 1,513 hrs. 103,956
With respect to GE's proposed number of hours for sample task No. 1, which was the highest proposed by any offeror and which considerably exceeded the government estimate of 24,400 hours, the evaluators noted that they were of the proper "type" to perform the required work; GE's rather low "acceptable" rating for the engineering level of effort subelement was attributed in the evaluation report to a relatively large percentage of hours in the program manager and clerical support categories.
With respect to the protester's proposed number of hours for the same task, the evaluators noted that it was the lowest received and commented that the relatively low figure (which was a fraction of the government estimate) was justified since the firm planned on reusing certain software "code" it had already written for the DTD; /4/ the evaluators also noted that "other contractors have proposed using the same code but they do require a greater learning curve since they were not the originators of the code."
During the initial evaluation, the offerors' labor rates were examined and compared and determined to be realistic because they fell within "the same range." All costs were accepted as proposed. With regard to proposed sample task costs, the agency reports that they were primarily used in the technical evaluation to determine an offeror's understanding of the tasks. They were not compared among offerors but were examined to see if each offeror had properly "costed out" its own unique technical approach. Sample task costs played no role in the agency's consideration of proposals after the submission of best and final offers (BAFOs).
Discussions, which were limited to clarifications involving procedural accounting matters that did not alter the technical rankings and did not involve the offerors' proposed number of engineering hours for any sample task, were completed by May 21. On June 21, offerors were requested to submit BAFOs containing overall costs based on a revised matrix of estimated hours broken down by labor category. GE and GTC submitted the following final overall cost estimates for the entire 3-year contract effort:
GTC $ 13,100,373
GE $ 12,881,698
Despite the difference in subelement ratings involving engineering hours, the contracting officer concluded that GE and GTC had submitted technically equivalent proposals; GE was awarded the contract based on an assessment that its lower overall BAFO costs represented the best value to the government.
GTC challenges the adequacy of both the technical evaluation and the cost evaluation with particular regard to GE's proposed solution to the first sample task involving software for the DTD, /5/ and argues that the Army's failure to follow the RFP evaluation criteria led to a selection decision which was not rationally based and which will result in the government's paying a substantial premium for the software services it needs.
GTC principally contends that the number of hours GE proposed to perform sample task No. 1 was excessive and should have caused the evaluators to severely downgrade the GE proposal because, in the protester's view, the number of hours proposed showed that the awardee did not have an adequate technical understanding of the work to be performed. This evaluation defect, in GTC's opinion, rendered the selection decision, based on a purported equivalency between GE's proposal and its own, irrational and not in accord with the RFP evaluation scheme which placed considerably more emphasis on technical excellence than cost.
The evaluation of technical proposals is primarily the responsibility of the contracting agency; the agency is responsible for defining its needs and the best method of accommodating them and must bear the consequences of any difficulties resulting from a defective evaluation. Therefore, our Office will not engage in an independent evaluation of technical proposals and make an independent determination of their relative merits. Litton Sys., Inc., B-239123, Aug. 7, 1990, 90-2 CPD Para. 114. Rather, we will examine the agency evaluation to ensure that it was reasonable and consistent with the evaluation criteria listed in the RFP; the fact that the protester disagrees with the agency's evaluation does not render it unreasonable. Id. Also, in reviewing the results of a technical evaluation, a contracting officer has considerable discretion in reaching the conclusion that technical ranking differences may simply be due to the advantages of incumbency and that, therefore, they are not necessarily indicative of any actual technical superiority for the purposes of a particular procurement. See Grey Advertising, Inc., 55 Comp.Gen. 1111 (1976), 76-1 CPD Para. 325.
From our review of the evaluation record and the Army's position taken during the protest, as well as from our reading of GE's proposed solution to the sample task at issue, it appears that the agency has, in effect, regarded the number of hours proposed by GE as a disadvantage caused to a great extent by its lack of experience with the source code rather than its lack of technical ability.
It is clear that, with the exception of the number of hours GE proposed, the evaluators were impressed with the technical approach GE presented to solve the DTD task. As with the other sample tasks, the exercise involved in sample task No. 1 was to provide an "outline of the engineering approaches and proposed methods and techniques an offeror intends to use in accomplishing the sample Task Order." A final "solution" was not really anticipated. In describing its approach GE began with a software reuse assessment of the specification containing the source code and indicated that, during later stages of work under the task, this assessment would be completed to further determine the extent to which portions of the code could be reused in any solution. This constituted a planned approach, not a final solution, and the proposal clearly anticipated additional work with the code in reaching a final solution. On this basis alone, we believe the agency had an adequate foundation for its conclusion that, with continued familiarity with the code and discovery of its reusable portions, GE's hours would likely decrease.
Implicit in the agency's evaluation was that the protester enjoyed an advantage of incumbency through having developed the code and thus being familiar with its reusability; nonetheless, the agency estimated that 24,400 hours-- a figure far closer to the number of hours proposed by GE than the number proposed by GTC-- would be required by an acceptable contractor to perform the first sample task. Thus, while GTC's proposed hours were predictably far less than this estimate and viewed favorably with an outstanding rating, this does not necessarily mean, as the protester would have it, that any proposal exceeding GTC's number of hours by a large degree or the government estimate by some lesser degree would be technically unacceptable. /6/
Under the circumstances, we think that the agency "penalized" GE appropriately by giving the firm the lowest acceptable ranking possible on the engineering hours subelement. That subelement was, however, listed seventh in order of importance out of eight; taking this factor into account together with the fact that the excess level-of-effort impacted only one of three sample tasks, we believe that the agency acted reasonably and consistently with the evaluation criteria in the manner in which it rated GE's technical proposal. Further, in view of the fact that GE received only one rating below "outstanding" under all of the technical elements and that was on a relatively low ranked element, we do not think that the agency's judgment in assigning an overall technical rating of "outstanding" to GE was unreasonable. Nor under the circumstances here do we have a legal basis to disturb the agency's conclusion that the GE and GTC technical proposals were essentially equal. The fact that GTC disagrees with the conclusions reached by the evaluators and the contracting officer in exercising the range of discretion accorded to them does not make their judgments unreasonable. Litton Sys., Inc., B-239123, supra. COST EVALUATION
GTC argues that the Army failed to follow the RFP cost evaluation criteria which provided that an offeror's proposed sample task costs would be considered "in the evaluation for award" together with its overall proposed costs for the 3-year effort and that in order to "minimize built- in cost growth the Government will evaluate the realism of the offeror's proposed cost in terms of the overall contractual requirements and the sample tasks." In the protester's view, had GE's high sample task costs /7/ been used in some reasonable fashion along with its overall proposed contract costs in the final selection as contemplated by the RFP, the agency would have concluded that GE's proposal did not represent the best value to the government given the wide disparity in GTC's favor in sample task costs and the relatively minor advantage in overall wage rates enjoyed by GE.
In response, the Army basically argues that it gave sufficient consideration as contemplated by the RFP to proposed sample task costs prior to the submission of BAFOs and that no further consideration was required during the final award selection. Noting that sample costs were primarily considered in the evaluation process to determine an offeror's technical understanding of the tasks, the agency stresses that they were also examined to ascertain if an offeror properly understood such things as pricing procedures to be used for task orders under the contract, i.e., the correct loading of labor rates and the correct manner of accounting for billable time. The agency states that it would be impractical to use the proposed sample task costs along with overall costs as suggested by GTC since there is no way of determining what tasks will actually have to be performed during contract performance and GE's costs associated with lack of familiarity with the source code would decrease overtime. The sample task costs are simply, in the Army's view, not an accurate predictor of the cost of contracting to the government.
An agency's evaluation of proposed costs and cost realism is aimed at determining the extent to which the offeror's proposal represents what the contract should cost the agency. See Systems Research Corp., B-237008, Jan. 29, 1990, 90-1 CPD Para. 106. The level of cost analysis required in a given procurement, including which cost elements are to be analyzed and the extent of the analysis, are dependent upon the type of contract to be awarded and the cost evaluation scheme described in the RFP. See Research Management Corp., 69 Comp.Gen. 368 (1990), 90-1 CPD Para. 352. The time and materials contract here has elements of both fixed-price and cost-type contracts. The contract price is fixed only to the extent that offerors were required to propose fixed, burdened labor rates for the government's estimate of staff-hours in various labor categories to be required over a 3-year period. This estimate is admittedly speculative insofar as the Army states that it cannot determine the precise quantity of services and materials needed during the period. Thus, in addition to the costs of travel, materials, and subsistence, which offerors were required to propose and which are to be paid for on a cost-reimbursable basis, the cost of performance to the government will vary depending on the actual task orders issued and, in large measure, with a given contractor's efficiency in performing the resulting tasks. Id.
The RFP contained a vehicle-- the sample tasks-- for reviewing competing offerors' approaches and their efficiency under simulated contract conditions which required the submission of detailed cost proposals. The RFP stated that the sample task cost proposals would be used to "determine ... the prospective contractor's understanding of the project and his ability to organize and perform the proposed contract." It also stated that "in evaluating the offeror's proposed cost" these estimated costs for the sample tasks would be considered.
Under such circumstances, it is incumbent upon the contracting agency to do what it represented it would do-- take advantage of the cost evaluation tools provided in the evaluation scheme and to perform an analysis that includes the sample costs in order to ensure that the government obtains the lowest overall cost of performance. Research Management Corp., 69 Comp.Gen. 368, supra. (This is especially true here where the agency has determined that closely-scored competing proposals are technically equivalent and, thus, bases its award decision on cost alone.)
According to the evaluation record, the offerors' estimated costs to perform the sample tasks had no role in the relative evaluation of the estimated costs of contract performance. The only treatment accorded proposed sample task costs was to see that they comported with an offeror's own technical approach and to see whether that offeror understood the accounting procedures that were to be employed under the contract. These factors do not measure the relative efficiency of competing sample proposals. For example, a proposal with an excellent but time-consuming (and therefore, expensive) approach which was accurately reflected in its cost proposal would presumably, under the method of cost analysis employed by the Army, be seen in the exact same light as a technically excellent competing proposal which was less time consuming and therefore reflective of a lesser cost of contracting to the government.
We think that the RFP provisions required that the evaluators consider more in this case. While the RFP could have been more clearly worded, it did not permit, as the agency seems to argue, the Army to ignore a significant difference in offeror proposed costs on the sample tasks simply because each offeror's proposed costs were consistent with its own technical approach. Where, as here, two offerors propose differing technical approaches which resulted in widely disparate cost estimates for performing the same task, and overall cost forms the basis for the selection, we believe that for the cost evaluation to have a rational basis, the disparity in the proposed sample costs should receive some consideration in the selection decision.
As with GE's technical proposal for the first sample task, the Army could reasonably conclude that the number of estimated hours and their resulting estimated costs do not fully represent what would occur during contract performance. However, just as the agency gave appropriate consideration to GE's estimated level-of-effort for the first sample task in the technical evaluation, it was required to give the resulting cost estimate some weight in the overall cost evaluation. Since those widely varying proposed sample costs did not receive any consideration in the selection decision, and since the method used to evaluate competing cost proposals must provide a reasonable basis for source selection and be conducted in accordance with the stated RFP criteria, we sustain the protest. Cf. Electronic Warfare Integration Network, B-235814, Oct. 16, 1989, 89-2 CPD Para. 356.
We recommend that the Army promptly reevaluate the offers received in a comparative manner consonant with this decision and the terms of the solicitation in considering the effect of proposed sample task costs on the selection decision. In recommending that the agency reconsider its award decision, we do not suggest how much weight should be given to sample task costs in evaluating overall costs; however, the method chosen by the Army must be consistent with the terms of the RFP by taking into account the relative impact that varying contractor efficiencies may have in the overall cost to the government of contracting for the software services it seeks. If that reevaluation results in a determination that the protester's proposal represents the best value to the government, then the Army should terminate GE's contract for the convenience of the government and award to GTC. Jack Faucett Assocs., B-233224, Feb. 3, 1989, 89-1 CPD Para. 115.
We also find the protester entitled to its costs of filing and pursuing the protest. Bid Protest Regulations, 4 C.F.R. Sec. 21.6 (d)(1) (1990).
/1/ The most significant aspect of cost was the fixed hourly rates for the various categories of workers which included wages, overhead, general and administrative (GA) expenses, and profit.
/2/ The other two offerors received overall technical ratings of "highly acceptable" and "acceptable."
/3/ GTC has objected to the release of its proposed number of hours and costs as proprietary; accordingly, we have reviewed the data in camera and only offer general descriptions of it in the decision. GE has not objected to the release of its data.
/4/ GTC developed the "source code" for the DTD under a government contract and this code is contained in National Security Agency (NSA) Specification No. 89-27, which is formally titled "Reusable Ada Code Specification for Application Software for DTD."
/5/ We find no support in the record with respect to GTC's objection to the evaluation of the other sample tasks on the basis that, by proposing the same number of hours and the same cost for each, the firm's offer reflected a lack of technical understanding.
/6/ There is some confusion as to whether GE actually used the source code in preparing its response to sample task No. 1. In the initial stages of the protest and during the evaluation, the agency did not believe so. It now seems clear that GE did indeed use the code. In our view, the fact that GE had the code does not diminish the validity of the premise that its lack of familiarity with the code, as compared to the code's author-- GTC-- could be the cause of the level of-effort disparity.
/7/ The record discloses that the awardee's sample task No.1 costs were significantly higher (more than three times higher) than GTC's sample task costs which were, by far, the lowest proposed among the four offerors.