Matter of: JB Industries File: B-251118.2 Date: April 6, 1993

Matter of: JB Industries File: B-251118.2 Date: April 6, 1993

PROCUREMENT Competitive Negotiation Offers Evaluation Downgrading Propriety Protest that agency improperly downgraded protester's proposal for inadequate manning and failed to consider protester's unique labor saving approach is denied, where record shows that agency's doubts regarding the effectiveness of protester's labor saving approach in certain areas were reasonable and protester received credit in other areas where labor saving techniques were recognized as effective. PROCUREMENT Competitive Negotiation Offers Risks Evaluation Technical acceptability Agency reasonably found that high risk of nonperformance in one area created potential for nonperformance in other areas where area of concern was central to effective performance in other areas. PROCUREMENT Competitive Negotiation Contract awards Administrative discretion Cost/technical tradeoffs Technical superiority Agency reasonably made award to higher priced, technically superior offeror where award to protester involved potential risk of inadequate performance in at least one area, even though other areas of protester's proposal were highly rated.

Attorneys

DECISION JB Industries protests the award of a contract to Caltech Service Corporation under request for proposals (RFP) No. F04693-91-R- 5005, issued by the Department of the Air Force for base maintenance and civil engineering support services at Los Angeles Air Force Base and auxiliary locations around the Los Angeles area. JB argues that the Air Force improperly evaluated proposals and made an unreasonable cost/technical tradeoff in awarding the contract.

We deny the protest in part and dismiss it in part.

This acquisition was conducted as a competitive set-aside for section 8(a) firms under the Small Business Act. The RFP contemplated the award of a cost-plus-award-fee contract for base maintenance and civil engineering support services for a base year and 4 option years. The requirement covers a wide variety of services such as family housing maintenance and painting, pavement sealing, fire inspections and maintenance of equipment at the base and its auxiliary locations on an as-needed basis. Offerors were required to develop a manning level and labor mix adequate to meet the work requirements contained in the RFP's statement of work.

The RFP provided that proposals would be evaluated, in descending order of importance, under technical, cost and management criteria. In the technical area, proposals were evaluated for understanding of the requirement and identification and use of resources in seven areas, also listed in descending order of importance--Civil Engineering Management, Civil Engineering Support, Operations, Engineering and Environmental Planning, Military Family Housing Maintenance, Fire Protection, and Purchasing and Subcontracting. In the cost area, proposals were evaluated for realism, completeness and reasonableness. In the management area, proposals were evaluated for soundness of management approach and past performance in three areas, again listed in descending order of importance --Program Management, Quality Control, and Phase-in Planning.

The Air Force received proposals from JB and Caltech. The Air Force conducted an initial evaluation, assigning color/adjectival ratings of blue/exceptional, green/acceptable, yellow/marginal or red/unacceptable, and risk ratings of high, moderate or low to each aspect of the proposals. The risk rating was based on the potential risk of nonperformance or negative mission impact associated with an offeror's approach. In performing the cost evaluation, the Air Force considered whether an offeror's proposal was realistic in view of its technical and management approach and in light of an independent government estimate, whether the proposal completely accounted for the costs which would be incurred as a result of the offeror's technical and management proposals, and whether the firm's proposed costs were reasonable in light of the competition received and in comparison to the independent government estimate.

Based on the initial evaluation, the Air Force determined that both JB and Caltech were in the competitive range. It then prepared a list of deficiency reports and clarification requests which it presented to the offerors during two rounds of written discussions, following which the agency solicited and received best and final offers (BAFO). The BAFOs, together with the offerors' responses to the discussion questions, were used by the Air Force to arrive at final technical and management ratings for each firm, and also to evaluate cost.

Under the technical and management evaluation, JB received blue/exceptional ratings with low risk assessments in all but one of the seven technical areas; in the Engineering and Environmental Planning area, JB received a yellow/ marginal rating and a high risk assessment. This rating was based on the Air Force's determination that JB had proposed insufficient staffing to perform all of the requirements in this area. Under the management criterion, JB received green/acceptable ratings and low risk assessments in all three areas. Caltech received blue/exceptional ratings in five of the seven technical areas. In the Civil Engineering Support, and Purchasing and Subcontracting areas, Caltech received green/acceptable ratings and low risk assessments. Under the management criterion, Caltech received blue/exceptional ratings and low risk assessments in all three areas.

Caltech's proposed cost of $44 million was found to be realistic, complete and reasonable; no cost adjustments were made to the firm's final proposed price to arrive at its evaluated price. JB's proposed cost of $37 million was found to be complete and reasonable, but not realistic to perform the effort, given the nature of its technical and management approaches. Adjustments were deemed necessary to account for the cost of an additional five engineers the Air Force found would be needed for JB to meet all contract requirements, and to account for a 2.5 percent escalation of its other direct costs, which JB had not escalated for the option years. The agency therefore adjusted JB's proposed costs upward by $1.6 million, which resulted in an evaluated price of $38.6 million.

On the basis of these evaluation results, the Air Force made award to Caltech as the firm whose proposal represented the best overall value to the government. JB's yellow/marginal rating and high risk assessment under the Engineering and Environmental Planning area was the determinative consideration in the award decision. The agency found that JB's deficiencies in this area impacted on its proposal overall; the firm's high risk rating for Engineering and Environmental Planning rendered its proposal high risk. In contrast, Caltech had received acceptable or exceptional ratings in all of the technical and management areas and was found to be low risk.

TECHNICAL EVALUATION

JB argues that the Air Force improperly assigned a yellow/marginal, high risk rating to its proposal for inadequate staffing under the Engineering and Environmental Planning evaluation element. JB maintains that the agency's conclusion was based on its failure to give due consideration to its approach of "crossutilizing" personnel from one area to accomplish tasks called for elsewhere. JB believes its approach was sufficient to accomplish all contract work, while offering the added benefit of reduced costs due to its reduced staff, and that the agency applied preestablished manning estimates instead of considering the benefits of its approach.

In reviewing allegations concerning the propriety of an agency's technical evaluation, our Office does not independently evaluate proposals or substitute its judgment for that of the agency. Schweizer Aircraft Corp., B-248640.2; B-248640.3, Sept. 14, 1992, 92-2 CPD Para. 200. We will question an agency's technical evaluation only where the record shows that the evaluation does not have a reasonable basis or is inconsistent with the RFP's stated evaluation criteria. Id.

We find that the Air Force's evaluation of JB's proposal was reasonable. As discussed above, the record shows that the Air Force's primary concern with the JB proposal was that the firm offered inadequate manning to accomplish the RFP's Engineering and Environmental Planning requirements. Specifically, the evaluators found that JB's manning for design and construction management was significantly lower than the government estimate for this requirement, that JB had not identified personnel to perform numerous functions in this area, and that an energy monitor had not been identified.

Although JB maintains that its proposal addressed all manning requirements, it did this not by means of a fully staffed approach, but by using engineers identified for one function to perform other functions in the Engineering and Environmental Planning area, or by proposing to hire temporary staff. For example, JB proposed to use its design engineering staff to meet the RFP's contract programming, construction management and design requirements, and also proposed using these same individuals to complete a base comprehensive plan and to perform the duties of energy monitor. The Air Force brought its concerns to JB's attention through detailed discussions in this area. The Air Force was not satisfied with JB's explanation of how its approach of using limited staffing and temporary employees would enable it to perform all of these functions effectively, and concluded that its approach reflected a lack of understanding of the scope of the work requirements in this area; it was for this reason that the agency downgraded JB's proposal and determined there was a high risk that JB would not meet all of the requirements in this area.[1] We find no basis for questioning the agency's view that JB's cross-utilization and temporary staffing approach was inadequate under these circumstances, so as to warrant a yellow/marginal, high risk rating.

Moreover, although JB maintains that the Air Force ignored the labor saving aspects of its proposal and instead applied preestablished manning estimates, the record shows that the agency in fact accepted JB's approach in those instances where JB adequately explained how it would accomplish the contract requirements with its reduced staff. For example, to perform the military family housing maintenance requirements, JB proposed a reduced staff which could be used for a combination of tasks, and also proposed to reduce the number of supervisory personnel that would be required for this aspect of the contract. The Air Force was satisfied that JB could meet the requirements in this area with its proposed approach, and found that it had exceeded the RFP's requirements by combining operational functions, reducing supervisor positions and using multiskilled personnel. The agency did not increase JB's staffing in this area by some predetermined amount but, rather, rated the proposal blue/exceptional under this evaluation element. It thus is clear that the agency did not apply preestablished manning requirements but, rather, actually considered JB's approach in determining to what extent its proposed manning was deficient.

COST EVALUATION

JB argues that the Air Force improperly made upward cost adjustments to its proposed cost in arriving at a most probable cost figure for JB. Specifically, JB maintains that the Air Force improperly added the costs of five full-time engineers to its proposal and also improperly added a 2.5 percent escalation figure to its other direct costs.

Where an agency contemplates the award of a cost reimbursement contract, the offerors' proposed costs should not be considered as controlling, since they may not provide a valid indication of the actual costs which the government is, within certain limits, required to pay. Bendix Oceanics, Inc., B-247225.3, July 27, 1992, 92-2 CPD Para. 54. Consequently, the agency must perform a cost-realism evaluation in order to independently determine what the likely costs of performance will be given an offeror's particular approach, assuming reasonable economy and efficiency. Our review is limited to considering whether the evaluation was reasonable. Id.

JB's argument regarding the cost of the additional engineers is premised on its position that the Air Force's technical evaluation under the Engineering and Environmental Planning element improperly failed to consider JB's cross-utilization and temporary staffing approach. As discussed above, however, we find that the Air Force reasonably concluded that JB's approach to satisfying the requirements under this evaluation element was deficient, and thus reasonably downgraded the proposal for this reason. Since the RFP also called for the agency to conduct a cost- realism analysis, the Air Force necessarily had to give consideration to the potential cost of the personnel which were viewed as necessary for JB to perform effectively given its technical approach. The agency therefore properly included the cost of the additional personnel in arriving at a most probable cost estimate for JB.

Regarding the 2.5 percent escalation of JB's other direct costs (ODC), the record shows that this adjustment was made because JB had proposed the same amount for its ODC for each of the contract's 5 years, and the upward adjustment was made to reflect the likely effect of inflation on these costs. JB maintains that it did propose to escalate its ODC for most contract elements and only failed to escalate its subcontractor's ODC for some of the contract's later options. An examination of JB's BAFO, however, shows that JB proposed the same amount for materials and ODC for each of the contract's 5 years. We do not understand JB's position regarding the upward adjustment to its ODC given that the record contradicts the factual basis of its allegation. Since JB does not otherwise challenge the propriety of the adjustment, we conclude that the Air Force acted reasonably in escalating JB's proposed costs in this area.

COST/TECHNICAL TRADEOFF

JB argues that the Air Force's cost/technical tradeoff lacked a reasonable basis. JB maintains that the Air Force improperly concluded that its high risk rating in the Engineering and Environmental Planning area rendered its entire proposal high risk so as to offset its substantial cost advantage. According to the protester, the agency's actions in this respect were unreasonable because this was only the fourth most important evaluation element, and it received low risk assessments in all of the other areas. JB believes its low-cost proposal, with blue/exceptional ratings in six of the seven technical areas, represented the best overall value to the government.

Where, as here, the solicitation provides that technical considerations will be more important than cost, source selection officials have broad discretion in determining the manner in which they will make use of the technical and cost evaluation results in arriving at a source selection decision. University of Dayton Research Inst., B-245431, Jan. 2, 1992, 92-1 CPD Para. 6. Such cost/technical tradeoffs are governed only by the test of rationality and consistency with the RFP's stated evaluation criteria. Miller Bldg. Corp., B-245488, Jan.3, 1992, 92-1 CPD Para. 21.

JB is correct that its evaluated $5.4 million cost advantage was significant, and we think the agency reasonably could have determined that, together with the high rating of JB's proposal in many areas, this cost advantage warranted selecting JB for award. However, in light of the agency's broad discretion in this area, the fact that the agency could have justified a different tradeoff decision does not establish that the decision the agency did make was unreasonable. We think the agency's tradeoff was reasonable based on the technical and management differences in the proposals.

As already noted, the agency reasonably determined that JB's proposal was yellow/marginal and high risk in the Engineering and Environmental Planning area because of its unacceptable approach to performing this aspect of the contract. While JB is correct that this was only the fourth- ranked technical evaluation criterion, the work covered by it--functions such as completion of a base comprehensive plan, supervision of contract programming, and management of construction projects and design efforts-- is integral to, and thus will affect the performance of, other work under the contract. For example, the statement of work (SOW) requires the contractor to develop programming documents for the construction, maintenance and repair of real property facilities as part of the Engineering and Environmental Planning function. As part of the Operations function described elsewhere in the SOW, the contractor is required to operate, maintain, inspect, repair and overhaul all real property. In order to effectively perform in the Operations area, timely and effective performance in the Engineering and Environmental Planning area would appear to be necessary. For this reason, the agency concluded, reasonably we think, that JB's high risk rating under this one factor reflected a risk of degraded performance of the contract as a whole.

Although the Air Force engaged in extensive discussions with JB regarding the adequacy of its cross-utilization approach in this area, JB decided not to significantly alter its approach. The Air Force thus was left to either accept this risk of nonperformance, or pay a premium to Caltech to avoid the risk. Since Caltech also had been rated superior to JB under all three of the management evaluation factors (blue/exceptional compared to JB's green/acceptable ratings), the Air Force determined that the latter option was in the government's interest. Although, again, the cost premium the Air Force opted to pay is significant, we cannot find that it was unreasonable for the agency to pay this premium in lieu of accepting a lower rated, high risk proposal.

UNTIMELY ISSUES

In addition to the matters discussed above, JB argues that the Air Force improperly failed to apply its "lowest evaluated price" (LEP) method in evaluating proposals, despite references to this method in the RFP's executive summary and cover sheet.

This argument is untimely. Under our Bid Protest Regulations, protests of alleged solicitation defects must be filed no later than the time set for receipt of proposals. 4 C.F.R. Sec. 21.2(a)(1) (1992). The LEP technique calls for the assignment of point scores to both technical and cost proposals, with award generally being made to the proposal receiving the highest aggregate score. See generally Recon Optical, Inc., B-232125, Dec. 1, 1988, 88-2 CPD Para. 544. Here, the agency did not point score proposals, but instead used color/adjectival ratings. The agency's decision not to use the LEP method was reflected in amendment No. 5 to the RFP which, in clarifying the evaluation scheme through responses to questions submitted by offerors, stated that "point scoring will not be used to rate proposals."[2] Since this statement reflected an intent inconsistent with the references (inadvertently left in the RFP, according to the agency) to the LEP method, there was no basis for JB to assume that the LEP method would be used. To the extent JB found the agency's intent unclear, JB was required to protest on this ground prior to the submission of proposals. Since JB did not raise this argument until after the submission of proposals, the matter is untimely and will not be considered. Genesys Research, Inc., B-245421, Jan. 2, 1992, 92-1 CPD Para. 5.

In any event, we do not understand how JB could possibly have been prejudiced by the Air Force's use of one evaluation methodology rather than the other. In all cases, the results of an agency's evaluation (whether expressed numerically or adjectivally) are merely guides to intelligent decision making. JB has not suggested that it would have made changes to its proposals based on the agency's selection of a particular evaluation methodology, and given that the relative weight of the criteria would have been the same whether numeric or adjectival ratings had been used, we cannot envision JB making such changes.

JB also alleged for the first time in its comments on the agency's report that (1) the Air Force's past performance evaluation of its management proposal was improper, and (2) Caltech had improperly subcontracted more than 50 percent of the work to be performed to a firm not eligible under the Small Business Administration's section 8(a) program.

These allegations are also untimely. Protests other than those based on alleged solicitation improprieties must be filed within 10 working days after the protest basis was or should have been known. 4 C.F.R. Sec. 21.2(a)(2). JB received the Air Force's report on December 2, 1992. All of the agency's written evaluation materials as well as a copy of Caltech's proposal were furnished to JB's counsel as part of that report. These allegations, both of which should have been clear from the report materials, were not raised until JB filed its comments in our Office on December 23, 1992, more than 10 working days after it received the report. (JB did file a supplemental protest within 10 working days which contained other allegations based on the report.) These arguments therefore also are untimely and will not be considered.

We deny the protest in part and dismiss it in part.

1. JB's proposal to use additional temporary engineers potentially could augment JB's staff, but this would be at JB's discretion since JB's offer did not include an objective means for determining when JB believed additional staff would be needed. JB's proposal stated (in response to Air Force discussion questions in this area) only that "should [JB] find that certain work requirements necessitate additional, specialized manpower, we will acquire such personnel from the local labor pool (agencies) on a temporary basis." The proposal of temporary staff therefore did not eliminate the agency's concern that JB's approach simply did not include enough staff to perform all of the work for which they were proposed.

2. JB maintains that this statement in amendment No. 5 was for informational purposes only and was not made a part of the RFP. This argument is inconsistent with the terms of amendment No. 5 which stated only that the offerors' questions and answers would not be part of any resulting contract but did not state that the materials were not part of the RFP.