Matter of: Engineering and Computation, Inc. File: B-261658 Date: October 16, 1995

Matter of: Engineering and Computation, Inc. File: B-261658 Date: October 16, 1995

Source selection decision cannot be determined to be reasonable where it is based on unsupported evaluation conclusions and agency fails to rebut protester's assertions that its proposal was misevaluated.

Attorneys

DECISION

We sustain the protest.

The RFP listed the following equally weighted evaluation factors: mission suitability, cost, relevant experience and past performance, and other considerations (for example, phase-in; corporate policies, procedures, and practices; labor relations; corporate resources; and small business and small disadvantaged business concerns plans). As is relevant here, mission suitability was the only factor scored and it consisted of the following three subfactors: understanding the requirement, management approach, and staffing plan. The RFP defined these subfactors in detail. The offerors were asked to provide a risk analysis for each subfactor which identified "risk areas, if any," and their recommended approach to minimize the impact of those risks on the program.

NASA received proposals from three firms, including ECI. After initial evaluations, ECI was notified that its proposal was outside the competitive range, although it received a "good" rating. After meeting with ECI, NASA reconsidered its decision and decided to include ECI's proposal in the competitive range. Written discussions were conducted, and the three firms subsequently submitted best and final offers (BAFO). The evaluation committee submitted its final evaluation report, with ratings and scores, to the source selection official (SSO). [1] The adjectival ratings and scores were as follows:

Firm A ECI IIT

Mission suitability 840 points; very good 850 points; very good 950 points; excellent

Relevant experience and past performance excellent good excellent Other considerations excellent very good excellent

Proposal cost; most probable cost $7.8 million; $8.6 million $8.1 million; $8.3 million $7.6 million; $8.6 million

In reviewing the evaluation results, the SSO noted that IIT's proposal was "scored significantly higher than the other two proposals in mission suitability," and IIT had an advantage in every subfactor under mission suitability. The SSO concluded that the IIT proposal was clearly the superior proposal under mission suitability. The SSO also found that relevant experience and past performance and "other considerations" provided no significant discriminator among the three firms. The SSO decided that after upward adjustments were made to all cost proposals to project a most probable cost (MPC), the difference in cost between the firms was not "appreciable." The SSO concluded that since IIT, the incumbent, was significantly higher scored under mission suitability and there were no major discriminators under the remaining evaluation factors, IIT should be awarded the contract.

ECI argues that its proposal was unreasonably downgraded for risk concerns which are not present in its proposal, and that, if properly evaluated, its low cost proposal would have been determined to constitute the best value to the government.

In reviewing protests against allegedly improper evaluations, it is not our role to reevaluate proposals. Rather, our Office examines the record to determine whether the agency's judgment was reasonable and in accord with the RFP's stated evaluation criteria. Abt Assocs., Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD Para. 223. In order for us to review an agency's selection determination, an agency must have adequate documentation to support its selection decision. Arco Management of Wash., D.C., Inc., B-248653, Sept. 11, 1992, 92-2 CPD Para. 173. While adjectival ratings and point scores are useful as guides to decision-making, they generally are not controlling, but rather, must be supported by documentation of the relative differences between proposals, their weaknesses and risks, and the basis and reasons for the selection decision. Federal Acquisition Regulation (FAR) Secs. 15.608 and 15.612(d)(2); see also S&M Prop. Management, B-243051, June 28, 1991, 91-1 CPD Para. 615.

After reviewing all of the supporting documentation submitted by NASA, we conclude that the technical evaluation is not adequately supported. We also conclude that without adequate support for the technical evaluation, a proper award determination could not be made. See Redstone Technical Servs.; Dynamic Science, Inc., B-259222 et al., Mar. 17, 1995, 95-1 CPD Para. 181; see Arco Management of Wash., D.C., Inc., supra.

The SSO's selection decision memorandum summarizes the evaluation of ECI's proposal by the technical evaluation committee as follows:

"The ECI proposal had an overall adjective rating of "Very Good" and was scored higher in Mission Suitability than [firm A], but lower than [IIT]. ECI demonstrated a thorough understanding of the comprehensiveness of the system proposed for receiving/originating, planning, scheduling, processing, controlling, completing, reporting status of all tasks to be completed. ECI's proposal indicated an understanding of the system's work controls, efficiency, and capability for rapid reaction to changes in priority assignments. ECI's risk analysis was incomplete for Understanding the Requirement. It did not clarify risks associated with performing tests, safety issues/hazardous conditions, and maintenance of equipment. ECI presented a sound approach in their Management Plan, Configuration Control Plan, Personnel Certification Plan, and Government Property Management Plan. ECI's risk analysis was incomplete for Management Approach. It did not clarify risks associated with the depth of management or risk of placing emphasis on project manager. ECI proposed qualified and available key personnel and full local autonomy. ECI's risk analysis was incomplete for Staffing Plan. ECI did not address availability and replacement of personnel and flexibility and cross-utilization of work force."

As quoted above, the technical evaluation committee reported that ECI's proposal, although very good and demonstrating a thorough understanding of the system and its components, was incomplete with regard to risk analysis under each of the subfactors. However, other than three conclusory statements concerning risk under each subfactor in the final evaluation document, there are no other evaluation documents in the record which support or explain these conclusions, the scoring, and the final rating for mission suitability. In contrast, the protester has provided unrebutted, detailed comments to our Office, pointing out specific areas in its proposal where it addressed the agency's concerns. NASA has not provided an explanation of its concerns about risk in response to the protest, and although we have examined the evaluation record in detail, we cannot determine whether the information in the proposal was considered by the agency or, if considered, why the proposal was viewed as posing risk under the three mission suitability subfactors.

In their report to the SSO, the evaluators state that ECI did not clarify the risks associated with performing tests, safety issues/hazardous conditions, and maintenance of equipment. The record shows that ECI submitted specific plans for operations, tests, and safety and equipment, including a safety and health plan. ECI argues that its plan minimized any significant risk and therefore completely addressed the agency's concerns. There is no explanation in the evaluation record as to why the information furnished by ECI was viewed as incomplete or what specific risks in ECI's plans for tests and safety issues, etc. were not clarified.

The evaluators also found that ECI did not clarify risks associated with the depth of management or the project manager. The protester's proposal addressed its management structure in some detail and specified an organizational structure with several key people to perform different management-related functions. One individual was proposed as the project manager and another as an associate project manager. The proposal also provided that senior technical staff would serve as individual task managers and provided organizational charts outlining the interrelationships among the management. There are no specific evaluation comments, other than the one quoted above which explain what specific concerns the agency had with ECI's management approach. For example, to the extent the concern was with ECI's management depth, ECI proposed an associate project manager, with responsibilities divided among key personnel and task managers; the record does not show why this approach was viewed as inadequate.

Finally, ECI's staffing plan risk analysis was found incomplete because ECI did not address "availability and replacement of personnel and flexibility and cross-utilization of work force." Again, ECI points to specific references in its proposal which addressed these matters. For example, ECI's proposal devotes several pages to its staffing. ECI planned to hire all incumbent employees who had performed the work for the incumbent for the past 5 years and furnished commitment letters from these individuals. The same number of employees ECI proposed currently perform the work. ECI also stated that alternate resumes were available if these employees became unavailable. ECI provided a recruiting plan which explained how it would hire additional qualified staff. ECI also discussed how its organizational approach provided flexibility and cross-training of staff. Finally, ECI's approach addressed cross-utilization of its work force. In its proposal, ECI discussed how all maintenance personnel were to be cross-trained to allow at least three certified operators per unit to ensure that voluntary turnover or sickness would not stop the operation of a unit. ECI also proposed four key personnel because they could contribute effectively in different management operations, and ECI supported this with a chart showing their varied expertise and resumes. Thus, ECI addressed staff availability, recruiting, flexibility, and cross-utilization, which were the areas of concern listed by the agency. The record, again, fails to show the specific perceived inadequacies and risks of ECI's staffing approach.

In sum, while the agency may have had concerns about the risks of ECI's approach under the mission suitability subfactors which justify the selection decision, the evaluation record furnished to our Office--summary adjectival ratings and scores, the conclusory evaluation report, and the source selection document--contains no explanation of NASA's concerns about the risk associated with ECI's proposal. NASA, in its response to the protest, also fails to provide an explanation for its concern about risk. Moreover, our own scrutiny of the record reveals no obvious or apparent basis for that concern. Accordingly, on this record, we cannot determine that the selection decision, which in essence was based on unsupported and undocumented evaluation conclusions, was reasonable.

The protest is sustained. [2]

By letter of today to the Administrator, we are recommending that the agency, in accordance with the FAR, reevaluate the proposals, document its evaluation, and make a new selection decision. If after reevaluation NASA believes further discussions with offerors are warranted, it may reopen discussions and request another round of BAFOs. If the agency decides that IIT is no longer in line for award, the agency should terminate the award to IIT and make another award. In any event, ECI is entitled to the costs of filing and pursuing its protest, including reasonable attorneys' fees. 4 C.F.R. Sec. 21.6(d)(1) (1995). ECI should submit its certified claim for such costs, detailing the time expended and costs incurred, directly to the agency within 60 days after receipt of this decision. 4 C.F.R. Sec. 21.6(f)(1).

1. For mission suitability, the adjectival ratings were based on the following scoring: excellent (91-100 percent of the points); very good (71-90 percent of the points); and good (51-70 percent of the points).

2. ECI also argues that the incumbent awardee's project manager may have prepared a draft statement of work (SOW) for this RFP. The agency reported to our Office the results of the contracting officer's investigation of this allegation. The contracting officer interviewed the contracting officer's technical representative (COTR) who was responsible for preparing the SOW. The COTR states that the person in question had no involvement in the SOW's preparation; that the COTR wrote the SOW; and that the COTR used the current contract as his guide. The COTR reports that he did request the incumbent project manager to supply information for any work not described under the current contract SOW. In response, IIT furnished an updated equipment list. The COTR has provided an affidavit to our Office confirming these facts. The agency also points out that the incumbent's contract, including the SOW, was available and was in fact requested and released to the other competitors. In response to this report, the protester has not provided any basis to question the COTR's affidavit. We have no basis to conclude that there was any improper conduct.