B-244475.2, October 23, 1991

B-244475.2, October 23, 1991

DIGEST: 1. Evaluation of proposed specification and draft configuration management plan was in accordance with evaluation factors where, although specification called for submission of "draft" specification and configuration management plan, agency could reasonably expect documents of a certain level of quality, which would allow a contractor to deliver a document in final format within the schedule set forth by the solicitation. 2. Agency decision not to engage in discussions with protester was reasonable where, based on technical evaluation committee findings, contracting officer determined that the proposal was unacceptable and in need of major rewriting and substantial clarification in several areas.

Attorneys

CTA, Inc.:

CTA, Inc. (CTA) protests the rejection of its proposal under request for proposals (RFP) No. F33657-91-R-0005, issued by the Department of the Air Force as a 100-percent small business set-aside, for a simulator for electronic combat training (SECT). CTA contends that the agency improperly excluded its proposal from the competitive range and improperly has decided to cancel and reissue the solicitation on an unrestricted basis.

We deny the protest.

On February 13, 1991, the agency issued the solicitation for a cost plus- award-fee contract for development and acquisition of the SECT with a follow-on firm-fixed-price contract with economic price adjustment for logistics support. The solicitation sought development of a supportable, software-based, mission simulator for airborne electronic combat, which could be easily modified to address advances in Air Force electronic combat equipment, operational electronic combat missions, and the changing threat environment. /1/

The simulator was to consist of nine student stations capable of independent operation and two instructor consoles for monitoring and controlling the student stations; it was to simulate a representative cockpit environment and realistic interactive threat, with the capability of modifying the simulated mission on an on-line, real-time basis at the discretion of the instructors. The design would provide for overall system support through a training system support center; the contractor would provide all hardware, software, firmware, spares, support equipment, and operations and maintenance documentation required for the system to be fully operational.

The statement of work called for design, development, integration, test and validation of the SECT computer programs/data, incorporating ease of operation, software maintenance, future updates and modifications, and smart designs that could justify a reduction in required maintenance documentation. The successful contractor was to plan, develop, implement and maintain a software development effort, in accordance with MIL-STD- 1815A Ada programming language, on newly developed software; unless the offeror proposed software commercially available off-the-shelf and unmodified, the statement of work required the contractor to write reusable software in Ada. /2/

The agency instructed offerors to submit their proposals in seven volumes, including an executive summary, information on past performance history, technical, management, cost and supportability proposals, and contract administration information. The RFP further instructed each offeror to organize volume II, the technical proposal, into six parts, to demonstrate the offeror's understanding of system requirements and technical expertise. In the technical volume, each offeror was specifically to address the system requirement document (SRD), attachment 2 to the solicitation, and the "contractor-generated" specification.

In part 2 of the technical volume, after a brief summary of the offeror's approach in part 1, the offeror was to define all the TBDs ("to be determined") in the SRD, then use the SRD requirements to develop a system specification. The solicitation advised offerors that the specification would become part of the contract at award, and would constitute the "preliminary draft submittal" required by the contract data requirements list.

Part 3 of the technical volume was to cover engineering development, including a systems engineering master schedule with event-oriented milestones, setting forth the technical accomplishments necessary to meet each milestone, indicating essential tasks/plans including lead times in a sequential manner of completion for accomplishment of each milestone, with essential tasks fixed to days before the milestone review or demonstration, rather than to specific dates, for each offeror as well as its major subcontractors. The solicitation instructed each offeror to describe its approach to product assurance, support systems and engineering organization. Part 4 addressed computational system development, including a discussion of the offeror's proposed system and software development process.

Part 5, electronic combat simulation, required the discussion of the overall approach to electronic combat equipment simulation and simulations of the electronic combat environment, including the library for jammer, artillery, radar, and missile systems, scenario generation, instantaneous electronic combat environment, and cockpit environment. The RFP specifically required discussion of simulation of and limits on simultaneous signals. Part 6 provided information for reviewing software development capability/capacity, including management approach, management tools, development process, personnel resources and Ada technology.

Management Proposals, volume III, were to have four parts: 1) summary; 2) program management (resources/organization; master program schedule; subcontract management/teaming); 3) configuration/data management (overall configuration management including a preliminary version of the configuration management plan defining methods, policies and procedures for the engineering release system, control of engineering and specification changes, deviations, and waivers, interface control, functional and physical configuration audits, commercial configuration management, software configuration management, specification maintenance practices, and status accounting system; data management); 4) test and deployment management (system test control; system test program; system deployment).

The solicitation provided for a detailed evaluation considering the offeror's soundness of approach, its understanding/compliance with the requirements, and the risk associated with the proposed effort. The four areas of evaluation, in descending order of importance, were as follows: technical (engineering development, computational systems development, and electronic combat simulation), management (program management, configuration/data management and test and deployment management), cost (realism, reasonableness and completeness), and supportability (integrated logistics support and contractor logistics support).

The RFP provided for selection of one contractor for the development and support contracts based on an integrated assessment of each proposal in terms of its response to the solicitation, agreed upon terms and conditions, and ability to satisfy the requirements of the solicitation, and a determination of which contractor could best satisfy the government's needs based on the requirements of the solicitation.

The agency received 12 proposals by March 18, the date set for receipt of initial proposals. The agency eliminated one proposal from an offeror that had been proposed for debarment and evaluated those that remained. Upon review of the evaluations and discussions with the evaluation team, the contracting officer found that the 11 proposals were technically unacceptable and concluded that none was susceptible to correction. /3/ By letter dated June 3, the agency advised CTA and the other offerors that their proposals were outside the competitive range and that it was canceling the solicitation and would issue a new solicitation on an unrestricted basis. This protest followed.

The protester essentially argues that the deficiencies and weaknesses that the agency's evaluation team identified in its proposal were generally minor in nature, concerned more with form and administrative requirements than with technical substance. The protester takes issue with portions of the evaluation, but in any event asserts that it can make all necessary corrections to its proposal within a relatively short time. Since the protester has offered a relatively low price, well within program funds, CTA believes that the agency improperly canceled the RFP because the firm has a reasonable chance for award if the agency will allow it to correct its proposal through discussions.

Federal Acquisition Regulation (FAR) Sec. 15.609(a) requires that the competitive range include all proposals that have a reasonable chance of being selected for award, including deficient proposals that are reasonably susceptible of being made acceptable through discussions. Bay Tankers, Inc., 69 Comp.Gen 403 (1990), 90-1 CPD Para. 389. An agency is not, however, required to include an offer in the competitive range when the proposal, to be acceptable, would have to be revised to such a magnitude that as such it would be tantamount to a new proposal. Source AV, Inc., B-234521, June 20, 1989, 89-1 CPD Para. 578.

In reviewing protests against an agency's technical evaluation and decision to eliminate a proposal from the competitive range, we review the record to determine whether the agency's judgments were reasonable and in accordance with the listed evaluation criteria and whether there were any violations of procurement statutes or regulations. See Bay Tankers, Inc., supra, 69 Comp.Gen at 406, 90-1 CPD Para. 389 at 4; Suncoast Scientific, Inc., B-240689, Dec. 10, 1990, 90-2 CPD Para. 468. We find that the agency's technical evaluation in this case was reasonable and consistent with the evaluation criteria, and we conclude that the contracting officer's decision to eliminate the protester's proposal from further consideration was reasonable.

For each evaluation subfactor, the agency assigned two ratings, a color code for identifying strengths and weaknesses in the proposals and a risk assessment. The color codes were as follows: blue, exceptional; green, acceptable; yellow, marginal ("Fails to meet evaluation standards; and has low probability of satisfying the requirement; and has significant deficiencies, but correctable"); red, unacceptable ("Fails to meet a minimum requirement; and deficiency requires a major revision to the proposal to make it correct"). Risk assessments were as follows: high, moderate, and low.

The protester received an overall rating of red in the technical area, and a red/unacceptable rating in two of the three technical subfactors: engineering development (moderate risk) and computational systems development (high risk). In the third subfactor, electronic combat simulation, the protester was rated yellow/marginal with low risk. The agency rated the protester yellow/marginal in the management area, with a red/unacceptable rating in the subfactor of configuration/data management; the agency found the protester acceptable in the remaining management subfactors as well as the supportability area and subfactors.

In the engineering development area, the evaluators found that the protester's proposed system specification did not meet minimum requirements because it failed to provide a requirements verification paragraph for each paragraph of section 3 of the system specification, as required by the SRD. The evaluators also found the protester's text matrix, required by the SRD as a means of establishing the methods of verification and qualification to be applied by the procuring activity, to be inadequate because it did not provide for verifying by test such things as instructor-student interactions, training system support center functions and mission fidelity requirements. Further, by implying that certain requirements would be "goals" and failing to provide a firm commitment to meet all requirements, the protester exhibited, in the opinion of evaluators, a lack of understanding of the agency's essential requirements. The evaluators additionally found that the protester's systems engineering master schedule did not provide for completion of essential tasks and plans in terms of days before the milestone review and failed to address significant contract data deliverables such as safety analyses reports. The evaluators advised the contracting officer that these omissions from the system specification could cause performance degradation, while the inability to track progress towards meeting requirements could result in schedule disruption and increased cost. They advised the contracting officer that, at the very least, correction of the system specification would require a major revision of the proposal.

In the area of computational systems development, the agency found much of the protester's proposed software consisted of previously designed C code software for executive functions, with portions translated to Ada and new software written in Ada. Evaluators were unable, using information in the proposal, to evaluate the design or the design process, and found that the proposal did not address the methodology and effort involved in translating and integrating C and Ada codes.

The evaluators advised the contracting officer that the proposal as submitted did not incorporate the required ease of software maintenance and future updates, nor did it meet requirements for Ada structural modeling or communication between packages. Translation of C to Ada, as proposed by CTA, they advised, could present a high risk in the absence of adequate explanation of the process involved, threatening schedule disruption and system degradation; again, evaluators believed that correction would require a major rewrite of the proposal. The configuration management plan did not identify organizational relationships, authority/responsibility, or policies and procedures to establish formal configuration control. The proposal did not identify a software configuration control board for evaluation of software specific configuration changes, nor define the use and application of company standards, status accounting, and cost controls. The data management reviews appeared excessive in duration and the test/demonstration matrix did not cover the verification requirements missing from section 4 of the system specification. In the opinion of evaluators, the offeror demonstrated little understanding of the configuration management process and necessary procedures; evaluators concluded that the plan presented a high risk of cost overruns, schedule disruption and an inability to provide adequate logistics support capability.

The protester contends that these evaluation comments demonstrate the agency's failure to express in the solicitation its true technical requirements or the level of data expected; the protester concludes that the agency applied different, higher standards to the evaluation than those set forth in the solicitation. The protester argues that it was unreasonable for the agency to demand a refined, complete systems specification and configuration management plan in a development contract. The protester also contends that many of the weaknesses in its proposal were mere omissions of information and, in other instances, a failure by the agency to understand the proposal. The protester asserts that it could easily have addressed the agency's concerns in discussions.

Our review of the record shows that while the evaluation standards were indeed high and were strictly imposed, they were nevertheless consistent with those set forth in the solicitation. In its instructions to offerors, the solicitation clearly called for the submission of a proposed system specification addressing all portions of the SRD. While the RFP referred to a "draft" specification, the presolicitation question and answer session along with a reference in the contract data requirements list put firms on notice that the agency expected a document that addressed requirements and which could be updated to final form at a relatively early stage in contract performance. /4/ We do not find the agency unreasonable in evaluating proposals on that basis.

Our review further shows that the agency had a reasonable concern as to whether the protester could provide the required system specification, system, and plans as the schedule required. By failing to address verification requirements and to commit itself to meet requirements, as well as by omitting material from the system engineering master schedule, the protester failed to establish its understanding of what the agency expected, and failed to establish its willingness to commit itself to meet essential requirements on schedule, particularly inspection and testing.

Further, the agency reasonably found the omission of necessary detail on the method of translating C language programs into Ada and the interface between the operating portions of the system, which used the two programming languages, was a serious deficiency. As the evaluators noted, the failure to demonstrate an understanding of the importance and difficulty of insuring a smooth interface between portions of the software written in the different languages, both in the translation into Ada of software originally written in C and in the communication between software in two different languages, compromised the ultimate performance of the system and, in any event, could have severely disrupted schedules. Information that the protester now offers to demonstrate its experience and ability in this area does not excuse its failure to discuss the matter fully in its initial proposal.

Finally, the protester's failure to address basic concerns regarding organizational relationships, authority, and responsibility in its configuration management plan, a significant tool to insure timely delivery of a system meeting specifications, gave rise to a reasonable concern regarding the protester's understanding of and commitment to its management responsibilities.

The determination that a proposal is unacceptable and outside the competitive range may be based upon the aggregate of many deficiencies, even where many of the individual deficiencies may be susceptible to correction through discussion, where the number of deficiencies preclude an agency from making an intelligent evaluation. See Ensign Bickford Co., B-211790, Apr. 18, 1984, 84-1 CPD Para. 439. While the protester asserts that in many instances the agency merely misunderstood its proposal and that in others CTA could have provided information to substantiate its offer or rewritten its proposal with a minimum of effort, its mere disagreement with the agency's assessment provides no basis for sustaining the protest. Further, while we agree that many of the deficiencies noted related to omission of information and data, omissions that might have been cured through discussions, the agency could reasonably regard others, such as the failure to address interface and translation between C and Ada, the discussion of requirements as goals, and the inadequate configuration management plan, as failing to demonstrate an understanding of the requirements and the problems and effort needed to meet those requirements. Accordingly, the agency had a reasonable basis to exclude CTA from the competitive range.

The protest is denied.

/1/ Training includes strategic/covert penetration, standoff jamming/direct support, electronic intelligence collection, and suppression of enemy air defenses and addresses the full flight profile for electronic combat missions.

/2/ Ada is a computer language specified for use in Department of Defense data processing applications.

/3/ The evaluation report did indicate a potential competitive range of four proposals, based on the number of major deficiencies, for consideration if the contracting officer disagreed with the report's conclusion that no firm had a reasonable chance for award.

/4/ A question/answer log sent to all prospective offerors contained the following:

"Q: 'Does the System Specification submitted with the contractor's proposal, and as approved at contract award, become part of the contract?'

"A: 'Yes.'"

The contract data requirements list in fact referred to the final submission as an "update" rather than a revision of that proposed.