B-244475.3, October 23, 1991

B-244475.3, October 23, 1991

DIGEST: Agency was not required to conduct discussions with protester where based on evaluators' advice that proposal was unacceptable in all three technical areas, as well as one of three management areas, contracting officer reasonably determined that proposal was technically unacceptable; protester fails to rebut agency conclusion in two of three technical areas; and in any event, evaluations of proposed specification and draft configuration management plan were in accordance with evaluation factors where, although solicitation called for submission of "draft" specification and "preliminary" configuration management plan, agency could reasonably expect documents of a certain level of quality, which would demonstrate an offeror's ability to deliver a document in final format within the schedule set forth by the solicitation.

Attorneys

Merit Technology, Inc.:

Merit Technology, Inc. protests the rejection of its proposal under request for proposals (RFP) No. F33657-91-R-0005, issued by the Department of the Air Force as a 100 percent small business set-aside, for a simulator for electronic combat training (SECT). The protester contends that the agency improperly excluded its proposal from the competitive range and therefore should not have canceled the RFP.

We deny the protest.

On February 13, 1991, the agency issued the solicitation for a cost plus- award-fee contract for development and acquisition of the SECT, with a follow-on firm-fixed-price contract with economic price adjustment for logistics support. The solicitation sought development of a supportable, software-based, mission simulator for airborne electronic combat, which could be easily modified to address advances in Air Force electronic combat equipment, operational electronic combat missions, and the changing threat environment. /1/

The simulator was to consist of nine student stations capable of independent operation and two instructor consoles for monitoring and controlling the student stations; it was to simulate a representative cockpit environment and realistic interactive threat and be capable of modifying the simulated mission on an on-line, real-time basis at the discretion of the instructors. The design was to provide for overall system support through a training system support center; the contractor would provide all hardware, software, firmware, spares, support equipment, and operations and maintenance documentation required for the system to be fully operational.

The statement of work called for design, development, integration, test and validation of the SECT computer programs/data, incorporating ease of operation, software maintenance, future updates and modifications, and smart designs that could justify a reduction in required maintenance documentation. The successful offeror was to plan, develop, implement and maintain a software development effort, in accordance with MIL-STD-1815A Ada programming language, on newly developed software; unless the offeror proposed software commercially available off-the-shelf and unmodified, the statement of work required the contractor to write reusable software in Ada. /2/

The agency instructed offerors to submit their proposals in seven volumes, including an executive summary, information on past performance history, technical, management, cost and supportability proposals, and contract administration information. The RFP further instructed each offeror to organize volume II, the technical proposal, into six parts, to demonstrate the offeror's understanding of system requirements and technical expertise. In the technical volume, each offeror was specifically to address the system requirement document (SRD), attachment 2 to the solicitation, and the "contractor generated system specification."

In part 1 of the technical volume, the offeror would provide a brief summary of its approach; in part 2, the offeror would define all the TBDs ("to be determined") in the SRD, then use the SRD requirements to develop a system specification. The solicitation advised offerors that the specification would become part of the contract at award, and would constitute the "preliminary draft submittal" required by the contract data requirements list.

Part 3 of the technical volume was to cover engineering development, including a systems engineering master schedule with event-oriented milestones, setting forth the technical accomplishments necessary to meet each milestone, indicating essential tasks/plans including lead times in a sequential manner of completion for accomplishment of each milestone, with essential tasks fixed to days before the milestone review or demonstration, rather than to specific dates, for each offeror as well as its major subcontractors. The solicitation instructed each offeror to describe its approach to product assurance, support systems and engineering organization. Part 4 addressed computational system development, including a discussion of the offeror's proposed system and software development process.

Part 5, electronic combat simulation, required discussion of the overall approach to electronic combat equipment simulation and simulations of the electronic combat environment, including the library for jammer, artillery, radar, and missile (JARM) systems, scenario generation, instantaneous electronic combat environment and cockpit environment. Part 6 provided information for reviewing software development capability/capacity, including management approach, management tools, development process, personnel resources, and Ada technology.

Management proposals, volume III, were to have four parts: (1) summary; (2) program management (resources/organization, master program schedule, subcontract management/teaming); (3) configuration/data management (overall configuration management including a "preliminary" version of the configuration management plan defining methods, policies, and procedures for the engineering release system, control of engineering and specification changes, deviations, and waivers, interface control, functional and physical configuration audits, commercial configuration management, software configuration management, specification maintenance practices, and status accounting system; data management); (4) test and deployment management (system test control; system test program; system deployment).

The solicitation provided for a detailed evaluation considering the offeror's soundness of approach, its understanding/compliance with the requirements, and the risk associated with the proposed effort. The four areas of evaluation, in descending order of importance, were as follows: technical (engineering development, computational systems development, and electronic combat simulation), management (program management, configuration/data management, and test and deployment management), cost (realism, reasonableness, and completeness), and supportability (integrated logistics support and contractor logistics support).

The RFP provided for selection of one contractor for the development and support contracts based on an integrated assessment of each proposal in terms of its response to the solicitation, agreed upon terms and conditions, and ability to satisfy the requirements of the solicitation, and a determination of which contractor could best satisfy the government's needs based on the requirements of the solicitation.

The agency received 12 proposals by March 18, the date set for receipt of initial proposals. The agency eliminated one proposal from an offeror that had been proposed for debarment and evaluated those that remained. Upon review of the evaluations and discussions with the evaluation team, the contracting officer found that the 11 proposals were technically unacceptable and concluded that none was susceptible to correction. /3/ By letter dated June 3, the agency advised the protester and the other offerors that their proposals were outside the competitive range and that it was canceling the solicitation and would issue a new solicitation on an unrestricted basis. This protest followed.

The protester essentially argues that the agency failed to evaluate its proposal in accordance with the specified evaluation factors; specifically, the protester argues that the solicitation called for a "draft" specification, a draft software development plan and a "preliminary" configuration management plan, but that in reviewing proposals, agency evaluators effectively evaluated proposals as if the RFP had required offerors to submit products in final form with their proposals. The protester further contends that in a solicitation for a development effort, the requirement for an acceptable specification was unattainable, unreasonable, arbitrary and capricious.

For each evaluation subfactor, the agency assigned two ratings, a color code for identifying strengths and weaknesses in the proposals and a risk assessment. The color codes were as follows: blue, exceptional; green, acceptable; yellow, marginal ("Fails to meet evaluation standards; and has low probability of satisfying the requirement; and has significant deficiencies, but correctable"); red, unacceptable ("Fails to meet a minimum requirement; and deficiency requires a major revision to the proposal to make it correct"). Risk assessments were as follows: high, moderate, and low.

The protester received an overall rating of red in the technical and management areas. The protester received a red/unacceptable rating, with high risk, in all three technical subfactors-- engineering development, computational systems development, and electronic combat simulation. the management area, the protester received a rating of red with high risk for configuration/data management, as well as a rating of yellow/moderate risk in the area of test and deployment management.

In the engineering development area, the evaluators found that the protester did not provide a system engineering master schedule in accordance with solicitation instructions in that it failed to show task completion in terms of days in advance of milestones and failed to break out subcontractor tasks. Not only did the protester fail to address many of the specification requirements, but where Merit's proposal did address them, in many instances, it merely parroted language of the SRD, with no indication of the offeror's understanding of and approach to solving potential problems. The evaluators also found that the protester failed to offer sufficient resources or adequate information to insure a system safety program in accordance with requirements. The evaluators advised the contracting officer that the proposal did not demonstrate the protester's ability to meet the minimum requirements of the solicitation or to produce a functional model.

In the area of software and hardware development, the agency found that the protester's software design methodology presented no architectural guidelines and failed to address maintenance and updating; the experience of its personnel was marginal; and the proposal did not meet standards for software work definition and subcontractor technical direction. The proposal also was viewed as unclear with reference to the use of off-the- shelf and nondevelopmental software. Regarding hardware, the agency found that the protester's experience with its tool set and compiler was questionable, and that in any event its design methodology was inappropriate to real-time simulation. In addition, the evaluators found that the protester's personnel turnover rate was high, and that there was no formal training program for its personnel. The protester also was found to have little experience with its tool set; further, the evaluators were concerned with the protester's proposed new and immature compiler. The evaluators believed that the proposal created a high risk of schedule disruption and degradation of system performance.

In the area of electronic combat simulation, the evaluators rated the protester's soundness of approach as poor and even "simplistic," producing a "canned" reaction to the various combat environments. The protester proposed but a single model for each JARM, changing only the parameters of that model to accommodate vastly different scenarios. The agency was concerned that the proposal did not demonstrate an understanding of the requirements, which involved complex combat environment simulations. The evaluators found that the proposal did not make any attempt to explain the design criteria for simulating lethality envelopes, JARM tactics, JARM capabilities and limitations, reloading time, or countermeasure effectiveness. The evaluators concluded that the proposal showed no understanding of the overall complexity of the work involved in creating a simulation to meet all the SECT requirements, with their emphasis on real- time equipment interaction and the real-world electronic combat environment.

Regarding configuration/data management, the evaluators found that the configuration management plan was incomplete, inaccurate and did not conform to solicitation requirements. The plan was contradictory in portions, failed to address critical requirements and left much of the proposed effort undefined. The evaluators concluded that the protester did not understand requirements or the necessity for ensuring configuration management of the hardware and software, and that there were no controls in place to insure that the protester could ever deliver a compliant system.

Merit concedes that its proposal contained deficiencies. However, Merit argues that those associated with the protester's proposed system specification and configuration management plan were the result of an improper evaluation and that the remaining deficiencies were minor and readily correctable. An agency is not required to include an offeror in the competitive range when the proposal, to be acceptable, would have to be revised to such a magnitude that as such it would be tantamount to a new proposal. Source AV, Inc., B-234521, June 20, 1989, 89-1 CPD Para. 578. Even where individual deficiencies may be susceptible to correction through discussion, the aggregate of many such deficiencies may preclude an agency from making an intelligent evaluation, and the agency is not required to give the offeror an opportunity to rewrite its proposal. Ensign-Bickford Co., B-211790, Apr. 18, 1984, 84-1 CPD Para. 439.

Our review of the record shows that while the evaluation standards were indeed high and were strictly imposed, they were nevertheless consistent with those set forth in the solicitation. In its instructions to offerors, the solicitation clearly called for the submission of a proposed system specification addressing all portions of the SRD. While the RFP referred to a "draft" specification, the presolicitation question and answer session along with a reference in the contract data requirements list put firms on notice that the agency expected a document that addressed requirements and which could be updated to final form at a relatively early stage in contract performance. /4/ We do not find the agency unreasonable in evaluating proposals on that basis.

Further, regardless of whether the protester submitted a draft or a final format, the evaluation factors clearly called for Merit to demonstrate some degree of understanding of solicitation requirements and the effort required to meet them in accordance with the solicitation schedule. Here, the Air Force found that Merit's proposal in the area of engineering development failed to provide adequate scheduling information, failed to fully delineate the prime and subcontractor relationship for performance of system engineering tasks, and in certain cases, parroted back system requirements. While Merit asserts that it provided a "draft" in this regard, which is all that the RFP allegedly required, we think the Air Force reasonably could conclude that a draft parroting back the system requirements and omitting critical schedule milestones did not establish Merit's understanding of RFP requirements. We have no basis for concluding that the evaluation was either unreasonable or inconsistent with the evaluation criteria.

In any event, apart from the system specification and configuration management plan, Merit was found technically unacceptable in two technical areas where Merit has not attempted to rebut the evaluation. In software development, the Air Force found that the proposal failed to establish the firm's ability to furnish the necessary essential software in a timely manner. Evaluators concluded that the software design methodology lacked architectural guidelines, contained no provision for maintenance and updating, and did not propose personnel with the experience necessary to perform the software effort. Furthermore, under electronic combat simulation, the protester does not rebut the Air Force's findings that the simulations were "simplistic" and not adequately supported by a design approach for simulating certain critical tactical problems. We think the Air Force could reasonably conclude that the software development and electronic combat simulation approaches were unacceptable. Since we find that the agency's technical evaluation was reasonable and consistent with the evaluation criteria, we find that the contracting officer's decision to eliminate the protester's proposal from further consideration was also reasonable. See Bay Tankers, Inc., 69 Comp.Gen. 403 (1990), 90-1 CPD Para. 389; Suncoast Scientific, Inc., B-240689, Dec. 10, 1990, 90-2 CPD Para. 468

The protest is denied.

/1/ Training includes strategic/covert penetration, standoff jamming/direct support, electronic intelligence collection, and suppression of enemy air defenses and addresses the full flight profile for electronic combat missions.

/2/ Ada is a computer language specified for use in Department of Defense data processing applications.

/3/ The evaluation report did indicate a potential competitive range of four firms, based on the number of major deficiencies, for consideration if the contracting officer disagreed with the report's conclusion that no firm had a reasonable chance for award.

/4/ A question/answer log sent to all prospective offerors contained the following:

"Q: 'Does the System Specification submitted with the contractor's proposal, and as approved at contract award, become part of the contract?'

"A: 'Yes.'"

The contract data requirements list in fact referred to the final submission as an "update" rather than a revision of that proposed.