Matter of: Aerospace Design, Inc. File: B-247793 Date: July 9, 1992

Matter of: Aerospace Design, Inc. File: B-247793 Date: July 9, 1992

PROCUREMENT Competitive Negotiation Offers Evaluation errors Evaluation criteria Application PROCUREMENT Competitive Negotiation Requests for proposals Evaluation criteria Production capabilities Quality control Where solicitation evaluation scheme emphasizes manufacturing approach and matters related to product quality rather than simply knowledge of the components and the workings of a device to be manufactured, the assignment of high evaluation scores to a proposal with an approach that stresses up-front engineering and design and quality management as a means to best assure successful and timely completion of the contract, and the assignment of lower or unacceptable scores to offers that failed to propose such an approach, is consistent with the evaluation scheme. PROCUREMENT Competitive Negotiation Offers Evaluation Technical acceptability PROCUREMEMT Competitive Negotiation Offers Evaluation errors Evaluation criteria Application The evaluation of technical proposals is primarily the responsibility of the contracting agency; the agency is responsible for defining its needs and the best method of accommodating them and must bear the burden of any difficulties resulting from a defective evaluation. Therefore, General Accounting Office (GAO) will not make an independent determination of the merits of technical proposals; rather GAO will examine the agency evaluation record to ensure that it was reasonable and consistent with the stated evaluation criteria and applicable statutes and regulations. Mere disagreement with the agency does not render the evaluation unreasonable. PROCUREMENT Competitive Negotiation Offers Competitive ranges Exclusion Administrative discretion Proposal was properly excluded from competitive range where agency reasonably concluded that proposal was unacceptable because it failed to address many required areas of the solicitation, failed to meet the delivery schedule and failed to demonstrate that the offeror could successfully manufacture devices in a timely manner.

Attorneys

DECISION

Aerospace Design, Inc. protests the exclusion of its proposal from the competitive range under request for proposals (RFP) No. N60530-91-R-0358, issued by the Navy for electro-mechanical safety-arming and firing devices.

We deny the protest.

The solicitation contemplates the award of a firm fixed-price contract to manufacture MK-29 arming and firing devices for the High Speed Anti-radiation Missile. These devices are part of the command destruct system which allows the destruction of a missile that deviates from its approved test firing pattern. The solicitation requested offers for 30 first article units, 15 lot acceptance test units and 110 production units and includes an option for 15 additional lot acceptance test units and 105 production units.

The solicitation lists the following evaluation factors and subfactors. The factors are listed in descending order of importance while the subfactors are of equal weight:

A. TECHNICAL

1. Understanding of specifications/drawing requirements. 2. Manufacturing approach and controls for quality product. 3. Understanding of quality product requirements.

B. COST

C. MANAGEMENT

1. Detailed production schedule 2. Organizational structure 3. Resources and facilities 4. Related previous successes

The Navy received four proposals. In evaluating the proposals, the agency assigned the following weighted scores to the proposed awardee and the protester:

Maximum Possible Texas Points Inst. ADI

Technical 200 183 78 Management 80 65 22

Price $1,946,312 $441,115

Price score 120 22 97.2

Total 400 270 197.2

The agency's technical evaluators recommended including only the Texas Instruments proposal in the competitive range. In a memorandum to contracting officials, the technical evaluators stated that Texas Instruments was the only one of the four offerors that responded to the solicitation requirement for a proposal that integrates the principles of total quality management and that addressed a manufacturing approach which would deliver a quality device on the required schedule. According to the evaluators, ADI's proposal was based upon a plan to purchase all the parts and assemble them; it did not mention engineering support or total quality management approaches to the job and took exception to the required delivery schedule.

In deciding to include only the Texas Instruments proposal in the competitive range, the contracting officer stated that ADI and the two offerors other than Texas Instruments proposed an approach that on previous contracts resulted in quality deficiencies, schedule problems and the need for massive government assistance and oversight. According to the contracting officer, the Navy has historically procured these devices from the acceptable offeror with the lowest price. That approach, which led contractors to manufacture directly from the existing drawings, with minimal up-front engineering review and planning, resulted in problems in the manufacturing process and quality problems with the devices delivered. The contracting officer's competitive range determination stated that because of this history, the selection of a contractor was to be based on the proposal that offered the needed engineering support up-front and throughout contract performance. This approach, while expected to cost more, is expected to save the costs associated with unnecessary government expenditures due to quality problems.

In deciding to exclude ADI's proposal from the competitive range, the contracting officer stated that although ADI's proposal showed a number of strengths, including experience with a similar device, the MK-35 S&A, the stated availability of complete tooling which could result in a cost savings, an understanding of the device and its critical components, and personnel that have experience with similar items, the proposal was unacceptable. The contracting officer, like the evaluators, was concerned that ADI did not propose to meet the required delivery schedule, and proposed to purchase and assemble all parts of the devices, a practice which he thought would result in greater schedule risk.

The contracting officer further stated that ADI's proposed approach was to use personnel and tooling acquired from Micronics International, the last contractor for MK-29 devices. According to the contracting officer, ADI is owned and operated by individuals who were associated with Micronics, a firm that had several government contracts that resulted in quality problems, delinquent deliveries and finally default and bankruptcy. The competitive range determination stated that Micronics defaulted on its previous contract for MK-29 devices.

PROTEST ALLEGATIONS

ADI argues that its proposal met the RFP requirements and offered the government a cost savings and therefore it was improperly excluded from the competitive range. ADI primarily maintains that the evaluators gave Texas Instruments credit for its proposed emphasis on engineering and design skills in spite of the fact that the solicitation did not require such skills. According to the protester, since the devices are to be built to 10-year old drawings and specifications, such skills are largely unneeded. Beyond the evaluators' emphasis on engineering and design, ADI argues that the evaluation of its proposal was unreasonable, particularly in comparison to the evaluation of Texas Instruments' proposal. In addition, ADI argues that the competitive range decision did not include consideration of price. Finally, ADI maintains that the evaluators and contracting personnel were biased against ADI because of the firm's employment of former Micronics' personnel. According to ADI, the exclusion of the firm's proposal from the competitive range amounted to a determination of nonresponsibility which should have been referred to the Small Business Administration (SBA) for review under the certificate of competency (COC) program.

ANALYSIS

We conclude that the agency's evaluation of the competing proposals both reasonable and was consistent with the terms of the solicitation. We also conclude that the agency reasonably found ADI's proposal technically unacceptable and properly excluded it from the competitive range. Finally, we find no support for the contentions that contracting personnel were biased against ADI or that ADI's exclusion amounted to a finding of nonresponsibility.

The Solicitation

ADI argues that the emphasis in the evaluation of the proposals on engineering support and design was inconsistent with the terms of the solicitation. ADI states that the MK-29 is not a state-of-the-art device requiring up-front engineering and design support to manufacture. Rather, ADI states, the device is based on a 10-year old drawing package and the Navy has purchased as many as 1,000 of these items at prices ranging from $900 to $1,800 compared to Texas Instruments' price of almost $2 million for 275 of the devices. ADI notes that the RFP incorporates a 51-page specification, AS-1306, dated June 18, 1982, which includes numerous detailed drawings. According to ADI, notwithstanding the RFP terms, which require offerors to follow detailed government drawings and specifications with little engineering support expressly or implicitly required, in evaluating the proposals the Navy sought an extensive engineering effort with little regard for price.

We do not agree with ADI that an evaluation that rewarded proposed engineering and design support was inconsistent with the solicitation. In our view, the solicitation evaluation scheme emphasized proposed approach to successfully producing the device rather than a firm's knowledge of the components of the MK-29 and the workings of the device itself. For instance, two of the three subfactors under the most important evaluation factor related to manufacturing approach, quality controls and understanding quality product requirements. The solicitation also included management subfactors which stressed the agency's concern with offerors' capabilities, resources and past successes and the production schedule under which an offeror would perform the work.

In addition, the solicitation included instructions that technical proposals were to cover the offeror's "understanding of the proposed work" and its "proposed method of approach to attain contract objectives," and should demonstrate that the offeror has "a thorough understanding of the requirements for, and technical problems inherent in, the achievement of the specifications and work program herein described, and has a valid and practical solution for each contemplated problem." Also, according to the solicitation, technical proposals were to "convincingly show a depth of understanding of the specification and drawing requirements," were to include "[e]vidence of the offeror's ability to integrate all of the functions required to produce Arm-Fire Devices," and "should convincingly show a thorough understanding of what is necessary to deliver a quality product," including a showing of the offeror's understanding of the principles of total quality management.

The solicitation stated that proposals were to include a description of the management and administrative organization available or to be established for the contract and should include information concerning the firm's experience in performing government contracts for similar services. In addition, the proposal was to provide a production schedule with sufficient detailed planning to show an understanding of the contract requirements.

Although ADI is correct that the solicitation did not specifically state that offerors were required to propose up-front engineering and design effort, under the solicitation's evaluation and proposal preparation instructions it was clear that the agency was concerned with offerors' proposed approaches to performing the work. Since the RFP's evaluation scheme clearly emphasized manufacturing approach and matters related to product quality, the assignment of high scores to a proposal with an approach that stresses up-front engineering and design and quality management as a means to best assure successful and timely completion of the contract is not inconsistent with the RFP evaluation factors. Neither is the assignment of lower or unacceptable scores to proposals that failed to propose such an approach.

The Evaluation

ADI specifically challenges the evaluation and scoring of its proposal under a number of the evaluation subfactors. For instance, ADI notes that its proposal included 12 pages of analysis and discussion of the MK-29 device and argues that the agency report concedes that the evaluators found that ADI's proposal demonstrated a good understanding of the drawings and the long lead times for components and assemblies. ADI states that it was given a low score under the understanding of specifications/drawing requirements subfactor and argues that this was unreasonable in light of the extensive discussion of the device included in its proposal. ADI also argues that the high score given to Texas Instruments under that subfactor likewise was unreasonable in light of the comparative lack of detailed discussion of the MK-29 device in that firm's proposal.

The evaluation of technical proposals is primarily the responsibility of the contracting agency; the agency is responsible for defining its needs and the best method of accommodating them and must bear the burden of any difficulties resulting from a defective evaluation. Therefore, our Office will not make an independent determination of the merits of technical proposals; rather, we will examine the agency evaluation record to ensure that it was reasonable and consistent with the stated evaluation criteria and applicable statutes and regulations. Mere disagreement with the agency does not render the evaluation unreasonable. Litton Sys., Inc., B-239123, Aug. 7, 1990, 90-2 CPD Para. 114.

ADI's score under this subfactor was considered acceptable and the evaluators concluded that ADI's proposal showed an understanding of the MK-29 device and its components. Nonetheless, the Navy argues that understanding of the device itself was only part of this subfactor; the evaluators were also looking for an understanding of the specification and drawing "requirements," which the Navy maintains should include a recognition that the devices could not be successfully manufactured on schedule without significant production engineering. We think that the evaluators were reasonably concerned with whether a proposal demonstrated an understanding of what would be required to meet the requirements of the solicitation to successfully manufacture the devices on schedule and in accordance with the specifications and drawings.

Although ADI argues that its proposal included approximately 12 pages describing the MK-29 device and its operation, the description to which ADI refers essentially consists of a general explanation of how the MK-29 device is intended to operate. We think the acceptable rating given to ADI recognized the firm's understanding of the device and its operation. The solicitation, however, in our view, placed offerors on notice that more would be required. The solicitation stated that the technical proposal should cover the offeror's "understanding of the proposed work and your proposed method of approach to attain contract objectives." Also, offerors were instructed that their technical proposals should be sufficiently specific, detailed and complete to clearly and fully demonstrate "a thorough understanding of the requirements for, and technical problems inherent in, the achievement of the specifications and work program," and "a valid and practical solution for each contemplated problem." In other words, an understanding of the device itself was not sufficient for higher than an acceptable rating under this subfactor; we think that the solicitation reasonably indicated that the agency was looking for evidence that an offeror would perform the contract in a manner that would assure success.

Further, although Texas Instruments' proposal did not have the detail of ADI's regarding the functioning of the MK-29 device, the agency's evaluators were impressed that the proposal showed the firm's ability to successfully address and resolve manufacturing problems in a timely manner. The evaluators noted that Texas Instruments' proposal showed an understanding of the long lead times for components and the associated risks and thoroughly discussed risk management. Also, the evaluators were impressed that Texas Instruments proposed to devote management attention and engineering skills early in the program and that program controls were strongly represented with the objective of controlling and meeting contract schedules. Although ADI argues that Texas Instruments' proposal was overrated compared to ADI's proposal since it lacked a detailed explanation of the MK-29 device, we think that the credit given to Texas Instruments based on its discussion of its approach to manufacturing the devices was consistent with the evaluation scheme set out in the solicitation. We therefore have no basis to interfere with the evaluators' judgment that this represented a superior approach to the RFP. Litton Sys., Inc., supra.

ADI also argues that its proposal was unfairly criticized compared to Texas Instruments' proposal under the manufacturing approach and controls for quality product technical subfactor. ADI notes that it was given a low score and its proposal was criticized by the evaluators because it proposed to buy all the necessary parts and assemble them in-house. ADI argues that Texas Instruments also proposed to purchase all of the parts except a single circuit board yet it was given a much higher score under that subfactor.

We have no reason to dispute the Navy's evaluation of the proposals under this technical subfactor. The evaluators were concerned that ADI's proposal to purchase all parts for the devices would result in greater schedule risk. Although Texas Instruments stated it would purchase most of the parts, it proposed to manufacture the circuit board. The Navy explains that this circuit board has been particularly difficult to produce in a timely manner and has created significant problems for several previous MK-29 manufacturers to acquire. Given the Navy's high regard for Texas Instruments' proposed approach to producing a high quality product on schedule, it was in our view reasonable for the evaluators to rate highly the firm's plans to produce the circuit board itself rather than to rely on another vendor.

In addition, the Navy argues that another significant difference between the two firms was Texas Instruments' proposal to use production engineering techniques to work with its parts vendors to assure that parts are correctly manufactured in accordance with the drawings and specifications. Texas Instruments' proposal explains that on an earlier contract for a similar device it found that parts provided by the existing suppliers for that contract and other firing device contracts did not meet drawing specifications or quality requirements. In its proposal, Texas Instruments explains that although it costs more, it will purchase parts only from suppliers that have proven they will meet its quality requirements. The evaluators considered this to be an effective and desirable risk reduction technique and the Navy notes that ADI proposed nothing similar. Under the evaluation scheme set out in the solicitation, and given the Navy's concern with the problems of manufacturing MK-29 devices under previous contracts, we have no basis to question the different scores assigned under the manufacturing approach and controls for quality product technical subfactor.

ADI also challenges the evaluation of the proposals under the management subfactor, resources and facilities. ADI notes that the evaluation record states that one of ADI's proposal strengths was that complete tooling for the MK-29 was available while according to Texas Instruments' proposal it does not have tooling for the contract. According to ADI, under the evaluation plan prepared for this solicitation, a firm with tooling already in existence should have received a higher score than a firm that would have to create or acquire the needed tooling. Under these circumstances, ADI maintains that its low rating under this subfactor was unfair.

The Navy explains that Texas Instruments proposed to design, fabricate, test and validate new tooling and the agency's evaluators found this aspect of the proposal to be outstanding. With respect to ADI, the Navy explains that it was given a marginal rating because, although it stated that it had the necessary tooling on hand, its proposal did not identify the tooling that it had in its possession.

As far as the evaluation plan is concerned, alleged deficiencies in the application of such an agency plan do not alone provide a basis for questioning the validity of an award selection. Evaluation plans are internal agency instructions and as such do not give outside parties any rights. Mandex, Inc.; Tero Tek Int'l, Inc., B-241759 et al., Mar. 5, 1991, 91-1 CPD Para. 244. The agency is required to follow the evaluation scheme set forth in the RFP and to conduct its evaluation in a manner that will produce a rational result. Id. Here, nothing in the solicitation and its evaluation criteria prevented the agency from giving greater credit to Texas Instruments' proposal to create new tooling for the contract. Second, while the evaluation plan states that the evaluators should consider whether there are current facilities in place or whether they need to be obtained, given the agency's concern with the quality of devices manufactured under previous contracts for these items, we do not think that it was unreasonable, even under the terms of the evaluation plan, for the evaluators to criticize ADI's blanket proposal to use existing tooling. [1]

ADI also argues that the agency's conclusion that its proposal, as a whole, was unacceptable was unreasonable. ADI states that the contracting officer's competitive range determination cited a number of strengths in the ADI's proposal including experience with a similar device, the MK-35 S&A, the availability of complete tooling, which would result in a cost savings, an understanding of the firing device and its critical components, and personnel that have experience with similar items. According to ADI, given these strengths, the agency could not reasonably conclude that its proposal was unacceptable.

In spite of the strengths which the evaluators found in ADI's proposal, we think the evaluation and the determination that ADI's proposal was unacceptable were reasonable. The evaluators were concerned that ADI and the two offerors other than Texas Instruments proposed an approach that on previous contracts resulted in quality deficiencies, schedule problems and massive government assistance and oversight. The agency states that in this acquisition the devices were to be procured using a new approach which involved the selection of a contractor based on the proposal that offered the needed engineering support up-front and throughout the contract. As stated above, we think that a reasonable reading of the RFP and its evaluation factors should have made this clear to the offerors.

While the protester chooses to emphasize the positive comments made by the contracting officer concerning its proposal, the evaluation record also contains a considerable amount of less favorable material. For example, the contracting officer stated that ADI's proposal included no indication that total quality management principles are practiced in the company or that statistical process controls are used to assure a quality product. Also, according to the contracting officer, ADI proposed to purchase and assemble all parts of the devices, a practice that results in greater schedule risk. In addition, the contracting officer concluded that ADI took exception to the delivery schedule required by the solicitation. [2] Considering the RFP's stress on the quality aspects of the manufacturing process, and the fact that the contracting officer's concerns were mostly in that area, we think that the agency could have reasonably determined that ADI's proposal was unacceptable, despite the firm's strengths which primarily related to familiarity with the production of the device under prior contracts which did not have the same quality emphasis.

The Competitive Range Exclusion

Since we have found no basis upon which to question the agency's conclusion that the protester's proposal is unacceptable, we likewise find no ground upon which to conclude that the agency unreasonably excluded ADI's proposal from the competitive range. The regulations provide that the competitive range must include all proposals that have a reasonable chance of being selected for award and that any doubt as to whether a proposal is in the competitive range should be resolved by inclusion. Federal Acquisition Regulation Sec. 15.609(a). While the determination of whether a proposal is to be included within the competitive range is principally a matter within the reasonable exercise of the procuring agency's discretion, we closely scrutinize any evaluation that results in a competitive range of one; such a competitive range, however, is not per se illegal or improper. See Institute for Int'l Research, B-232103.2, Mar. 15, 1989, 89-1 CPD Para. 273; Comten-Compress, B-183379, June 30, 1975, 75-1 CPD Para. 400.

Here, ADI's proposal tied for the lowest score under the technical evaluation factor and received the lowest score under the management factor based on the agency's reasonable conclusions that ADI's proposal did not address many required areas, failed to meet the required delivery schedule and failed to demonstrate that ADI could successfully manufacture and deliver the devices in a timely manner. Although ADI contends that any deficiencies in its proposal were only "informational," we think the record reasonably supports the agency's conclusion that the proposal could not be made acceptable without major revisions and additions. In this regard, an agency's decision to exclude an offer from the competitive range is proper where the offeror's technical proposal is so deficient that it would require major revisions before it could be made acceptable. Comten-Compress, supra; Intraspace Corp., 69 Comp.Gen. 351 (1990), 90-1 CPD Para. 327.

While ADI contends that its proposal should have been included in the competitive range because of its lower price, price is not a factor when a proposal is technically unacceptable. Intraspace Corp., supra. Since ADI's proposal was technically unacceptable its lower price is not controlling here. Accordingly, we find rejection of ADI's proposal to have been proper.

Other Issues

We also reject ADI's contentions that the evaluators and contracting personnel were biased against ADI as result of the firm's employment of former Micronics' personnel and that the exclusion of ADI's proposal from the competitive range amounted to a determination of nonresponsibility which should have been referred to the SBA for review under the COC program. The agency's primary concern regarding Micronics was that ADI had proposed to use an approach similar to that used by Micronics and other vendors on previous contracts for MK-29 devices or similar devices. Although the record includes criticism of ADI for proposing to use former Micronics personnel on this contract, the major concern of the evaluators and the contracting officer in his decision to exclude ADI was that the approach which was used by these firms in the past, and proposed by ADI here, had resulted in quality problems. Under the circumstances, regardless of any concerns about the performance of Micronics or the former personnel of that firm, as we have explained in detail above, the agency reasonably rejected ADI's proposal based on concerns about the approach proposed by the firm. We, therefore, do not agree that the rejection of ADI's proposal was the result of bias on the part of the agency personnel. See MGM Land Co.; Tony Western, B-241169; B-241169.2, Jan. 17, 1991, 91-1 CPD Para. 50, recon. denied, Tony Western--Recon., B-241169.3, May 21, 1991, 91-1 CPD Para. 489.

Finally, with respect to the contention that the exclusion of ADI amounted to a determination of nonresponsibility, the deficiencies identified in ADI's proposal related to evaluation factors set forth in the solicitation under which the agency has performed a comparative evaluation of the merits of the various proposals. These included understanding of the requirements, manufacturing approach, resources and facilities and related previous successes. Where an offer is found deficient in the context of a comparative evaluation conducted under the criteria specified in an RFP, the matter is one of technical acceptability, not responsibility. Modern Sanitation Sys. Corp., B-245469, Jan. 2, 1992, 92-1 CPD Para. 9. In a negotiated procurement, traditional responsibility factors may be used as technical evaluation factors, and so long as the agency performs a proper comparative evaluation/relative assessment of the competing proposals under those factors, if a small business proposal is found to be deficient, referral to the SBA is not required. Id.

The protest is denied.

1. ADI also argues that the evaluators improperly considered information which was outside of Texas Instruments' proposal. ADI refers to information in the evaluation record that indicates that contracting personnel communicated with the Defense Contract Audit Agency (DCAA), Dallas to verify labor rates provided by Texas Instruments. This information had nothing to do with the evaluation of Texas Instruments' technical or management proposal but rather related to proposed price negotiations with that firm. There was nothing improper in this communication with DCAA.

2. ADI denies that it failed to propose to meet the required delivery schedule. However, while the solicitation required delivery of first articles to the Navy 275 days after contract award, the firm's proposal only obligated the firm to complete inspections and tests to be conducted at ADI "[a]pproximately 270 days after contract award" and then to prepare the items for shipment. In addition, although the protester now disputes the Navy's conclusion that ADI would not meet the schedule, the firm argued in its initial protest submission that no offeror could meet the delivery schedule in the solicitation.