Matter of: Alliant Techsystems, Inc. File: B-276162; B-276162.2; B-276162.3 Date: May 2, 1997 * Redacted Decision

Matter of: Alliant Techsystems, Inc. File: B-276162; B-276162.2; B-276162.3 Date: May 2, 1997 * Redacted Decision

DIGEST

Attorneys

DECISION

Alliant Techsystems, Inc. protests the Department of the Air Force's downselection and award of the Wind Corrected Munitions Dispenser (WCMD) Pilot Production Option under contract No. F08626-95-C-0106, P00010 to Lockheed Martin Corporation. Alliant contends that the agency's technical and cost evaluations were flawed and that the source selection authority's best value decision failed to take certain risk and cost factors into account.

We deny the protest.

BACKGROUND

The WCMD is a modification kit that replaces the tail sections on a portion of current inventory cluster munitions. The kit will enable the tail dispensers which carry the cluster munitions to correct for the effects of launch transients, ballistic errors, and unknown winds between the release point and the dispenser's functioning point. Alliant and Lockheed were each awarded 24-month contracts in January 1995, to perform engineering and manufacturing development (EMD) of the WCMD on a cost- plus-fixed fee basis. In accordance with the terms of these contracts, the Air Force issued a call for improvements/request for proposals (CFI/RFP) to each offeror to submit updated proposals for the 20-month pilot production phase and optional low-rate initial production lots (LRIPs 1 and 2), and prices for three full rate production lots (FRP). The solicitation contemplated award of only the pilot production option to one contractor, but reserved the right to exercise the options with both EMD contractors, or to make no option award at all. Exercise of the LRIP 1 option was to be made not later than 14 months after award of the pilot production option and LRIP 2 not later than 14 months after the exercise of LRIP 1. Timing of the FRP awards was not included in the CFI/RFP.

Clause H.16 of both EMD contracts sets forth four evaluation areas in descending order of importance: affordability, technical, management, and instant contract cost. General considerations also were to be evaluated, but were of lesser importance than the specified criteria. Clause H.16 also identified evaluation factors under three of the four criteria. Under affordability, proposals were to be evaluated under four factors, in descending order of importance: average unit procurement price requirement (AUPPR); [1] manufacturing maturity; production risk management; and warranty. Under the technical area there were three factors of equal importance: system performance; system design; and system integration. Under management there were two factors of equal importance: system engineering management and cost/schedule control. With the exception of the AUPPR factor, all factors were to be evaluated in three ways: color/adjectival rating; proposal risk rating; and performance risk rating.

The color rating was to depict how well the proposal met the evaluation standards and solicitation requirements. The color and adjective ratings used were blue (exceptional), green (acceptable), yellow (marginal), and red (unacceptable). "Proposal risk" was defined as an assessment of the risk associated with the offerors' proposed approach as it related to accomplishing the requirements of this solicitation. "Performance risk" was defined as an assessment of the probability of the offeror's successfully accomplishing the proposed effort based on demonstrated present and past performance. Performance and proposal risk ratings included high, moderate, and low.

According to clause H.13, the AUPPR is a specified cost in base year (BY) 1994 dollars that has been proposed by the contractor. The AUPPR is calculated by taking the total applicable procurement cost of the kits and dividing it by 40,000, which represents the total number of units to be produced under the program (LRIPs and FRPs combined). It represents a composite of recurring and non-recurring costs that include all fully burdened contractor procurement costs incurred in the manufacture of a usable end item, the equipment and materials required to support it, and initial spares. The clause provides that the proposed AUPPR is a system requirement which the contractor is required to meet or better during the life of the program. Failure to meet the AUPPR during performance results in the agency's right to impose several possible penalties including competition of production lots and requiring provision of a reprocurement data package at no cost to the government.

Each offeror's cost and fee information submitted in support of their estimated AUPPRs was evaluated for realism, reasonableness, and completeness. The AUPPR also received an affordability risk rating which was given equal consideration with the evaluated cost of the AUPPR. Similarly, the "instant" contract cost (for pilot production and the LRIP options) received a performance risk rating to be given equal consideration with the evaluated cost of the instant contract.

Offerors also were required to propose a production price commitment agreement (PPCA) to be executed by the awardee which obligates it to the proposed AUPPR and places the risk of unreasonable deviation from that estimate on the contractor. The PPCA is in effect for both LRIPs 1 and 2 and FRPs 1-3. A similar clause will be negotiated for FRPs 4-6. Under the PPCA, for FRPs 1-3, if the contractor submits a proposal that exceeds the agreed upon price, the contractor will have the opportunity to address the cause of the excessive pricing prior to the agency's invoking its rights. If the agency concludes that it is dissatisfied with the contractor's pricing, it has the right to levy similar penalties to those addressed in clause H.13.

Both Lockheed and Alliant submitted proposals by the November 1, 1996, closing date. The agency conducted a "fly-off" of the offerors' production representative tail kits from November 4 through December 6. Both offerors made oral presentations in November. The agency conducted discussions with them in December and completed the technical and affordability evaluations in January 1997. As part of its January affordability submittal, Lockheed proposed a technology insert to replace the inertial measurement unit (IMU) and related components. The replacement IMU would use micro-machine technology resulting in a significantly smaller component set. The insert was to be included in the [deleted] of FRP [deleted] and in all subsequent FRPs. The insert would not change the performance of the WCMD, but would result in a significant unit price reduction which was reflected in Lockheed's AUPPR.

The evaluators reviewed this aspect of Lockheed's proposal from a technical and cost standpoint. The proposed IMU is in a developmental stage and current versions do not meet the specifications of a weapons- grade component. Based on the evaluators' knowledge of this developing technology, including ongoing Air Force research and development in the same area, they concluded that the insert would be ready at the time proposed by Lockheed or shortly thereafter. Because this technology insert would constitute a "level one change," which could not be implemented until after qualification testing and the Air Force's determination that the insert met all contract requirements, Lockheed would be required, and represented that it would continue, to produce the approved (fly-off qualified) version until the insert was approved. As a result, the evaluators concluded that the only potential impact of the insert on the program was with regard to AUPPR cost risk and took this into account by adjusting Lockheed's AUPPR affordability risk rating. The final evaluation results are as follows:

Factors Lockheed Martin Alliant Techsystems Color/Prop Risk/Perf Risk Color/Prop Risk/Perf Risk

Affordability

A.1 AUPPR $8937 Moderate-Low [deleted] Moderate-Low

A.2 MFG Maturity Blue/Low/Low Blue/Low/Low

A.3 Production Risk Blue/Low/Low Blue/Low/Low

A.4 Warranty Blue/Low/Low Blue/Low Low

Technical

T.1 Sys Performance Blue/Low/N/A Blue/Low/N/A

T.2 Sys Design Blue/Low/Low Blue/Moderate/Low

T.3 Sys Integrity Green/Low/Low Green/Low/Low

Management

M.1 Sys Eng'g Mgt Green/Low/Low Green/Low/Low

M.2 Cost/Schedule Green/Low/Low Green/Low/Moderate

Instant Contract $41,995,685 [deleted] Cost

The evaluators briefed the source selection authority (SSA) on the evaluation results. After obtaining clarification of the affordability impact of Lockheed's technology insert, the SSA determined that Lockheed's proposal represented the best value to the government. In making this determination, the SSA considered the risks to both the Air Force and Lockheed if the technology insert were delayed. For example, if Lockheed could not successfully introduce the insert on time, Lockheed would suffer significant losses since the approved IMU it would have to continue supplying is more expensive than the proposed IMU. If Lockheed attempted to pass on increased costs in the later FRPs, its AUPPR would increase, thereby triggering the Air Force's right to penalize Lockheed. Thus, while there was a potential cost risk to the Air Force, the SSA was satisfied that the risks to Lockheed provided significant incentives to timely obtain approval of and to introduce its proposed insert. After receiving notice of the award and a debriefing, Alliant filed this protest. The agency has stayed performance pending this decision.

Alliant challenges the technical and affordability/cost evaluations of Lockheed's proposal, especially with respect to Lockheed's proposed technology insert. [2] Evaluating the relative merits of competing proposals is a matter within the discretion of the contracting agency since the agency is responsible for defining its needs and the best method of accommodating them, and it must bear the burden resulting from a defective evaluation. Advanced Tech. and Research Corp., B-257451.2, Dec. 9, 1994, 94-2 CPD Para. 230 at 3. Consequently, in reviewing an evaluation we will not reevaluate proposals but instead will examine the agency's evaluation to ensure that it was reasonable and consistent with the stated evaluation factors. Id. The fact that the protester disagrees with the agency's judgment does not render the evaluation unreasonable. As discussed below, we have examined the agency's evaluation here and conclude that it was both reasonable and consistent with the stated evaluation criteria.

EVALUATION OF THE TECHNOLOGY INSERT

Alliant contends that the Air Force failed to conduct any meaningful evaluation of the technology insert. Alliant observes that Lockheed's proposed IMU and related components are not yet developed to the point of meeting the WCMD specifications. Since the bulk of the 40,000 units to be produced over the life of the entire program are proposed to include the new, as yet undeveloped IMU, Alliant asserts that the agency should have evaluated the technology insert under a number of factors and rated the proposal's adjectival and risk ratings far less favorably. The Air Force takes the position that its evaluation properly accounted for the insert's impact under the affordability and technical factors. [3]

The record reflects that the agency conducted a review of the insert that was both thorough and appropriate for the proposal. [4] Here, Lockheed proposed a fully functional design for its WCMD which performed well in the fly-off evaluation. It required no major changes before the transition to pilot production and LRIP. In accordance with the CFI/RFP instructions, at the time it submitted its cost proposal and proposed AUPPR, Lockheed provided a detailed description of the new IMU and related components it proposed as a technology insert in FRP [deleted]. Lockheed also described the steps it was taking to ensure timely development of the IMU, including the planned investment of approximately [deleted] million for IMU development by Lockheed, [deleted] (its primary IMU subcontractor) and [deleted] (its backup IMU subcontractor). The proposal made clear that the new IMU would not affect performance of Lockheed's WCMD, but would result in a significant lowering of its AUPPR.

The agency's information on the merits of the technology insert was not limited to Lockheed's January 1997 affordability proposal. From the beginning of the WCMD program, the IMU was identified as the prime candidate for future cost reduction through technology advancement because the IMU represented the largest percentage of system costs. IMU technology is the subject of numerous governmental studies and technology contracts as well as industry funded efforts. In view of the large numbers of IMUs required for precision guided weapons, it is generally accepted in the industry that the future of tactical grade IMUs is in micro-machine technology. Accordingly, the evaluators attended IMU symposia and interviewed the leaders in the advanced IMU industry.

In October 1996, Lockheed and [deleted] briefed the Air Force evaluators, including the technical chief, affordability chief, and the A.1 factor captain on a planned micro-machined IMU technology insertion, addressing the investment planned, the state of development, and the development and production schedule. In December, Lockheed with [deleted] and [deleted] subcontractor briefed the technical and affordability chiefs on [deleted] approach to the same issues. The previous September, all the affordability evaluators and the technical area chief had conducted site visits to [deleted] to observe the production of IMUs and reviewed their production planning. Lockheed's proposal provided detailed schedules and milestones that confirmed the earlier briefings. In addition, Alliant's own proposal [deleted]. [5]

Lockheed also submitted a time-phased summary of several low cost IMU development initiatives being pursued in parallel by a number of companies including [deleted]. In addition, after reviewing the proposals, the evaluators consulted with the Air Force's Wright Laboratory. Wright had contracted with Hughes to develop a micro-machined IMU and expects it to be ready for transition to production in 1999, [deleted] Lockheed proposed to furnish the insert. Wright reviewed Lockheed's proposal and concluded that there was a low risk that the contractor would not meet its proposed schedule.

Since the proposed IMU is in the developmental stage, the evaluators could not conduct a detailed evaluation of the item itself. However, the evaluation did not simply rely on Lockheed's promise. The evaluators did extensive research into the area and assembled a significant base of information from which to make their determination. When considered together this information provides a rational basis for their conclusion that the IMU would be developed to meet all requirements. The only unknown was when this would happen, which translated to schedule risk. Under the evaluation scheme, such schedule risk was properly viewed as affecting cost. That is, since Lockheed's lower AUPPR was based on timely insertion of the less expensive IMU, any slippage in the schedule could result in increased costs to Lockheed which it might attempt to pass on to the agency through higher prices for FRPs 4-6. Accordingly, the evaluators rated the risk of insertion on schedule as moderate to moderately high.

Alliant agrees that it was appropriate to evaluate the insert under A.1, but argues that since the insert represents a major change in the proposed system, the Air Force should have evaluated it for its impact on manufacturing maturity and production risk management under the affordability area (factors A.2 and A.3), and for its impact on system performance, design, and integration under the technical area (factors T.1, T.2, and T.3).

According to clause H.16, factor A.2 was designed to assess the contractor's progress in concurrently developing an affordable design and the manufacturing processes required to build the initial units during development and future production units. Included as part of the evaluation was an assessment of future manufacturing technology insertion. Factor A.3 was designed to assess the contractor's progress in complying with planning to ensure the production phase of the program is entered into with low risk. Factor T.1 focused on how well the contractor executed the design, development, build and test of the WCMD under development and was to assess the technical approach the contractor was pursuing to complete development. Factor T.2 was to evaluate the degree to which the contractor had demonstrated that its system design would meet the performance requirements. The maturity of the fly-off hardware and software was to be compared to the proposed final production configuration. Factor T.3 was to assess the degree to which the contractor had demonstrated the ability to accomplish the technical tasks remaining under the pilot production and LRIP.

After evaluating the technology insert, the evaluators concluded that none of the above-listed factors was appropriate for evaluating the potential impact of the insert. For example, impacts to the pilot production and LRIP options would be nonexistent since Lockheed did not plan the insert until the [deleted] of FRP [deleted]. Thus, the evaluators' assessments about Lockheed's existing production design remained valid and unchanged. In addition, Lockheed had proposed, and the contract structure mandated, that Lockheed continue to produce its approved WCMD until the Air Force approved use of the new IMU in production. Further, apart from the insertion, the remainder of Lockheed's WCMD system design would remain the same. Since the Air Force would continue to receive a WCMD which met or exceeded all requirements, until such time as the Air Force alone determined that the new IMU and related components met or exceeded all requirements, any potential negative impact or risk could be eliminated by the agency's preventing introduction of the new IMU.

Therefore, the evaluators determined that there would be no impact on manufacturing maturity, production risk management, or system performance, design and integration. [6] Instead, they concluded that the only appropriate factor under which to assess the impact of the insert was affordability factor A.1 concerning schedule and cost risk. Based upon all the information before them indicating that the IMU would be fully developed within the next several years [deleted], we believe the evaluators' conclusion was reasonable.

THE AFFORDABILITY EVALUATION

In the "affordability" area, specifically factor A.1, the evaluators were concerned with determining the realism, reasonableness, and completeness of program-life costs as expressed in the AUPPR. [7] As described above, the AUPPR represented the unit cost, in BY 1994 dollars, of each of the 40,000 units to be produced throughout the WCMD program, that is, the end of FRP 6. At the time of contract award, prices for units to be produced under FRPs 4-6, representing 28,581 or more than 50 percent of the total units, were unknown because they were to be negotiated at the time the winning contractor submitted a proposal for FRP 3. Since the proposed AUPPR basis included non-binding estimates for FRPs 4-6, there was the potential for an upward change in the AUPPR. However, because meeting or bettering the proposed AUPPR was a system requirement, the AUPPR combined with stated penalties for not meeting it had the effect of acting as a cap on future costs.

Alliant contends that the Air Force's affordability evaluation was flawed because the agency failed to properly evaluate the realism, reasonableness, and completeness of Lockheed's proposed AUPPR. In Alliant's view, the agency merely accepted the proposed AUPPR without any meaningful evaluation. Had the Air Force properly evaluated the proposed AUPPR it would have found it unrealistic and deserving of a less favorable risk rating. Alliant's arguments demonstrate a basic misunderstanding of what was required in this evaluation.

The affordability evaluation conducted by the agency here was different from those done with respect to a pure cost reimbursement contract. Typically, in a cost reimbursement procurement, the proposed costs are not binding on the offeror; the agency is required to pay a contractor's actual and allowable costs. Prospect Assocs. Ltd., B-249047, Oct. 20, 1992, 92-2 CPD Para. 258 at 7. Thus, an agency must determine the likely cost it will be required to pay and often will adjust upward the proposed cost to reflect the anticipated cost. While in the EMD procurement the solicitation evaluation scheme provided for calculation of a most probable AUPPR, the agency specifically modified Alliant's and Lockheed's contracts to eliminate this aspect of the evaluation under the CFI/RFP at issue here so that the contractors were not required to submit cost information at the same level of detail as required in a cost reimbursement contract. Rather, the cost information to be provided was to include contractor assumptions, ground rules, methodology, and supporting data, identification of included and excluded elements of cost, identification of recurring and non-recurring costs, and identification of costs using the work breakdown structure (WBS) identified in the CFI/RFP. The agency was required to accept the AUPPR proposed by each contractor, evaluating it only to determine whether that figure was realistic, reasonable, and complete.

With regard to realism, the evaluators were to assess the compatibility of the proposed cost with the scope of the estimates and the schedule durations. To this end, they determined whether the materials and efforts were consistent with the directed program, whether the costs were consistent with the schedule proposed, and whether the estimates were based on the quantities provided by the CFI/RFP. They were to assess reasonableness based upon the acceptability of the methodology and logic used in developing the cost estimates. In this regard, they evaluated the cost estimates for organization and logical development and determined whether the contractor's ground rules and assumptions were reasonable; the contractor used appropriate historical data, inflation rates, and estimating methodology; and the contractor explained any unusual approach. With regard to completeness, they were to assess the responsiveness of the offeror in ensuring that all major milestones and activities were estimated and that the documentation provided traceability of costs. Here, the evaluators ensured that all pertinent costs were included, were shown in appropriate categories, were identified per WBS task and level, and were documented in sufficient detail to allow cost estimate replication. Most of Alliant's criticisms of these evaluations concern the failure to compare Lockheed's costs with its proposed prices.

Since the agency is not bound to reimburse the contractors' actual costs, these criticisms are without merit. The agency's evaluation followed the stated criteria and resulted in a reasonable conclusion that the information supporting both contractors' AUPPRs was realistic, reasonable, and complete. From our review of the agency's evaluation, we find nothing objectionable. [8]

Alliant specifically contends that the evaluators' assessments are suspect because they do not specifically account for the "precipitous" drop in Lockheed's price between FRP 3 and FRP 4. In this regard, Alliant notes that even though Lockheed proposed to make the technology insert in FRP [deleted], its costs and prices are not reduced until FRP 4. Alliant also argues that Lockheed's costs appear to be greater than can be accounted for in the pricing estimates. From these and other observations, Alliant concludes that Lockheed has "gamed" its estimates and the evaluators failed to recognize it.

The agency explains that it was well aware of the drop in price and when it occurred. The agency found the drop to be attributable to several factors including Lockheed's [deleted] pricing strategy in FRPs 1-3. [deleted]. The agency also found that Lockheed anticipated more savings between FRPs 3 and 4 based on redesign of components related to the IMU technology insert. Further, in conducting these evaluations, the evaluators recognized that the contractors would not necessarily recoup all estimated costs since the PPCA and AUPPR served to restrict what the agency would be required to pay. We believe these are reasonable conclusions. In any event, since this is not a cost-reimbursement contract, the fact that Lockheed's costs may exceed its proposed prices does not necessarily make the AUPPR suspect as there is nothing inherently objectionable about a below-cost offer. Oshkosh Truck Corp., B-252708.2, Aug. 24, 1993, 93-2 CPD Para. 115 at 6.

With regard to affordability risk, the evaluators assessed the contractors' ability to achieve the proposed AUPPR based on an evaluation of three risk drivers: product (40 percent), process (30 percent), and cost methodology (30 percent). The WCMD was divided into seven WBS areas, each with a predetermined weight (e.g., IMU 30-percent, fin mechanism 10 percent). Each WBS area was evaluated under the product, process, and methodology drivers based on uncertainty factors ranging from low (1 point) to high (10 points). Each WBS area received three scores: optimistic, pessimistic, and most probable, under each of the three drivers. These scores were multiplied by their weights and combined to calculate a single set of optimistic, pessimistic, and most probable scores under each of the three risk drivers. Using the risk driver weights, the three sets were then combined to calculate a single optimistic, pessimistic, and most probable score for each contractor. These were then plotted on a triangular shaped graph, on which a 90 percent confidence index was plotted to derive a single risk score. [9] The evaluators calculated a risk score of [deleted] for Alliant and [deleted] for Lockheed. Both scores equate to a "moderately low" risk rating.

Alliant argues that Lockheed's proposal should have received a less favorable affordability risk rating given the risk involved in the technology insert. For example, Alliant contends that Lockheed's product risk rating should be high because it involves "significant design changes proposed, retesting and validation required." Similar criteria are present under the other risk drivers of process and cost methodology.

Contrary to Alliant's contentions, the risk evaluation did account for the higher risk attributable to the technology insert. However, the evaluators concluded that the insert was better assessed as a "major modification to fly-off design, significant testing required" and rated it as moderately high for the most probable and pessimistic scores. While the IMU is an important component, it represents only 30 percent of the entire WCMD proposed by Lockheed. Accordingly, when the risk evaluation had calculated the appropriate scores based on WBS weights, the higher risk of the technology insert was mitigated by the lower risk associated with the balance of the proposed WCMD. Alliant's disagreement with the evaluators' judgment does not render the evaluation unreasonable. Medland Controls, Inc., B-255204; B-255204.3, Feb. 17, 1994, 94-1 CPD Para. 260 at 5.

THE AWARD DETERMINATION

In a negotiated procurement, the government is not required to make award to the lowest-cost, technically acceptable offeror unless the solicitation specifies that cost or price will be a determinative factor for award. General Servs. Eng'g, Inc., B-245458, Jan. 9, 1992, 92-1 CPD Para. 44 at 9. The CFI/RFP, while making cost an important consideration, advised the contractors that award would not necessarily be made to the lowest offeror. Agency officials have broad discretion in determining the manner and extent to which they will make use of technical and cost evaluation results. Cost/technical tradeoffs may be made; the extent to which one may be sacrificed for the other is governed by the test of rationality and consistency with the established evaluation factors. Id.

Here, the SSA reviewed the evaluations and noted that both proposals were more than adequate when measured against the evaluation criteria. In this regard, he noted their generally comparable proposal ratings in all areas. However, he specifically noted that Lockheed's AUPPR was [deleted] percent ([deleted]/unit) lower than Alliant's AUPPR and represented a significant reduction. While both performed well in the fly-off, he noted that Lockheed's fly-off configuration hardware and software were evaluated as very mature with few changes needed for production and that Alliant's tail kit would require [deleted] which could pose a schedule problem. In the management area, he noted that both were rated acceptable, but that Lockheed's performance in cost and schedule control during the EMD portion was better than Alliant's. While Alliant's proposed instant contract cost was [deleted] percent lower than Lockheed's, the SSA found that Lockheed's advantage in AUPPR, more mature system design, and better track record in cost and schedule control offset this advantage.

Alliant contends that the SSA's award determination was flawed because he did not properly consider that Lockheed was not committed to the prices in FRPs 4-6; that Lockheed's price, even absent schedule slips or cost growth, was significantly higher than Alliant's price until sometime in FRP [deleted]; and that there were significant risks attributable to the technology insert. Based on our review of the entire record, including the SSA's source selection statement and supplemental declaration, we find the SSA's determination was reasonable and reflected an appropriate consideration of all these matters.

While the SSA's selection statement and supplemental declaration do not specifically mention the lack of price commitment or the higher initial cost of an award to Lockheed, it is plain from the record that the SSA was aware of and considered these matters before making his award determination. For example, the SSA was aware of the provisions of the Lockheed and Alliant contracts and the terms of the CFI/RFP, none of which required a unit price commitment for FRP's 4-6. These prices were to be negotiated at the time the chosen contractor submitted prices for FRP 3. The "commitment" to pricing in these three contracts would be supplied by the system requirement that the contractor meet or better the proposed AUPPR which would necessarily be based in part on these prices. With regard to the higher initial cost of Lockheed's proposal, the SSA's briefing slides, which were based on the contractors' proposals, clearly show that the SSA was apprised that selection of Lockheed committed the Air Force to spend approximately [deleted] more by the end of FRP 3 than it would spend if award were made to Alliant. Implicit in the slides is the fact that by the end of FRP 6, if Lockheed met its AUPPR, the Air Force would save more than [deleted] by awarding to Lockheed. While not specifically addressed in the selection statement, it appears that the SSA considered these matters since his award determination was based on an integrated assessment of the contractors' proposals.

While the difference in cost at the end of FRP 3 is substantial, the evaluation was not designed or intended to consider interim cost differentials. Rather, the cost evaluation considered the instant contract cost, where the difference between the proposals was [deleted], and the AUPPR which encompassed the entire production program. Where, as here, a cost evaluation encompasses both base and option work, the fact that effective cost savings are not available until near the end of the option period does not invalidate the agency's conclusion that an offeror has proposed the lowest overall evaluated costs. Halifax Technical Servs., Inc., B-246236.6 et al., Jan. 24, 1994, 94-1 CPD Para. 30 at 10-11.

With regard to the risk associated with the technology insert, the SSA's supplemental declaration provides additional insight to his reasoning. He states that he was familiar with the development of IMU technology, including development efforts by contractors in other programs with which he dealt. He further states that he was fully briefed on the insert including the fact that it was only in the developmental stage, could not presently meet WCMD specification, and would not improve performance of the WCMD. He also considered the [deleted] investment Lockheed intended to make in development of the insert, but recognized that Lockheed was not contractually bound to invest any money. He specifically challenged the evaluators' determination to evaluate the insert under affordability factor A.1, but not under the other affordability and technical factors. Based upon their rationale concerning the Air Force's right to require the fly-off design until the insert was approved, he agreed that there was no threat to manufacturing maturity or other technical aspects and agreed that the risk of the insert was properly assessed as pertaining to AUPPR cost. In this regard, he had the evaluators calculate potential changes in Lockheed's AUPPR based on a 1- or 2-year schedule slip, and if the insert were never approved. These calculations show that a 1-year slip could increase Lockheed's AUPPR, but would still result in cost savings to the Air Force.

Notwithstanding the potential risks to the government, the SSA was convinced that Lockheed represented the best value. This conviction was based in part on the significant risk he saw to Lockheed if it failed to make the insert on schedule in order to meet its AUPPR. In this regard, he noted that, as in other Air Force programs, failure to meet the AUPPR could result in the levy of substantial contractual penalties against Lockheed. He also noted that failure on Lockheed's part could result in poor past performance ratings which would affect future contract opportunities with the government. The SSA also was aware that Lockheed's profit depended on introduction of the insert. The SSA reasoned that these provided significant incentives to Lockheed to timely complete development and qualification of the insert in order to meet its AUPPR.

In view of the risks to Lockheed and the Air Force's ability to control the introduction of the insert and schedule of the FRPs, we believe the SSA's reasoning was sound. The contract awarded to Lockheed is only for pilot production and LRIPs 1 and 2. Both LRIPs are options which do not have to be exercised and the Air Force has no contractual commitment to award any of the FRP contracts. Thus, it can stop the program, or postpone full production at the end of pilot production, or either of the LRIPs. If it were to stop at the end of LRIP 2, the agency would have invested only [deleted] more by awarding to Lockheed, only [deleted] percent of the cost associated with Alliant's proposal.

Clause H.24 of Lockheed's contract sets forth the PPCA unit prices for FRPs 1-3. If the Air Force proceeds, it will obtain a proposal from Lockheed for each FRP it decides to award. If Lockheed proposes prices which exceed the PPCA, without an acceptable explanation, the agency may take any or all of the following steps: compete the production lots; give the remaining production to the competing EMD contractor (Alliant); require Lockheed to deliver a complete technical data package suitable for reprocurement, at no extra cost to the government; and require Lockheed to develop and qualify a second source contractor for production of the WCMD at no additional cost to the government, paying up to $5 million in liquidated damages if it fails to timely provide a qualified second source. Lockheed plans to make the technology insert, if approved, during the [deleted] of FRP [deleted]. If Lockheed is unsuccessful or the schedule appears to be slipping, the agency will have sufficient time to take action to protect itself before spending significantly more money. For example, Lockheed's contract incorporates by reference FAR Secs. 52.249-2 and 52.249-6 which allow termination for convenience of both fixed-price and cost aspects of the contract. If Lockheed's schedule for the insert should slip, it would still be bound to produce its approved design at the prices proposed for FRPs 1-3.

Any failure to meet the delivery schedule could result in the imposition of default remedies including termination. [10] Should any slip be significant enough to affect FRPs 4-6, the agency will have sufficient time to avoid a significant investment. At the time Lockheed submits a proposal for FRP 3, it must submit a proposal for FRPs 4-6. If these submitted prices are too high, the agency will not be required to award a contract for FRP 3 or FRPs 4-6. [11]

While Lockheed is not bound to propose particular prices for FRPs 4-6, the AUPPR requirement effectively commits it to propose appropriately low prices or face other penalties from the Air Force. Clause H.13 provides that failure to meet or better the proposed AUPPR would result in several possibilities: competition of production lots, giving total/remaining production to another contractor, requiring the contractor to provide a tailored reprocurement data package at no cost to the government. Thus, if Lockheed's insert schedule slipped, it would be faced with either proposing all remaining units at prices consistent with the AUPPR and absorbing the losses or attempting to convince the agency to allow Lockheed to exceed the AUPPR. Since Lockheed's original AUPPR was based on the technology insert, notwithstanding various schedule risks, any attempt to increase the AUPPR due to schedule slippage could reasonably be rejected by the Air Force. Accordingly, these penalties provide significant incentives to meet the schedule or absorb any losses in order to avoid them.

Moreover, the fact that an offeror may have a "loophole" allowing it to exceed a cost cap does not render an agency's evaluation unreasonable, where, as here, the agency has retained adequate controls over the "loophole" to prevent increased costs. Vitro Corp., B-247734.3, Sept. 24, 1992, 92-2 CPD Para. 202 at 8-9. Under these circumstances we conclude that the SSA's determination was reasonably based on a thorough consideration of the appropriate risks attendant to a Lockheed award.

The protest is denied.

Comptroller General of the United States

1. As part of this evaluation area, the solicitation also called for an evaluation of the average field installation unit price. Since the protest does not specifically challenge the evaluation of this figure, references to this evaluation will only concern the AUPPR.

2. The protester submitted a number of arguments in support of these and other protest grounds; the agency responded to each argument, justifying its actions. We have reviewed the entire record, considered all of the arguments, and find no basis for sustaining the protest. This decision will address only the more significant arguments.

3. The Air Force argues in part that it was not required to evaluate the insert under factors A.2, A.3, T.1, T.2, and T.3 because these aspects of the evaluation concerned the instant solicitation only. Since the insert was not to be made until after completion of the pilot production and LRIP options, the evaluation should properly only reflect the system design proposed for those options. While the Air Force is correct that some factors concern only pilot production and the LRIPs, we need not resolve this dispute because, from our review of the record, it is clear that the agency evaluated all technical and affordability concerns with the insert, but reasonably determined that the only impact was to the AUPPR affordability risk factor.

4. Alliant argues that the lack of contemporaneous documentation of the agency's evaluation calls into question whether the agency adequately evaluated the insert. An agency's evaluation must be sufficiently documented to allow review of the merits of a protest. KMS Fusion, Inc., B-242529, May 8, 1991, 91-1 CPD Para. 447 at 7. In the absence of documentation, an agency runs the risk that the inadequate supporting rationale in the record for the agency's selection decision does not provide a reasonable basis for the decision. Engineering and Computation, Inc., B-261658, Oct. 16, 1995, 95-2 CPD Para. 176 at 3. However, in making this determination, we do not limit our review to contemporaneous evidence, but consider all the information provided, including the parties' arguments and explanations. KMS Fusion, Inc., supra. While Alliant is correct that we generally accord greater weight to contemporaneous materials, this does not mean that we will ignore post hoc materials simply because they were not produced at the time of the evaluation. This is especially so, where, as here, the post hoc materials do not conflict with contemporaneous documents.

5. The agency considered this proposal, but noted that Alliant had [deleted].

6. Alliant speculates that Lockheed will be unable to maintain a dual production plan while awaiting qualification of its new IMU. [deleted]. The Air Force's acceptance of this plan represents an affirmative determination of Lockheed's responsibility. Alliant's speculation provides no basis for us to disturb this determination. See King-Fisher Co., B-236687.2, Feb. 12, 1990, 90-1 CPD Para. 177 at 2.

7. In addition to the affordability evaluation, the agency also conducted an evaluation of the "instant contract cost" for the pilot production (cost reimbursement) and LRIP (fixed price) aspects of the procurement. These costs and prices were evaluated to determine if they were realistic for the work to be performed, reflected a clear understanding of the requirements, and were consistent with the elements of the proposal. Alliant has not challenged this aspect of the cost evaluation.

8. To accomplish their assessments of realism, reasonableness, and completeness, the evaluators used a checklist as a guide. Alliant argues that the agency's checklist did not reflect a thorough evaluation. However, the agency explains that the checklist was simply a guide for areas to be reviewed, and was not intended to reflect the full extent of the evaluation.

9. Alliant has challenged this methodology, arguing first that the agency should have used the "convolution method." However, after the agency pointed out that it had briefed both contractors on the triangular method to be used, Alliant argued that the final score was improperly calculated because the confidence index was biased toward the pessimistic score regardless of the most probable score. While the agency used a 90-percent confidence index to arrive at the final risk score, Alliant argues that a 50-percent index would be more accurate because it is closer to the most probable point. At the most probable point, Alliant's score is [deleted], while Lockheed's is [deleted], a difference of more than [deleted]. In Alliant's view, this should have resulted in its receiving a "low" risk score, while Lockheed would have received the less favorable "moderately low." We have had the agency's and protester's scores reviewed by a statistician within our Office. Under his analysis, Alliant is correct that the Air Force's scores were not accurate due to a methodological error in plotting the 90-percent confidence index. However, when correctly plotted, both offerors continue to have comparable scores. At the 50-percent point, Alliant's score should be [deleted] and Lockheed's should be [deleted]. At the 90-percent point, Alliant's score should be [deleted] and Lockheed's should be [deleted].

Since the Air Force's scoring methodology rounds scores to the nearest whole number, all four scores would be rounded to a three or four. Each number represents a "moderately low" risk score since scores between three and five were defined as "moderately low." Since there would be no change in the contractors' risk ratings, there was no prejudice to Alliant from the Air Force's plotting error.

10. Alliant notes that clause H.24 limits the agency's default remedy. This is only partially true. If the default were based on Lockheed's failure to meet its PPCA prices, the above stated remedies would be used in lieu of normal default remedies. However, clause H.24 provides that for other defaults, Federal Acquisition Regulation (FAR) Sec. 52.249-8 is fully effective.

11. Alliant argues that clause H.24 would require the agency to award the FRP contract so long as Lockheed proposed unit prices consistent with the PPCA. While the clause does say that each production lot proposal which is equal to or less than the respective price in the PPCA schedule "will be awarded to the contractor," Alliant ignores the balance of the statement "provided the government's price analysis determines the proposed price to be fair and reasonable." If the agency found the FRP prices unreasonable or unfair, it would not be required to award a contract to Lockheed.

DOCUMENT FOR PUBLIC RELEASE

A protected decision was issued on the date below and was subject to a GAO Protective Order. This version has been redacted or approved by the parties involved for public release.