Lockheed Martin MS2 Tactical Systems

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: Lockheed Martin MS2 Tactical Systems

File: B-400135; B-400135.2

Date: August 8, 2008

DIGEST

Agency reasonably determined, in procurement for unmanned maritime surveillance aircraft, that awardee had significant advantage over protester with respect to past performance where: protester's subcontractor, responsible for approximately 50 percent of contract effort, had recent past performance history of being unable to resolve staffing and resource issues, resulting in adverse cost and schedule performance on very relevant contracts for unmanned aircraft; record did not demonstrate that protester's subcontractor had implemented systemic improvement that resulted in improved performance; while operating division of the awardee also had performance problems on very relevant contracts for unmanned aircraft, many had been addressed through systemic improvement; and overall performance of awardee's team on most evaluated contract efforts was rated better than satisfactory, while the overall performance of protester's team on 11 of 26 contract efforts was only marginal.

Lockheed Martin MS2 Tactical Systems (LM) protests the Naval Air Systems Command's (NAVAIR) award of a contract to Northrop Grumman (NG) under request for proposals (RFP) No. N00019-07-R-0001, for the Broad Area Maritime Surveillance (BAMS) Unmanned Aircraft System(UAS). LM challenges the evaluation of proposals and resulting source selection.

We deny the protest.

The Navy generally expects that the BAMS UAS will provide a persistent maritime Intelligence, Surveillance, and Reconnaissance (ISR) data collection and dissemination capability to the fleet, with BAMS on station 24 hours a day, 365 days a year for 20 years. Hearing Transcript (Tr.) at 47. The Statement of Objectives for BAMS (SOO) included in the RFP established the objective that each BAMS orbit (unit) provide (with no more than three aircraft aloft simultaneously) continuous surveillance capability for a minimum of 24 hours per day for 7 consecutive days, with an Effective Time on Station (ETOS) of no less than 80 percent, at a minimum mission range of 2,000 nautical miles (nm). BAMS SOO sections 1.0-3.1. Likewise, the Performance Based System Specification for BAMS (PBSS) included in the RFP required that the BAMS UAS –be capable of maintaining 80 percent (Threshold) and 95 percent (Objective) ETOS executed within a period of 168 continuous hours at a mission radius of 2000 Nautical Mile (nm) from its operating base.— PBSS sect. 3.1.1.

At Initial Operational Capability (IOC), defined as one base unit with sufficient assets to operationally support one persistent ISR orbit, the BAMS UAS missions will include maritime surveillance, collection of enemy order of battle information, battle damage assessment, port surveillance, communication relay, and support of maritime interdiction, surface warfare, battlespace management, and targeting for maritime and littoral strike missions. (At full operational capability, the BAMS UAS will provide for up to five simultaneous orbits worldwide.) While the objective is to achieve IOC in fiscal year (FY) 2113 or earlier, the minimum threshold requirement is for an IOC in FY 2014. SOO sections 1.0-2.0.

The solicitation provided for award of a cost-plus-award-fee contract for system development and demonstration (SDD), with cost-plus-award fee options for low rate initial production (LRIP 1), and for furnishing the BAMS UAS to Australia. Award was to be made to the offeror whose proposal represented the best value to the government based on four evaluation factors: (1) technical (including two subfactors of equal importance: design approach and program/schedule); (2) past performance; (3) experience; and (4) cost (including four subfactors: SDD/LRIP1 option contract cost, which was significantly more important than operations and support (O&S), which was more important than production, which was significantly more important than Australian option cost). The technical factor was more important than past performance or experience, which were of equal importance, while cost was least important. All factors other than cost, combined, were significantly more important than cost.

Three offerors responded to the solicitation by the closing time on May 3, 2007. LM proposed to design and fabricate the BAMS Mariner, a modified version of its proposed subcontractor General Atomics Aeronautical Systems' (GA-ASI) Predator B aircraft (flown by the U. S. Air Force), with such modifications as a 22-foot extension of the wingspan, [REDACTED]. The BAMS Mariner, with a gross takeoff weight of 12,528 pounds (approximately 2,000 pounds greater than the existing model), was to be powered by a single engine with a turboprop propeller. Based upon an evaluated ingress speed of [REDACTED] knots and an egress speed of [REDACTED] knots, NAVAIR assessed that for a station approximately 2,000 miles from its operating base, each BAMS Mariner would be on station [REDACTED] hours, while the evaluated ETOS of a 4'aircraft orbit was 84.6 percent. NG proposed to design and fabricate the BAMS Global Hawk (RQ'4N), a modified version of its RQ-4B Global Hawk B (flown by the U.S. Air Force), with such modifications as [REDACTED]. The BAMS Global Hawk, with a gross takeoff weight of 32,250 pounds, was to be powered by a single turbofan (jet) engine. Based upon an evaluated ingress speed of [REDACTED] knots and an egress speed of [REDACTED] knots, each BAMS Global Hawk would be on station [REDACTED] hours, while the evaluated ETOS of the proposed 3'aircraft orbit was 96.2 percent. Boeing proposed to design an unmanned version of the [REDACTED]. The Boeing [REDACTED], with a gross takeoff weight of [REDACTED] pounds, was to be powered by [REDACTED]. Based upon an evaluated ingress speed of [REDACTED] knots and an egress speed of [REDACTED] knots, each Boeing [REDACTED] would be on station [REDACTED] hours, while the evaluated ETOS of the proposed [REDACTED]-aircraft orbit was 92.8 percent.

All three proposals were included in the competitive range. After conducting several rounds of written and oral discussions with the offerors, NAVAIR requested the submission of final proposal revisions (FPR). FPRs were evaluated as follows:

Source Selection Advisory Council (SSAC) Proposal Analysis Report (PAR) at 3.

Although each proposal received the same adjectival ratings under the technical factor, NG's (as well as Boeing's) proposal was evaluated as having a strong advantage over LM's. Specifically, NG's proposal was evaluated by the SSAC as having a strong advantage over LM's under the design approach subfactor of the technical factor. In this regard, the solicitation provided that evaluation under the design approach subfactor would encompass the extent to which the proposed BAMS UAS will be able to meet the mandatory as well as the offeror-specific tailored requirements of the BAMS PBSS, with emphasis on ETOS, open systems architecture, mission performance, due regard, and growth potential in the areas of unmanned aircraft (UA) Space, Weight, and Power (SWaP). NG's proposed BAMS was evaluated as having a significant strength in the emphasis area of ETOS, offering a greater persistent ISR capability with an evaluated ETOS of 96.2 percent, significantly higher than LM's evaluated ETOS of 84.6 percent. In addition, NG's proposal was evaluated as having a significant strength in the emphasis area of SWaP, including a significantly higher weight margin (ability to add weight without breaching the ETOS threshold requirements) of approximately [REDACTED] pounds, nearly [REDACTED] times greater than LM's margin of only approximately [REDACTED] pounds. Tr. at 122-49; Hearing Exhibit 102. NAVAIR noted that NG's superior SWaP future growth capability could be used to incorporate future capability increments without breaching the ETOS threshold requirements, and that the resulting significantly greater design margin would reduce the risk of unknown design challenges requiring increased SWaP, particularly weight. In contrast, LM's proposal was evaluated as having a significant weakness under the design approach subfactor for failure to provide a validated computer model of the performance of its proposed engine. The agency determined that NG's strong advantage under the design approach subfactor offset a slight advantage for LM under the program schedule subfactor, such that NG's proposal had a significant overall advantage over LM's under the technical factor. SSAC PAR at 3-7.

In addition, the past performance of LM and GA-ASI (its principal subcontractor, which was proposed to perform approximately 50 percent of the work), was determined to represent a high risk, giving rise to substantial doubt that the LM/GA'ASI team could perform the proposed contract effort. In this regard, the SSAC noted reports of poor past performance on very relevant work, including prior GA-ASI contracts for Predator-related work, with practically no systemic improvement demonstrated. According to the evaluators, the customer feedback for GA-ASI's contracts was remarkably consistent across contracts, identifying difficulties in managing workload, problems with executing systems engineering and systems integration tasks, and problems with properly staffing a project. Further, the SSAC expressed concern that the additional resources required for the BAMS UAS program would further exacerbate the identified existing staffing and management shortcomings. Finally, the SSAC noted that LM itself had experienced difficulties in furnishing adequate staffing on its contract (the Po Sheng contract) to upgrade the command control system for Taiwanese F-16 fighter aircraft. Based on this record, the SSAC expressed concern that the LM team would encounter significant schedule delays and be required to make technical trade-offs in order to produce the Mariner. PAR at 7-10.

In contrast, the NG division serving as the prime contractor/system integrator and performing approximately [REDACTED] percent of the NG contract effort (NG Integrated Systems, Eastern Region in Bethpage, New York) was evaluated as having satisfactory to very good past performance. NG Executive Summary at 1i-1. Although the performance of NG Integrated Systems, Western Region, Unmanned Systems Development Center in Rancho Bernardo, California''which (with its predecessor, Teledyne Ryan, see NG Past Performance Proposal at 3.4a-1) developed the Global Hawk and will perform approximately [REDACTED] percent of the contract effort''on very relevant Global Hawk'related work was viewed as a significant concern, with the potential to translate into risk for the BAMS effort, the SSAC noted that most past performance problems had been addressed through demonstrated systemic improvement. Consequently, the agency determined that only some doubt existed that NG would be able to successfully perform the required effort, thus warranting a moderate overall risk rating. PAR at 10; NG Executive Summary at 1i-1.

The SSAC concluded that NG had a significant advantage over LM/GA-ASI with respect to past performance. According to the SSAC, while NG had implemented systemic improvements that were successful in improving performance for nearly all problems found on relevant contracts, nearly all LM/GA-ASI problems appeared to persist without implemented systemic improvement that resulted in improved performance. Moreover, the SSAC expressed particular concern that the poor past performance of the LM team was focused on GA-ASI, which was proposed to perform approximately 50 percent of the BAMS work, leaving a critical gap in LM's capability to reliably perform the BAMS effort. The SSAC (with one dissenter) concluded that NG's proposal's strong advantage over LM's proposal under the technical factor (the most important evaluation factor), and NG's significant advantage under the past performance factor (the second most important factor), offset LM's significant advantage with respect to cost. PAR at 7-11, 13-16.

The Source Selection Authority (SSA) concurred with the SSAC's recommendation that NG's proposal was most advantageous. According to the SSA,

[LM's] past performance record identifies systemic problems at GA-ASI, which coupled with the lack of demonstrated systemic improvement, indicate that these problems will likely be repeated on the BAMS UAS program negatively impacting a significant portion of the work. Furthermore, while [NG's] advantage in Past Performance in itself justifies the [cost] premium, the technical advantage of providing a large design margin further increases [NG's] probability for success, mitigating some of the doubt associated with their Moderate Past Performance risk. . . . [NG's] design approach has significant Space, Weight and Power (SWaP) growth capability, which may be used to incorporate future increments without breaching the Effective Time on Station (ETOS) threshold requirement. This significant strength can also be used as a design margin which acts to reduce the risk of unknown design challenges requiring increased SWaP, particularly weight. Finally, it is important to note that [LM's] lower proposal evaluated costs do not include any costs associated with poor performance. Performance difficulties can result in a significant cost to the Government in terms of time (schedule slips), money (cost over'runs and internal Government manpower and resources), and technical capability for the warfighter.

Source Selection Decision (SSD) at 2-3. Upon learning of the resulting award to NG, and after being debriefed, LM filed this protest with our Office.

PAST PERFORMANCE

Background

LM principally challenges NAVAIR's evaluation of its own and NG's past performance. In this regard, the RFP provided that the government would

evaluate the offeror's, and (if applicable), its principal subcontractors' and critical team members' demonstrated past performance in delivering quality products and in meeting technical, cost and schedule requirements on similar programs for SDD, Production, and Operations and Support. The currency and relevance of the information, source of the information, context of the data, and general trends in contractor's performance will be considered. Problems not addressed by the offeror will be considered to still exist. However, consideration for discounting problems may be given when those problems are addressed through demonstrated systemic improvement.

RFP sect. M.II.B.

In furtherance of the past performance evaluation, offerors were required to identify contracts –whose performance is within five years from the RFP release and contain efforts similar to those efforts, e.g., tasks, contract type, location, contract dollar value, etc., required by this solicitation.— RFP sect. L.3.1. The information provided –should be related to similar programs in the same division, or cost centers in which the Offeror proposes to perform this effort,— and correspond to the descriptions of the offeror's experience under the experience factor. Id.The experience section of the RFP, in turn, required that the experience –be relative to proposed roles and responsibilities of the Offeror/Subcontractor in this solicitation,— and identified several tasks considered relevant, including –[p]erforming SDD tasks such as design, integration, fabrication, and test of a system similar in scope to the BAMS UAS,— –[p]erforming logistics tasks for major military weapons systems similar in scope to the BAMS UAS,— and –[p]roduction and manufacturing of a system similar in scope to the BAMS UAS.— RFP sect. L.4.0.

For each relevant contract, offerors were required to describe performance in meeting technical and quality requirements, meeting schedule requirements, controlling cost, and managing the contracted effort (e.g., program management, cooperation with customer, subcontract management). RFP sect. L.3.4.2. In addition, and of particular importance here, offerors were required, –[f]or each past performance problem identified, [to] describe the status of the systemic improvement efforts and, where applicable, demonstrate the impact that the systemic improvement effort had on resolving the problem such that it would not reoccur.— RFP sect. L.3.1. Further, in addition to –[i]dentify[ing] those systemic improvement actions taken to resolve past problems,— offerors were required to –[p]rovide the records of such results and indicate where they are documented, preferably in Government record systems. Describe the techniques, elements, and tools used to correct problems on the contract and, if applicable, how these techniques, elements, and tools will be used during this program.— RFP sect. L.3.4.3. Finally, offerors were cautioned that –[t]he Government does not assume the duty to search for data to cure the problems it finds in the information provided by the Offeror. The burden of providing thorough and complete past performance information remains with the Offeror.— RFP sect. L.3.1.

The record indicates that, in evaluating past performance, NAVAIR undertook an in'depth, detailed examination of the recent relevant past performance of each team. In this regard, the agency reviewed 30 contracts for the LM team, 19 of which (including 6 LM and 7 GA-ASI contracts) were determined to be either somewhat relevant or very relevant; received 23 past performance questionnaires for the team; conducted 7 interviews; and reviewed 34 CPARS. The agency reviewed 27 contracts for the NG team, 24 of which (including 17 NG contracts) were determined to be either somewhat relevant or very relevant; received 40 past performance questionnaires for the team; conducted 13 interviews; and reviewed 45 CPARS. SSEB at 40. Based on the totality of the information available from the above information sources, NAVAIR determined that the performance of the LM team was exceptional on 1 contract effort, very good on 6, satisfactory on 8, and marginal on 11. (Some contracts were divided into separate delivery/task orders for purposes of these ratings When considered on a contract-by-contract basis, without division into separate task order(s) contract efforts, the LM team's performance was exceptional on one contract, very good on six, satisfactory on seven, and marginal on five. However, of particular significance to evaluation, of the five very relevant GA-ASI contracts for Predator-based aircraft, GA'ASI's performance was marginal on three contracts.) In contrast, NAVAIR determined that the overall performance of the NG team was exceptional on 3 contract efforts, very good on 10, satisfactory on 9, and marginal on only 3 contract efforts. Source Selection Evaluation Board Report (SSEB) at 31-40.

LM asserts that NAVAIR's evaluation of LM's, GA-ASI's, and NG's past performance was inconsistent with the solicitation and otherwise unreasonable in numerous respects.

Where a solicitation requires the evaluation of offerors' past performance, we will examine an agency's evaluation only to ensure that it was reasonable and consistent with the solicitation's evaluation criteria and procurement statutes and regulations; determining the relative merits of offerors' past performance information is primarily a matter within the contracting agency's discretion. The MIL Corp., B'297508, B'297508.2, Jan. 26, 2006, 2006 CPD para. 34 at 10; Hanley Indus., Inc., B'295318, Feb. 2, 2005, 2005 CPD para. 20 at 4. A protester's mere disagreement with the agency's judgment is not sufficient to establish that the agency acted unreasonably. Birdwell Bros. Painting & Refinishing, B-285035, July 5, 2000, 2000 CPD para. 129 at 5. Here, we find the overall past performance evaluation to be reasonable. We discuss LM's most important challenges to the past performance evaluation below.

Past Performance of LM Team

Although the SSEB noted that inadequate staffing and a shortfall in technical skills had adversely impacted LM's ability to execute a somewhat relevant (the Po Sheng) SDD contract to upgrade the command control system for Taiwanese F-16 fighter aircraft, the panel generally acknowledged that LM had demonstrated –high quality technical performance— on five of six relevant contracts. SSEB at 52. In contrast, however, GA-ASI's contract performance was a matter of great concern to the agency. Specifically, while recognizing that GA-ASI had demonstrated a willingness and ability to respond on short notice to evolving Global War on Terror (GWOT) warfighter requirements, the SSEB found that GA-ASI's performance demonstrated: inadequate staffing, resulting in performance problems on SDD contracts for the MQ'9 Reaper (a second-generation, Predator B model) and the MQ'1C Extended Range/Multipurpose (ER/MP) UAS (a second-generation Predator model); unfavorable schedule performance on four of seven relevant GA-ASI contracts, including very relevant contracts for the MQ-9 Reaper, UAS ground control stations, MQ-1C ER/MP, I-GNAT Extended Range UAS (a version of the Predator with some differences for the Army), and MQ-1 baseline Predator; poor performance in meeting technical quality requirements on three of seven GA-ASI contracts, including contracts for the MQ-9 Reaper, MQ-1C ER/MP, and I'GNAT Extended Range UAS; and workload exceeded the firm's capacity on five of seven GA-ASI contracts, including contracts for the MQ-9 Reaper, UAS ground control stations, MQ-1C ER/MP, I-GNAT Extended Range UAS, and MQ'1/MQ-9 maintenance support. In summary, the SSEB found the overall performance of GA-ASI on its very relevant contracts for the MQ-9 Reaper (most delivery orders), UAS ground control stations, MQ-1C ER/MP, and I-GNAT Extended Range UAS to be marginal. SSEB at 36-38, 52'62.

Based upon the above past performance problems, the SSEB determined that there was substantial doubt that LM would successfully perform the required effort, and that an overall high risk rating therefore was warranted. According to the SSEB:

The Lockheed Martin team delivers a high quality, technical product and both Lockheed Martin and GA-ASI are motivated to meet the warfighter's requirements. Lockheed Martin, as the prime contractor however, will be substantially challenged to ensure that GA-ASI will remain on schedule. The proposal includes a prominent role for GA'ASI including aircraft design, UA manufacture, flight test, logistics, training support, communications subsystem and MCS aircraft control segment which represents approximately 50% of the proposed effort. There is, therefore, significant risk to the BAMS UAS program if GA'ASI's future performance trend reflects identified past performance difficulties in managing increasing workload, a possibility which the [Past Performance Evaluation Team] assesses as likely to occur.

Lockheed Martin and GA-ASI have recent past performance histories of being unable to resolve staffing issues resulting in adverse cost and schedule performance. Furthermore, there are documented concerns regarding the amount of work that GA'ASI has taken on and the slow pace of implementing processes and process improvements that increased workloads and responsibilities require. Systemic improvement initiatives have been identified or are in work in several areas of concern; however, these efforts are not yet demonstrated to determine their effectiveness at lowering risk.

SSEB at 62.

LM disputes both the agency's evaluation of its performance under several of the individual contracts and the determination that there was little demonstrated systemic improvement.

(1) MQ-9 Reaper/GCS

GA-ASI's ongoing Air Force contract No. F33657-02-G-4035 included very relevant (according to both LM and NAVAIR) delivery orders for interim contract capability, SDD and production of the MQ-9 Reaper (again, a second-generation Predator B model), and for the pre-production and production of UAS ground control stations (GCS). NAVAIR received three Contractor Performance Assessment Reports (CPAR) for these contract efforts, the most recent completed on April 23, 2007 for the period from January 1 to December 31, 2006, with earlier CPARS for the periods October 1, 2004 to December 31, 2005 and September 18, 2002 to September 30, 2004. (NAVAIR had unsuccessfully requested updated 2007 performance information on contract No. 4035. NAVAIR E-mail to Air Force Point of Contact as suggested by Air Force Program Manager (as listed in LM Past Performance Proposal at 3-6), Nov. 28, 2007; Tr. at 2183-84.)

LM challenges the overall marginal rating for GA-ASI's performance under this contract on the basis that this overall rating was inconsistent with the category ratings in the latest 2006 Air Force CPAR of very good for technical, satisfactory for management, and marginal for schedule and cost control. However, while recognizing that GA-ASI –does an excellent job responding to quick reaction and rapidly evolving warfighter requirements in support of the Global War on Terror,— the 2006 CPAR nevertheless expressed significant reservations as to GA-ASI's performance in several areas:

Systems engineering was rated satisfactory overall but remains an area of concern for the program. The company has not been able to develop a sufficient systems engineering staff to keep pace with the numerous other contracted efforts.

While satisfactory overall, [software engineering] is an area of concern for the program. Although the company continues to increase its software engineering staff, there continues to be limited software engineering resources to complete all contracted work. . . . The contractor needs to continue to increase their engineering staff in order to meet contracted commitments in parallel.

Several projects under this [Basic Ordering Agreement] have suffered from schedule delays, to include the MQ-9 ICC and MQ-9 SDD efforts. The schedule variances for these efforts are -20% and -44.9% respectively as of Dec. 06. While [GA-ASI] has committed to expanding the workforce, the contractor has insufficient resources to execute the contracted work on schedule in several key areas. The resulting schedule delays directly impact the fielding of combat capability.

CPAR, Contract No. 4035, 2006 Period. The 2004/2005 CPAR for contract No. 4035 included similar criticisms of GA-ASI's performance, as well as marginal schedule and cost control ratings. Given the above continuing staffing and resources shortfalls, which resulted in –schedule delays directly impact[ing] the fielding of combat capability,— CPAR, Contract No. 4035, 2006 Period, and the repeated marginal schedule and cost control ratings in the most recent CPARs, we find no basis to question NAVAIR's evaluation of GA-ASI's overall performance under contract No. 4035.

LM further challenges NAVAIR's assessment that systemic improvement by GA-ASI on contract No. 4035 (as well as under other contracts) had not been demonstrated. In this regard, LM generally acknowledged in its December 4, 2007 discussions response with respect to a number of GA-ASI contracts (including the MQ-9 Reaper, MQ-1C ER/MP, and I-GNAT Extended Range UAS), that –the fundamental cause for GA-ASI Past Performance issues was availability of trained staff to meet the demand for our products and services—; that GA-ASI's workload had exceeded its capacity; that there had been –management task saturation—; and that there was a –valid CPAR comment— regarding (overly) –[c]entralized management structure.— LM Evaluation Notice (EN) Response, Dec. 4, 2007, LM-PP-003, -008, -009, -010, 011. However, LM maintained then, id., and asserts in its protest, that GA-ASI has undertaken such systemic improvements as increasing engineering and trained staff, hiring mid-level and senior program managers, and restructuring the decision-making process. According to the protester, the evaluation failed to acknowledge these systemic improvements.

The agency's evaluation in this area was reasonable. Although LM has suggested that the RFP did not require that there be documented results of any systemic improvements, and that merely hiring additional staff should be accepted as effective systemic improvement, as noted above, the RFP in fact required the offeror to –identify those systemic improvement actions taken to resolve past problems, . . . demonstrate the extent to which it will benefit the instant contract,— and –[p]rovide the records of such results and indicate where they are documented, preferably in Government record systems.— RFP L.3.4.3. Accordingly, in ascertaining whether there had been systemic improvement in correcting prior performance deficiencies and problems, the agency properly looked to see whether the record –demonstrate[d] the impact of the systemic improvement,— including whether there were any results of the claimed systemic improvement measures –in a record or documentation to show that action resulted in a tangible improvement to that program,— such that there was –independent verification [of] tangible improvement.— Tr. at 778-80.

Given the solicitation requirement that any improvements in contract performance be documented, the agency reasonably determined that overall systemic improvement by GA-ASI on contract No. 4035 had not been shown. In this regard, as noted above, notwithstanding the agency's November 2007 request to the Air Force for updated contract performance information, an updated CPAR or other updated past information had not been furnished by the Air Force. Further, while LM furnished its own updated Earned Value Management System (EVMS) data on contract No. 4035 in a December 6, 2007 discussions response, that data did not clearly establish that overall demonstrated systemic improvement on the contract had occurred. LM reported that the cumulative Schedule Performance Index (SPI) (ratio of work performed to the initial planned schedule, with an SPI of less than 1.0 indicating that work is not being completed as planned and the program may be behind schedule if the incomplete work is on the critical path) on the three ground control system delivery orders as of October 2007 was only 76.6 percent, 88.8 percent, and 91.9 percent, all below the 95'percent level at which performance began to be a matter of some concern to the agency. LM also reported that the cumulative Cost Performance Index (CPI) (ratio of work performed to actual costs for work performed, with a CPI of less than 1.0 being unfavorable because the work is being performed less efficiently than planned) on one of the orders likewise was below the 95-percent level (93.1 percent). LM Response to EN LM-PP-015, Dec. 6, 2007; Tr. at 1084-92; GAO Cost Assessment Guide, GAO-07-1134SP, at 226.[1] As for the four MQ-9 Reaper delivery orders, LM reported that one had been completed in December 2006 at a cumulative CPI of 92.4 percent, one of the remaining three orders was below the 95'percent CPI level in October 2007 (at 91.3 percent), and the third order had been rebaselined in October 2007 (and the index thus was reset to 1.0). LM also reported that one of the orders was below the 95'percent level for SPI in October 2007 (at 83.7 percent), while a second had been rebaselined in October 2007 after having an SPI of 55.2 percent in June 2007. We conclude that the agency reasonably determined that there was no documentation of systemic improvement on contract No. 4035.

(2) MQ-1C ER/MP

Both NAVAIR and LM considered ongoing Army contract No. W58RGZ-05-C-0069, for the MQ-1C Extended Range/Multipurpose (ER/MP) UAS (a second-generation Predator model using the basic structure of the Predator aircraft with the Predator B avionics suite), to be very relevant to LM's proposed BAMS Predator-based Mariner UAS. LM Past Performance Proposal at 3-9, 3-51. NAVAIR received for this contract: four past performance questionnaire (PPQ) responses, including December 10, 2007 and April 2007 responses from the Army Procuring Contracting Officer (PCO), an April 2007 response from the Army Product Manager, and a February 26, 2007 response from the Defense Contract Management Agency (DCMA) Administrative Contracting Officer (ACO); and a number of LM discussion responses that referred to the contract (as well as a number of other contracts).

LM challenges the overall marginal rating for GA-ASI's performance under this contract, primarily on the basis that this rating was inconsistent with the input from the DCMA ACO and LM's discussion responses.

We find that the agency reasonably rated GA-ASI's performance under contract No. 0069 only marginal. In this regard, the most recent detailed information received by NAVAIR for this contract was the Army PCO's December 10, 2007 PPQ response in which he rated GA-ASI's performance as marginal for technical/quality performance, schedule, cost performance, and program management. According to the Army PCO, while the agency was –confident the company can and will deliver a quality aircraft system,— nevertheless, –as the program continues, and [GA'ASI] takes on additional contracts, we are concerned about [GA-ASI's] ability to successfully manage and deliver products to all customers on time and within cost.— Army PCO PPQ Response, Contract No. 0069, Dec. 10, 2007. The Army PCO specifically reported the following performance problems on the MQ-1C ER/MP contract:

[GA-ASI has not met contracted . . . delivery schedules.

[GA-ASI] continues to struggle as the Systems Integrator.

[GA-ASI] has resisted hiring adequate engineering and technical staff to address all of the tasks they are currently contracted to perform.

The common theme within the delivery/schedule problems appears to relate back to the acceptance of contractual commitments which are physically beyond production capacity.

A major contributor is [that GA-ASI's] senior management continues to obligate the company without fully reviewing and understanding the current workload and commitments.

Management task saturation coupled with [GA-ASI's] highly centralized management structure both contribute towards the delays with the integration testing and coordination efforts . . . .

The engineering staff appears to be technically [competent], but in most cases are not empowered at the appropriate levels to make the necessary decisions to push the task forward in a timely manner to maintain schedule.

[GA-ASI] has made limited corrective actions and usually not without Government PMO insistence.

Army PCO PPQ Response, Contract No. 0069, Dec. 10, 2007. Furthermore, the April 2007 PPQs completed for Contract No. 0069 by the Army PCO and the Army Product Manager appeared to indicate that GA-ASI's performance problems had been continuing for some time, with references to GA-ASI –continu[ing] to struggle in identifying and executing system engineering and system integration tasks required to facilitate final integration of the subsystems,— and having –struggled in the area of staffing at adequate levels to properly resource the program schedule.— Army PCO PPQ Response, Contract No. 0069, Apr. 2007; Army Product Manager PPQ Response Contract No. 0069, Apr. 2007.

LM asserts that the overall marginal rating for GA-ASI's performance on contract No. 0069, for the MQ-1C ER/MP, did not reasonably account for the February 26, 2007 PPQ response completed by the DCMA which reported that GA-ASI's technical/quality and schedule performance was exceptional, its cost performance was very good, and its management performance was very good to exceptional.

We find LM's position unpersuasive. As an initial matter, we agree with the agency that the DCMA ACO furnished little detail in support of his very favorable performance ratings, and that the detail that was furnished appears in some measure inconsistent with the high ratings. In this regard, for example, while the DCMA ACO rated GA-ASI's cost performance as very good and its schedule performance as exceptional, the DCMA ACO reported cumulative, unfavorable EVMS ratings of 0.84 for CPI and 0.91 for SPI as of January 2007. Although the DCMA ACO stated that government-directed changes were the cause of schedule and cost issues, he also acknowledged that $18 million of a predicted $37 million cost overrun was believed to be the result of –cost growth within the contract scope,— as distinct from –scope growth,— and he referred to the fact that –[c]orrective actions are on-going,— including continued hiring by GA-ASI, thus seemingly implying that there was some contractor responsibility for performance problems. Tr. at 1784-93. In any case, the DCMA ACO's response was furnished in February 2007, while the more detailed responses by the Army PCO and Army Product Manager represented more recent assessments based on the contract performance as of April and December 2007. We note in this regard that the PPQs completed in April 2007 by the Army PCO and Army Product Manager both rated GA-ASI's performance as marginal to satisfactory for technical/quality and schedule performance, and satisfactory for cost and management performance, thus suggesting both that GA-ASI's performance had deteriorated since the DCMA ACO's assessment in February and continued to deteriorate up to the time of the marginal performance reported by the Army PCO in December 2007. Army PCO PPQ Response, Contract No. 0069, Apr. 2007; Army Product Manager PPQ Response, Contract No. 0069, Apr. 2007. Moreover, to the extent that the differences in assessment of GA-ASI's performance represented a difference of opinion, as distinct from a mere change over time in the quality of performance, we consider it significant that it was the views of the Army PCO and Army Product Manager, rather than those of the DCMA ACO, that were broadly consistent with the reports in the Air Force CPARs on GA-ASI's performance on the MQ-9 Reaper/GCS delivery orders, that is, the reports of continuing GA-ASI staffing and resources shortfalls resulting in schedule delays. In summary, based on the recent, detailed information concerning GA-ASI's performance problems on contract No. 0069, which information was consistent with reports of staffing and resource shortfalls resulting in schedule delays on other contracts, NAVAIR reasonably rated GA-ASI's performance on this contract as no better than marginal.

LM asserts that, in any case, NAVAIR failed to account for systemic improvement by GA-ASI, such as increased staffing. As noted above, however, the RFP required a showing of documented improvements in contract performance as a result of any claimed systemic improvement measures. RFP sect. L.3.4.3. While GA-ASI has apparently continued to increase its workforce, again, an increase in workforce by itself, without a documented improvement in contract performance, did not meet the solicitation standard for showing systemic improvement. Here, not only did the PPQ responses seem to suggest deteriorating performance on contract No. 0069 through December 2007, but, in addition, recent EVMS data furnished by LM during discussions was not favorable. In this regard, in its December 6, 2007 discussion response to EN LM'PP-015, LM indicated that the MQ-1C contract had been rebaselined after performance resulted in increasingly unfavorable EVMS numbers at the beginning of 2007''with a decline in the CPI from 83.8 percent in January to 80.3 percent in April and a similar SPI decline from 90.9 percent to 87.1 percent''but then, after the rebaselining was reflected in the EVMS data in September 2007 with fresh 100-percent CPI and SPI ratings, the numbers again began to decline, falling to 98.4 percent and 98.2 percent, respectively, in October 2007. LM Response to EN LM'PP-015, Dec. 6, 2007. In these circumstances, we find that NAVAIR reasonably concluded that the information known to the agency did not support a finding of systemic improvement on contract No. 0069.

(3) I-GNAT Extended Range UAS

Both NAVAIR and LM considered Army contract No. DAAH01-03-C-0124, ending in December 2007, for the I-GNAT Extended Range (ER) UAS (an Army version of the Predator), to be very relevant to LM's proposed BAMS Predator-based Mariner UAS. LM Past Performance Proposal at 3-9, 3-70. NAVAIR received three PPQ responses for this contract: a May 31, 2007 response from the Army Deputy Product Director, Unmanned Aerial Systems Program Management Office; an April 18, 2007 response from the Army PCO (who was also the Program Manager (PM) for this program (according to LM, LM Past Performance Proposal at 3-7), and the PCO for the MQ-1C ER/MP contract); and an April 2007 –coordinated— response from the DCMA ACO (who also was the ACO for the MQ-1C ER/MP contract) and the DCMA Program Integrator, which was subsequently updated by the DCMA ACO on May 24, 2007. In addition, NAVAIR received several LM discussion responses that referred to the contract (among a number of other contracts).

LM challenges the overall marginal rating for GA-ASI's performance under this contract, primarily on the basis that it fails to account for the DCMA input. In this regard, the record reflects what appears to be an irreconcilable difference between the Army and DCMA evaluations of GA-ASI's performance. On the one hand, the Army Deputy Product Manager and the Army PCO/PM agreed on marginal ratings for technical/quality, schedule, cost and management performance based on concerns that GA'ASI had –consistently failed to meet contractual delivery dates for the spares and Ground Data Terminals and [was] beginning to show moderate slippage on delivery dates for Air Vehicles and Ground Control Stations—; had demonstrated resistance to hiring adequate personnel; had overly centralized management structure that contributed to program delays; had difficulty in managing its subcontractors; and ultimately was –agreeing to contractual commitments which are beyond its production capacity.— PPQ Response, Contract No. 0124, May 31, 2007, Army Deputy Product Director; PPQ Response, Contract No. 0124, Apr. 18, 2007, Army PCO/PM. On the other hand, the coordinated response from the DCMA ACO and DCMA Program Integrator offered the summary conclusion that there had been exceptional technical/quality, schedule, cost and management performance, with –generally— on-time performance or, as of May 2007, –on schedule— aircraft deliveries and –[n]o major slippage on the production schedule.— PPQ Response, Contract No. 0124, Apr. 2007, DCMA ACO; E-mail from NAVAIR to DCMA ACO, May 24, 2008.

We find that NAVAIR's evaluation of GA-ASI's performance on contract No. 0124 was reasonable. Confronted with materially differing ratings from the Army and DCMA representatives as of May 2007, the agency unsuccessfully sought updates from both entities on November 28, 2007, E-mail to Army Deputy Product Manager, Army PCO/PM, and DCMA ACO, Nov. 28, 2007, and also raised the negative past performance information from the Army with LM in a series of ENs in October 2007. Of particular significance in this latter regard were LM's October 12 responses to EN LM-PP-009 and EN LM-PP-11, in which LM did not specifically refute the reports that GA-ASI had failed to meet a number of contractual delivery dates, but essentially maintained that it was simply –being responsive to the customer's aggressive requests— and that any performance difficulties were beyond its control. LM Response to EN LM-PP-009, Oct. 12, 2007. Given that the record before NAVAIR included negative performance appraisals from responsible officials of the Army, that is, the customer agency; the Army evaluations were consistent with those on several other Army and Air Force Predator-related contracts; the Army evaluations were more detailed than the general praise from the DCMA ACO; the DCMA ACO's use of such language as –generally— on-time or –[n]o major slippage— suggests that there were some schedule slippages, which would appear to be inconsistent with the DCMA ACO's exceptional ratings; and the fact that, when questioned by NAVAIR in this regard, LM did not refute the reports that GA-ASI had failed to meet a number of the contractual delivery dates, we find that NAVAIR reasonably rated GA-ASI's performance on this contract as marginal.[2]

In summary, we find that LM's challenges to the evaluation of its team's past performance provide no basis for questioning the agency's determination that the LM team''in particular, GA-ASI''had a recent past performance history of being unable to resolve staffing and resource issues, which resulted in adverse cost and schedule performance. We further find no basis for questioning the agency's determination that, notwithstanding such systemic improvement measures as hiring additional staff, LM did not establish documented improvements in contract performance as a result of the systemic improvement measures; these efforts therefore did not furnish a basis for reducing the risk associated with the LM team's unfavorable past performance.

Past Performance of NG

As noted above, NG's performance was rated as marginal on only 3 of 20 contract efforts, very good on 10 and exceptional on 3. The SSEB specifically found that the NG division serving as the prime contractor/system integrator (NG Integrated Systems, Eastern Region) had satisfactory to very good past performance, demonstrating –high quality technical performance— on each of seven relevant contracts and favorable or excellent cost control on six of those contracts. SSEB at 63. Furthermore, the SSEB found that the overall NG team demonstrated excellent program management on most relevant contracts.

However, the performance of NG Integrated Systems, Western Region, UnmannedSystemsDevelopmentCenter, on very relevant Global Hawk'related work was viewed as a significant concern. In this regard, the SSEB determined that the NG team had demonstrated: (1) unfavorable cost performance on 5 of 19 contract efforts, including Western Region Air Force contract No. F33657-01-C-4600, for Engineering and Manufacturing Development of the Global Hawk (Global Hawk EMD), and Western Region Air Force contract No. F33657-03-C-4310, for Low Rate Initial Production of the Global Hawk (Global Hawk LRIP); (2) underestimation of software development and integration resulting in schedule delays and cost overruns on three SDD contracts, including the Global Hawk EMD contract; (3) unfavorable schedule performance on 5 of 15 contract efforts, including both the Global Hawk EMD and Global Hawk LRIP contracts; and (4) marginal ability to manage subcontractors on three contracts, including the Global Hawk EMD contract. As a result of the above performance problems, the overall performance of NG's Western Region on the Global Hawk EMD contract was rated as marginal. (NG's performance was rated as satisfactory on Global Hawk LRIP contract No. 4310, as well as on three other Global Hawk LRIP contracts.) SSEB at 63-77.

Although NAVAIR recognized the potential for the NG Western Region performance problems on the Global Hawk EMD contract to translate into risk for NG's Global Hawk-based BAMS effort, the agency identified other considerations that mitigated this risk: NG's overall generally very good-to-exceptional technical/quality ratings; its flexible and responsive management as indicated in most performance reports; the favorable performance of NG Eastern Region, the proposed prime contractor/system integrator, in controlling cost; and demonstrated systemic improvement with respect to most prior performance problems. Regarding the evaluated demonstrated systemic improvement, the SSEB determined that: of the five contracts on which there had been unfavorable cost performance, systemic improvement had been demonstrated on one, and some systemic improvement had been demonstrated on three (including the Global Hawk EMD); of the three contracts on which there had been underestimation of software development and integration resulting in schedule delays and cost overruns, systemic improvement had been demonstrated on two; of the five contracts on which there had been unfavorable schedule performance, systemic improvement had been demonstrated on two (including the Global Hawk EMD), and some systemic improvement had been demonstrated on two (including the Global Hawk LRIP); and of the three contracts on which there had been marginal management of subcontractors, systemic improvement had been demonstrated on one and some systemic improvement had been demonstrated on another. Based on this evidence, the SSEB determined that only some doubt existed that NG would be able to successfully perform the BAMS effort, thus warranting a moderate overall risk rating (in contrast to the substantial doubt regarding successful performance and resulting high risk rating for LM's proposed contract effort). Id.

LM asserts that NG's moderate risk rating did not adequately account for NG's performance problems, particularly on the Global Hawk EMD contract, and that the agency unreasonably credited NG with systemic improvements on a number of contracts, particularly the Global Hawk EMD and Global Hawk LRIP contracts. Based upon our review of the record, we find no basis for questioning the evaluation of NG's past performance.

As indicated above, and as is otherwise amply demonstrated in the record, NAVAIR fully recognized the problems encountered by NG Western Region under the Global Hawk EMD contract, including those with respect to unfavorable cost performance, underestimation of software development and integration resulting in schedule delays and cost overruns, unfavorable schedule performance, and marginal ability to manage subcontractors, all resulting in an overall performance rating of marginal. SSEB at 67'76; Contract Performance Analysis Worksheet, Dec. 19, 2007. Furthermore, notwithstanding LM's claims to the contrary, this recognition included an understanding throughout the evaluation, and by the SSA, that there had been significant cost increases under the contract (15 percent or more above the baseline cost) such that notification to Congress was required in 2005. See 10 U.S.C. sect. 2433 (2008); Letter from Secretary of the Air Force to Congress, Apr. 13, 2005; Letter from Secretary of the Air Force to Congress, Dec. 29, 2005 (reporting cost breach –directly related— to breach previously reported in April 2005); Contract Performance Analysis Worksheet, Dec. 19, 2007, Cost; SSEB at 66-68; SSAC PAR at 10; Tr. at 1322-39, 1889'90.

LM asserts that NAVAIR unreasonably credited NG with systemic improvement with respect to most reported past performance problems. Again, however, the record supports the agency's conclusion. For example, the SSEB noted with respect to the prior unfavorable cost performance under the Global Hawk EMD contract, that NG had implemented a number of cost control measures, including a [REDACTED], designed to improve program oversight and management and maintain cost and schedule within two percent of the baseline. In this regard, the Air Force program director advised NAVAIR on November 16, 2007 that

the program has put in place a new Acquisition Program Baseline and is executing to it. . . . The contractor and government have made management changes and have put in place strong mechanisms to control cost and schedule and to predict growth in either. Overall, the program has made significant changes and I believe that it is now healthy and on a path to success.

E-mail From Program Director to NAVAIR, Nov. 16, 2007. In addition, NG responded to NAVAIR ENs by furnishing cumulative EVMS data for 2007 (after the prior cost breach and rebaseline), showing a CPI of 1.01 in January 2007, decreasing to .99 in July and increasing to 1.00 in October 2007. NG Response to EN NG-PP-008, Nov. 30, 2007. However, since the Air Force program director had reported in a May 23, 2007 CPAR that NG continued to have cost and schedule control challenges in several areas, and given the interim decline in the CPI to .99 in July 2007, the SSEB determined that NG had demonstrated only some systemic improvement. Regarding the prior unfavorable schedule performance, the SSEB noted NG's implementation of such measures as the [REDACTED] designed to improve program oversight and management. In addition, the SSEB noted that, while the Air Force program director had reported in May 2007 that NG still faced schedule control problems in several areas, the program director had also confirmed that improvements had been made in overall scheduling approach and process. Since the EVMS data furnished by NG in November 2007 had indicated a cumulative SPI ranging between .99 in January 2007 and .98 in November 2007, the SSEB determined that systemic improvement had been demonstrated in the schedule area.

In contrast, with respect to the prior underestimation of software development and integration resulting in schedule delays and cost overruns, the SSEB noted that as recently as September 7, 2007, the Air Force had expressed concern to NG as to a 20'month delay in the Block 10 software, which originally was scheduled for completion in January 2006, but instead was rescheduled for operational release in late September 2007. The SSEB further noted that, while NG had implemented a number of systemic improvement measures in this area, the contractor had conceded that many of the initiatives were still in the early stages of development. Based on this information, the SSEB determined that systemic improvement had not been demonstrated in this area. The SSEB reached a similar conclusion with respect to the previous determination regarding NG's marginal ability to manage subcontractors. SSEB at 66-76; Declaration of Past Performance/Experience Team Lead, July 8, 2008.

As is apparent from the above discussion, the SSEB undertook a detailed, reasoned approach to determining whether NG had demonstrated systemic improvement. LM has not shown the agency's methodology or conclusions to be unreasonable. While LM suggests that there was no cost data to demonstrate systemic improvement, and further asserts that the agency failed to consider the impact of the software problems on the overall schedule, we think that, given the favorable EVMS data for 2007''including a CPI of not less than .99 and an overall SPI of not less than .98 between January and October 2007''and the various systemic improvement measures undertaken by NG (including its [REDACTED], the agency reasonably determined that some systemic improvement had been demonstrated in the cost area and that systemic improvement had been demonstrated in the overall schedule.

We conclude that the agency reasonably determined that only some doubt existed that NG would be able to successfully perform the required effort, thus warranting a moderate overall risk rating.

TECHNICAL

As noted above, NG's proposal was evaluated as having a strong advantage over LM's under the design approach subfactor of the technical factor based in part on the evaluation of NG's proposed BAMS UAS as having significant strengths in the emphasis areas of: ETOS, with NG offering a greater persistent intelligence, surveillance, and reconnaissance capability with its proposed UAS orbit evaluated as having an ETOS of 96.2 percent, significantly higher than LM's evaluated ETOS of 84.6 percent; and SWaP, including NG's weight margin of [REDACTED] pounds, nearly [REDACTED] times greater than LM's margin of approximately [REDACTED] pounds. NAVAIR noted that NG's superior SWaP future growth capability could be used to incorporate future capability increments without breaching the ETOS threshold requirements, and that the resulting significantly greater design margin would reduce the risk of unknown design challenges that could require increased SWaP, particularly weight. NAVAIR concluded that NG's strong advantage under the design approach subfactor offset a slight advantage in favor of LM's proposal under the program schedule subfactor such that NG's proposal had a significant overall advantage over LM's under the technical factor. SSAC PAR at 3-7.

LM raises several challenges to the evaluation under the technical factor. We find that none furnishes a basis for questioning the reasonableness of the agency's overall determination of NG's advantage under the factor. For example, LM asserts that the ETOS calculation for its proposed Predator-based solution understated the time on station because it was improperly based on an orbit of four aircraft, ignoring the fact that LM had updated its original approach to add a fifth aircraft in its December 4 proposal revisions. However, the record supports the agency's interpretation of LM's revised proposal as continuing to propose a 4'aircraft orbit for purposes of the ETOS calculation. Neither the origin of the revisions nor the resulting revised proposal supports LM's interpretation of a 5'aircraft orbit for purposes of the ETOS calculation. In this regard, LM advised NAVAIR in its December 4 proposal revision that –[a]s a result of Government comments and after rerunning the Government's model received as part of EN LMGA-C-0040, we have added an additional aircraft to our LRIP 1 cost and schedule. We will now produce 5 aircraft as part of LRIP 1.— LM/GA Proposal Update, Response to the 28 November BAMS Face-to-Face Discussions, at 2-3. However, EN LMGA-C-0040 was a cost EN, not a technical EN, and the relevant section of the EN advised LM that the agency had concluded that its proposal of 33 aircraft would not be sufficient to sustain full operating capability for the required 20 years. Further, LM's submissions from before and after the December 4 revisions all referred to an orbit or input of four aircraft, including: LM's Technical Proposal Figure 2.1.4-6 –ETOS Calculator Inputs,— responsive to the RFP requirement that offerors –[i]dentify all assumptions used in developing ETOS predictions— and complete an –ETOS Input Table,— RFP L.2.1.4; the narrative in its Technical Proposal Section 2.1.4, –Effective Time on Station—; and the proposal's executive summary. LM Executive Summary at 3'4, 10, 22; LM Executive Summary Errata, Dec. 4, 2007; LM Technical Proposal sect. 2.1.4. Moreover, as noted by the agency, LM was informed during face-to-face discussions on November 28, 2007, that the agency was assuming that only 80 percent of the unmanned aircraft at a particular site would be able to fly for purposes of the ETOS calculation-''–Number of UAs per site = ETOS results/80% availability.— BAMS UAS Competition, LM Preliminary Discussions Evaluation Findings, Nov. 28, 2007, at 54. Since application of the 80-percent availability factor would require five aircraft in order that four would be available to fly for purposes of the ETOS calculation, the agency reasonably found that LM's proposal of a fifth aircraft for LRIP was not intended to change the proposal of four aircraft for the ETOS calculation. Tr. at 46-49, 62-65.[3] Thus, nothing in the circumstances of the December 4 revisions reasonably alerted the agency to an intention on LM's part to add a fifth aircraft for ETOS calculation purposes.

The protest is denied.

Gary L. Kepplinger
General Counsel