General Atomics Aeronautical Systems, Inc.

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Decision

Matter of: General Atomics Aeronautical Systems, Inc.

File: B-311004; B-311004.2

Date: March 28, 2008

DIGEST

Protest against exclusion of protester's proposal from competitive range (leaving only one proposal in the range) in procurement for airborne radar is denied where agency reasonably determined that (1) protester's proposal failed to satisfy solicitation requirement to substantiate compliance with specifications and/or took exception to required levels of performance, and (2) some of the deficiencies resulted from fundamental limitations on performance of protester's proposed radar system, or otherwise would require a major rewrite to correct.

General Atomics Aeronautical Systems, Inc. (GA) protests the exclusion of its proposal from the competitive range under request for proposals (RFP) No. W15P7T'07-R-P406, issued by the U.S. Army Communication-Electronics Command (CECOM), for production of a Synthetic Aperture Radar (SAR)/Ground Moving Target Indicator (GMTI) system. GA primarily asserts that the determination to exclude its proposal from the competitive range failed to reasonably account for the performance of its proposed SAR/GMTI system in the flight test conducted as part of the evaluation.

We deny the protest.

BACKGROUND

The SAR/GMTI system is a lightweight, high performance, all-weather, multi-function radar system intended for use both on the Extended Range/Multi-Purpose Unmanned Aerial System (UAS) and the Future Combat Systems Class IV Unmanned Air Vehicle (UAV). The SAR/GMTI system will provide a battlefield commander with increased situational awareness by imaging stationary targets with the Synthetic Aperture Radar (SAR) and detecting (ascertaining location, direction and velocity of) moving targets with the Ground Moving Target Indicator (GMTI), even in adverse weather and through battlefield obscurants (eg., smoke or dust).

A prior contract for system development and demonstration (SDD) of the SAR/GMTI system, covering the design, fabrication, integration and test of six articles (later increased to eight funded articles), was awarded to GA on November 20, 2004. Although the original SDD schedule provided for delivery of the test units 18 months after award, there has been a significant delay in delivery. During two independent operational tests of the SDD conducted by the Army Technical and Evaluation Command, the GMTI mode of GA's system was found noncompliant with the test specifications. While a new schedule was established under which eight systems were to be accepted by March 2008, as of that time, only three systems had been accepted, and these only through a waiver of some specifications. Contracting Officer's Statement at 2; Hearing Transcript (Tr.) at I/177-I/190.[1]

The solicitation here, issued on August 10, 2007, contemplated award of a contract for an initial production quantity of 10 to 15 systems compliant with the SAR/GMTI Performance Based Specification (PBS), dated August 2, 2007, with four 1-year production options for up to 35 systems each, and two payload performance enhancement options. The solicitation divided the performance requirements for the SAR/GMTI as set forth in the PBS into three categories: Band A –[m}andatory critical system performance requirements that must be met—; Band B –[s]ystem performance capabilities with levels of performance that are Highly Desired—; and Band C –[s]ystem performance capabilities with levels of performance that are Desired.— RFP sect. M.3.1.

Award was to be made to the offeror whose proposal was determined to be the most beneficial to the government (–best value—) when evaluated under four factors: (1) technical, with sub-factors for technical solution, (which was slightly more important than) schedule/production capability, (which was significantly more important than) supportability; (2) performance risk; (3) price/cost; and (4) small business participation. Overall, the technical factor was significantly more important than performance risk, which was slightly more important than price/cost, which was significantly more important than small business participation. The non'price/cost factors combined were significantly more important than price/cost.

Of significance here, the RFP provided that the evaluation under the technical factor would consider (1) adequacy of response, (2) understanding the requirement, (3) feasibility of approach, and (4) the results of a scheduled 2-3 day flight test to be performed as part of the proposal process. Regarding the required flight test, the solicitation generally provided as follows:

d. Flight Test. The Flight Test will be used to aid in the assessment of the Offeror's ability to produce the system as proposed. The Flight Test will be evaluated to help to determine the level of confidence provided the Government with respect to the Offeror's methods and approach in successfully completing the proposed tasks and technical requirements within the proposed schedule. The Flight Test will be evaluated in accordance with the Flight Test Plan attached to the RFP. The extent to which the Offeror successfully demonstrates higher levels of performance capabilities during the Flight Test will be considered. As a minimum, the Offeror must successfully demonstrate the following four areas: SAR Modes: Spot and Strip; GMTI Mode; SAR Strip Resolution of 0.3m and 1.0m; and SAR Spot Resolution of 0.3m. Failure to successfully demonstrate any one of these areas will be deemed a deficiency and will result in a Technical Factor rating of Unacceptable. Offerors are cautioned that they will be provided only one flight test session. Accordingly, Offerors will not be afforded an opportunity to correct any deficiencies received in the Flight Test.

(Emphasis in original.) RFP sect. M.3.1.

By the September 24 closing date, CECOM had received offers from GA and Northrop Grumman Corporation (NG). Both GA and NG demonstrated SAR/GMTI systems during the required source selection flight tests in October 2007 that were evaluated as demonstrating sufficient capability in the four required areas to pass the flight test. However, analysis of the source selection flight test data resulted in GA's SAR/GMTI system receiving only a moderate confidence rating with respect to meeting proposed SAR performance, and a low confidence rating with respect to meeting proposed GMTI performance.

While GA's system passed the flight test, its proposal was rated unacceptable under the technical factor. In this regard, the source selection evaluation board (SSEB) evaluated GA's proposal as having nine deficiencies under the technical solution subfactor, including such deficiencies as: modifying and not proposing to meet 15 mandatory Band A critical system performance requirements in its proposed Product Specification for incorporation into the contract; omitting Band A –shall— requirements from its proposed statement of work; failing to substantiate claimed operational ranges; failing to substantiate GMTI performance parameters; and failing to substantiate mandatory Band A requirements for reliability through test data or analysis. In addition to the nine deficiencies, GA's proposal was evaluated as having 13 significant weaknesses and 32 weaknesses under the technical solution subfactor. The SSEB nevertheless rated GA's proposal as susceptible of being made acceptable under the technical solution subfactor on the basis that its system had passed the flight test. However, the source selection authority (SSA) determined that, given the evaluated major deficiencies in GA's proposal and the SSEB's conclusion that a major rewrite of the proposal would be required to make it acceptable, a rating of unacceptable under the technical solutions subfactor was consistent with the rating definitions in the source selection evaluation plan. Competitive Range Determination at 3-4; Tr. at I/294-I/305.[2]

The SSEB also rated GA's proposal unacceptable under the schedule/production capability subfactor (on the basis of 8 deficiencies, 12 significant weaknesses, and 4 weaknesses) and under the supportability subfactor (on the basis of 3 deficiencies and 5 weaknesses). In addition to its overall unacceptable rating under the technical factor, GA's proposal was rated as posing moderate performance risk and as being only susceptible of being made acceptable under the small business factor.

In contrast, NG's proposal was rated as susceptible of being made acceptable under the technical factor on the basis of a susceptible rating under the technical solution subfactor, which was the result of a single evaluated deficiency (proposing in its Product Specification to modify a Band A requirement for operational slant range performance by adding a caveat regarding [REDACTED]). NG's proposal was rated acceptable under the schedule/production capability subfactor and good under the supportability subfactor. In addition, NG's proposal was rated as posing low performance risk and as outstanding under the small business factor.

The SSA determined that, in light of the fact that GA's proposal was evaluated as unacceptable (with major deficiencies) under the technical factor, and would require a major rewrite to become acceptable, it was not one of the most highly rated proposals, and on that basis excluded it from the competitive range. This resulted in a competitive range consisting only of NG's proposal. Upon learning of its exclusion from the competitive range, and after being debriefed, GA filed this protest with our Office.

GA challenges the determination to exclude its proposal from the competitive range primarily on the basis that the agency did not reasonably account for the performance of its SAR/GMTI system in the flight test or the information in GA's proposal. GA specifically denies that it took exception in its proposed Product Specification to the requirements of the solicitation PBS.

We will review an agency's evaluation and exclusion of a proposal from the competitive range for reasonableness and consistency with the solicitation criteria and applicable statutes and regulations. Novavax, Inc., B-286167, B-286167.2, Dec. 4, 2000, 2000 CPD para. 202 at 13. Contracting agencies are not required to retain in the competitive range proposals that are not among the most highly rated or that the agency otherwise reasonably concludes have no realistic prospect of being selected for award. Federal Acquisition Regulation sect. 15.306(c)(1); D & J Enter., Inc., B'310442, Dec. 13, 2007, 2007 CPD para. 8 at 2.

We have reviewed all of GA's challenges to the competitive range determination and find that they furnish no basis to question the exclusion of GA's proposal. We discuss GA's most significant arguments below.

FLIGHT TEST

In significant measure, GA's challenge to the exclusion of its proposal from the competitive range rests upon the assumption that there is an inconsistency between its exclusion and the fact that the performance GA demonstrated during the required flight test was sufficient to pass the test. Indeed, it appears from the record that GA's approach to this procurement seems to have been primarily focused on passing the flight test. In this regard, GA's director of radar programs for reconnaissance systems testified, in response to a question as to GA's –proposal approach, in terms of providing . . . basic radar parameters,— as follows:

Okay. When you read the solicitation . . . you must pass the flight test. This was about the flight test. Everything else was secondary to the flight test. . . .

Tr. at II/366-II/367.

As noted by the agency, however, the proposal requirements as established by the solicitation extended significantly beyond the requirement that the system demonstrated at the source selection flight test receive a passing grade. Thus, offerors were required to describe their proposed approach to meeting the PBS requirements in detail; the solicitation provided as follows:

[t]he Offeror shall present the extent to which its proposed solution and capabilities/characteristics compare to the solicitation, SOO, and PBS. The Offeror shall address all PBS Band A requirements, shall discuss how they will meet all specified levels of performance therein, and describe their proposed solution in meeting these requirements. The Offeror shall address all PBS Band B and C requirements. Offerors are not required to meet the desired performance capabilities with levels of performance set forth in Bands B and C in their proposed solution; however, they must discuss each Band B and C capability and the levels of performance that their proposed solution will achieve.

RFP sect. L.2.2.1.1.2. Furthermore, offerors were required not merely to describe in detail their proposed approach to all of the PBS requirements; they were also required to substantiate that the performance of their proposed system would meet the PBS requirements, at least insofar as they concerned the Band A requirements.[3] In this regard, the RFP provided as follows:

The Offeror shall deliver sample imagery and describe existing analysis, test data, and/or empirical data that substantiate the SAR/GMTI performance, environmental qualification, and reliability claims for the proposed system. Independent military test data is preferred. Data submitted in accordance with this section, along with data collected during the flight test, will be used for proposal evaluation. Proposals must contain supporting rationale for any claimed capabilities. Detailed test result information may be included under Section 4, Supporting Documentation, and will not be included as part of the Technical Volume page count. If the Offeror lacks actual test or usage data regarding performance, environmental qualification or reliability claims the Offeror must specify and justify the level of risk associated with this lack of data. Additionally the Offeror shall address how other techniques (modeling, analysis, etc.) provide the Government with confidence to verify the Offeror's claims based upon the lack of test data.

RFP sect. L.2.2.1.1.4.

While, as noted by the protester, the solicitation provided for consideration of the flight test results as part of an offeror's substantiation of its claimed level of performance, the 2-3 day flight test provided for under the solicitation was intended to address only a limited subset of the overall required performance capabilities, and then only in a limited manner. Thus, offerors were required to demonstrate, at a minimum, only four capabilities''SAR Spot-mode images at 0.3 meters resolution, SAR Strip-mode images at 0.3 and 1.0 meter resolutions, and GMTI''of the many capabilities required under the PBS. RFP sect. M.3.1; SAR/GMTI Source Selection Test Flight Plan at 1. Indeed, had the flight test been intended to comprehensively demonstrate the full range of required performance, the record indicates that the flight test would have required several months rather than the 3 days of testing contemplated under the solicitation. Tr. at I/22-I/24, I/216-I/217, I/319, II/113. Moreover, a demonstrated SAR/GMTI system could pass the flight test even if the performance shown during the test fell short of the mandatory Band A minimum level of performance; all that was required was a demonstration of –some capability— with respect to each of the four capabilities required to be demonstrated during the test. Tr. at I/323-I/324. In these circumstances, given the limited scope of the flight test and the fact that passing the test did not establish compliance with all mandatory PBS requirements, we see no necessary inconsistency between passing the test and being evaluated as not substantiating compliance with the mandatory PBS performance requirements.

FAILURE TO MEET MANDATORY BAND A REQUIREMENTS

Our review of the record indicates that the agency reasonably determined that GA failed to substantiate its compliance with and/or took exception to the required levels of performance with respect to significant mandatory Band A requirements. We discuss several examples below.

Probability of Detection

The solicitation PBS established a mandatory Band A requirement for the likelihood that the GMTI mode would detect a specific moving target if that target was within the field of view of the radar. Specifically, the PBS provided for a minimum GMTI probability of detection (PD), stating that –[t]he radar-payload shall be capable of providing 85% PD (Band A); 90% PD (Band C), at all range-extents, of a standard military target (as defined in paragraph 3.2).— PBS sect. 3.3.4.1, GMTI Probability of Detection (PD).

GA indicated in its Product Specification that its system would exceed the Band A 85 percent PD requirement, stating that its –radar-payload shall be capable of providing 90% PD of a standard military target within the operational range/altitude envelopes.— GA Product Specification sect. 3.3.4.1. However, the performance GA's GMTI mode demonstrated during the source selection flight test was inconsistent with its claim of a 90 percent PD, with GA's GMTI mode averaging only a [REDACTED] percent PD. Tr. at I/322, II/169; Analysis Report for GA SAR/GMTI LRIP Source Selection Flight Test at 24. GA did include in its proposal a summary data table listing 7 prior SDD test flights, in 6 of which the tested system reportedly demonstrated a PD in excess of the Band A requirement here for an 85 percent PD. GA Technical Proposal at 36/37. However, the agency determined, and GA has not shown otherwise, that GA failed to furnish a sufficient description of the test flights and explanation showing how the claimed PDs were calculated; as a result, the test results could not serve as substantiation for GA's Product Specification PD. Indeed, GA's failure to show how the prior test flight PDs were calculated was considered especially significant by the agency since, on prior occasions, GA had claimed higher PDs than the agency's own calculations had indicated. TR at II/34-II/35, II/159-II/169.

We conclude that the agency reasonably determined that GA's proposal failed to substantiate compliance with required Band A GMTI PD.

GMTI Minimum Detectable Velocity/Velocity Ambiguity

The record indicates that, in general, the difficulty in detecting a target with a GMTI system increases the slower the target is moving and the faster the air vehicle is moving. Tr. at I/95-I/100, I/325, II/40-II/41 II/172-II/173. In this regard, the solicitation PBS established a mandatory Band A requirement that the GMTI probability of detection requirements, which include a Band A 85 percent GMTI PD requirement, be met for targets with a radial velocity''a measurement of target velocity toward or away from the radar''of between 10 kilometers per hour (kph) (approximately 2.78 meters/second (m/s)) and 70 kph (19.4 m/s). Tr. II/38-II/40, II/170. Further, the PBS required that measurement of the velocity of the target, and thus the report to the consumer of the radar intelligence, not be ambiguous or incorrect within the required range of target velocity. Specifically, the PBS provided as follows:

The Payload shall be capable of meeting probability-of-detection requirements for targets moving at radial speeds between 10 kph and 70 kph (Band A), 4 kph and 120 kph (Band C) at all look-angles, within the radar look angle envelope, measured relative to the AV's velocity vector. The detected speeds must also be unambiguous within the proposed range.

PBS sect. 3.3.4.1.1, GMTI-Detectable Velocities.

GA provided in its Product Specification as follows regarding this requirement:

For AV [air vehicle] velocities of [REDACTED] kph, the payload shall be capable of meeting probability-of-detection requirements for targets moving at radial speeds between 10 kph and 70 kph at all look-angles, within the radar look angle envelope, measured relative to the AV's velocity vector. The detected speeds must also be unambiguous within the proposed range.

GA Product Specification sect. 3.3.4.1.1.

The agency determined that because GA's Product Specification on its face only obligated GA to meet the GMTI PD requirement for the required range of target velocity when the air vehicle was moving at the single velocity of [REDACTED] kph, GA had unacceptably qualified its obligation under the PBS to meet PD requirements for targets moving at radial speeds between 10 kph and 70 kph. We find the agency's determination of noncompliance in this regard to be reasonable since GA has pointed to nothing in the PBS that limited the GMTI PD requirement to instances where the air vehicle was moving at a velocity of [REDACTED] kph. Tr. at I/97-II/100.[4]

Furthermore, the agency determined that GA had failed to substantiate its performance over the entire required range of target velocity. Specifically, during the source selection test flight, the minimum detectable target velocity demonstrated by GA's GMTI mode was [REDACTED] m/s ([REDACTED] kph), significantly in excess of the mandatory minimum of approximately 2.78 m/s (10 kph). Analysis Report for GA SAR/GMTI LRIP Source Selection Flight Test at 24. Although in the course of this protest GA has explained that its failure during the source selection flight test to meet the lower end of the required target velocity range was simply the result of a radar setting by the GA radar operator, Tr. at II/335-II/338, GA has made no showing that this explanation was apparent from GA's proposal. In any case, we agree with the agency that GA's explanation is undermined by the fact that GA's technical proposal included a summary data table listing seven prior SDD test flights, all of which were shown as having a minimum detectable velocity of [REDACTED] m/s, and the fact that GA also included in its proposal two videos of SDD GMTI data collection that indicate a minimum detectable velocity of [REDACTED] m/s or greater. GA Technical Proposal at 36/37; Tr. at II/40'II/42, II/171-II/186; Agency Comments, Mar. 17, 2008, at 9. Moreover, the agency determined, and GA's director of radar programs for reconnaissance systems conceded in his testimony, that there was no description in its proposal of how it would meet the minimum detectable velocity requirement. Tr. at II/411.

In these circumstances, where the difficulty in detecting a target with a GMTI system increases the slower the target is moving and the faster the air vehicle is moving; GA appeared to impose a significant limit (through a restriction on air vehicle speed) in its Product Specification on its obligation to meet the 85 percent PD requirement; and the data from both its source selection flight test and prior flight tests all indicated a minimum detectable velocity of no less than [REDACTED] m/s, we think the agency reasonably determined that GA's GMTI mode appeared to have a –hard limit— of [REDACTED] m/s, and thus was noncompliant with the mandatory Band A minimum detectable velocity of approximately 2.78 m/s (10 kph). Tr. at II/41; Analysis Report for GA SAR/GMTI LRIP Source Selection Flight Test at 25.

Further, in addition to the apparent inability of GA's GMTI to meet the mandatory Band A requirements with respect to minimum detectable velocity, the agency observed that the performance of GA's GMTI mode during the source selection flight test failed to meet the Band A requirement that the detected speeds be unambiguous, that is, accurately reported, within the proposed range of target speeds. Specifically, while GA proposed to meet (in some limited circumstances) the mandatory Band A PBS requirement that the GMTI mode be capable of meeting PD requirements for targets moving at radial speeds between 10 kph (approximately 2.78 meters/second (m/s)) and 70 kph (19.4 m/s), the test reports indicate that the detected speeds were unambiguous only up to a maximum of approximately [REDACTED] kph ([REDACTED] m/s), and thus fell short of meeting the requirement that they be unambiguous up to 70 kph (19.4 m/s). PBS 3.3.4.1.1, GMTI-Detectable Velocities; Analysis Report for GA SAR/GMTI LRIP Source Selection Flight Test at 30. Moreover, the agency determined, and GA has not shown otherwise, that GA's proposal did not explain how GA would address the ambiguity problem. Tr. at II/186. The results of the flight test indicated to the agency that there may also be a hard upper limit to the possible radar performance in this regard, perhaps related to the radar's [REDACTED]. Tr. at I/326-I/329, II/46'II/47. GA has not shown this conclusion was unreasonable, given the information available to the agency in its proposal.

In summary, we find that the agency reasonably determined both that GA had failed to substantiate the compliance of its GMTI mode with the mandatory Band A minimum detectable velocity and velocity ambiguity requirements, and that the information in GA's proposal reasonably called into question whether it would in fact meet those requirements.

Basic Radar Data

Based on its review of GA's proposal, the agency determined that GA generally had failed to furnish basic radar design parameters, detailed hardware and software design configurations, details on radar modes, and test data, modeling and analysis sufficient to permit the agency to understand how GA's radar would perform over the range of expected actual operating conditions. The agency concluded that, in the absence of sufficient basic radar information, it could not adequately understand how GA's proposed system was likely to perform, and thus was unable to independently validate many of GA's claimed capabilities.

GA maintains that there was no solicitation requirement that offerors furnish basic radar parameters. We disagree. The solicitation required offerors to –deliver sample imagery and describe existing analysis, test data, and/or empirical data that substantiate the SAR/GMTI performance, environmental qualification, and reliability claims for the proposed system.— RFP sect. L.2.2.1.1.4. Further, where the offeror lacked –actual test or usage data— substantiating the performance of its system, it was required to –address how other techniques (modeling, analysis, etc.) provide the Government with confidence to verify the Offeror's claims.— Id.Since GA lacked sufficient test data to substantiate its compliance with all of the mandatory Band A requirements, GA was required by these provisions to furnish modeling and analysis to substantiate its system's compliance with the PBS requirements. The agency found, and GA's director of radar programs for reconnaissance systems conceded in his testimony, that GA's proposal did not include detailed modeling or analysis of its system's performance. Tr. at II/384-II/385.

GA nevertheless asserts that section four of its proposal (Supporting Documentation), included –all or most— of the basic radar information necessary to allow the agency to do an analysis of GA's system. GA Comments, Mar. 17, 2008, at 3. According to GA, a subject matter expert could –develop assumptions on the design—; –make an approximation of the antenna pattern assuming good design practices—; assume that GA used a particular one of the number of available weighting functions in order to reduce sidelobe signal levels; and otherwise make assumptions or derive information such that a subject matter expert –could establish a first order approximation of the predicted performance— of the GA system. GA Comments, Mar. 12, 2008, at 3-6.

In fact, however, the agency's radar subject matter experts at the Georgia Tech Research Institute were unable to model the theoretical performance of GA's SAR/GMTI radar system due to the lack of basic radar design information in GA's proposal. Tr. at II/93-II/94. Given GA's concession that it's proposal did not include such details as the radio frequency and signal processing losses at each stage of the processing of the radar signals, GA Comments, Mar. 12, 2008, at 3; given the various assumptions GA agrees a subject matter expert would need to make in order to model system performance; and given the fact that, at best, GA claims only that –a first order approximation of the predicted performance— could be obtained, we find no basis to question either the determination by the Georgia Tech Research Institute radar subject matter experts that they could not adequately model the theoretical performance of GA's SAR/GMTI radar system, or the conclusions of the agency's own radar experts that there was insufficient basic radar information about GA's system to permit them to independently validate many of GA's claimed capabilities. In any case, it is an offeror's responsibility to submit a well-written proposal, with adequately detailed information, that clearly demonstrates compliance with the solicitation requirements and allows a meaningful review by the procuring agency. CACI Techs., Inc., B-296946, Oct. 27, 2005, 2005 CPD para. 198 at 5; Ace Info. Solutions, Inc., B-295450.2, Mar. 7, 2005, 2005 CPD para. 75 at 8. Where, as here, an offer does not affirmatively demonstrate compliance with mandatory requirements, the offeror risks rejection of its proposal. HDL Research Lab, Inc., B-294959, Dec. 21, 2004, 2005 CPD para. 8 at 5.

We conclude that the Army reasonably determined that GA's proposal failed to substantiate compliance with, and/or took exception to the required levels of performance with respect to significant mandatory Band A requirements. Further, given the significant deficiencies in GA's proposal, and the reasonable determination on the part of the agency that some of these deficiencies resulted from fundamental limitations on the performance of GA's proposed system, or otherwise would require a major rewrite to correct, the agency reasonably excluded GA's proposal from the competitive range.

The protest is denied.

Gary L. Kepplinger
General Counsel