Missile Defense:

The Warfighter and Decision Makers Would Benefit from Better Communication about the System's Capabilities and Limitations

GAO-18-324: Published: May 30, 2018. Publicly Released: May 30, 2018.

Multimedia:

Additional Materials:

Contact:

Cristina Chaplain
(202) 512-4841
chaplainc@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Missile Defense Agency is developing a system to track and destroy enemy missiles. In our annual review of this system, we found that, as in prior years, MDA has made progress developing, testing, and delivering some parts of this system but did not always complete its goals. For example, while MDA delivered some kinds of missiles to the military as planned, other missiles were delayed.

We also found some of these assets are not delivered with information users need, like limitations about how they operate or how they were tested. Our six recommendations include actions MDA can take to better communicate this and other information.

Components of the Ballistic Missile Defense System

Photo collage showing sea-based, land-based, and ground-based missile defense systems, and a command and control station.

Photo collage showing sea-based, land-based, and ground-based missile defense systems, and a command and control station.

Multimedia:

Additional Materials:

Contact:

Cristina Chaplain
(202) 512-4841
chaplainc@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

In fiscal year 2017, the Missile Defense Agency (MDA) made mixed progress in achieving its delivery and testing goals.

MDA continued to deliver assets to the military services. However, system-level integrated capabilities, such as some discrimination and integrated cyber defense improvements, were delayed and delivered with performance limitations.

Several programs achieved notable firsts, including the first intercept of an Intercontinental Ballistic Missile. However, one program experienced a failure, and other tests were delayed or deleted.

Moreover, GAO found challenges in MDA's processes for communicating the extent and limitations of integrated capabilities when they are delivered. As a result, warfighters do not have full insight into the capabilities MDA delivers.

GAO found that the average length of the undefinitized period and the not-to-exceed price of MDA's undefinitized contract actions, which authorize contractors to begin work before an agreement on terms, specifications, or price have been agreed upon, have increased over the past 5 years. While MDA policy permits use of undefinitized contracts on a limited basis, GAO and others have found that they can place unnecessary cost risks on the government.

MDA does not completely assess BMDS performance using traditional flight tests. Instead, MDA relies on models, some of which produce data with limited credibility. According to Department of Defense and MDA policy, models used to operationally assess weapons systems must be accredited to ensure they reflect the real-world system. In addition, using unaccredited models increases the risk that test results could be distorted, and leaves decision makers without key information on how the system will perform. While MDA has taken steps to improve its models, it has used many models in system operational ground tests that were not certified for that use (see figure). Additionally, MDA does not communicate model limitations to some decision makers.

Percentage of Accredited Models Used in Operational Assessments of Ballistic Missile Defense System Capability Deliveries, 2015 through 2017

U:\Work in Process\Publishing\FY18 Rpt\301-400\324\Graphics\Highlights_v1_102076-rmw.tif

Why GAO Did This Study

Since 2002, MDA has been developing a Ballistic Missile Defense System that can identify and intercept enemy threats. MDA has received approximately $132 billion and is planning to spend an additional $47.8 billion through fiscal year 2022 to continue its efforts.

The National Defense Authorization Act for Fiscal Year 2012 included a provision that GAO annually assess and report on the extent to which MDA has achieved its acquisition goals and objectives. This report addresses (1) the progress MDA made in achieving fiscal year 2017 goals; (2) the extent to which MDA uses contracting vehicles known as undefinitized contract actions; and (3) the extent to which models provide credible information about the system's operational performance. To do this work, GAO reviewed planned fiscal year 2017 baselines and other documentation and assessed them against baseline reviews and GAO's acquisition best practices guides. In addition, GAO interviewed relevant officials.

What GAO Recommends

GAO is making six recommendations to, among other things, improve the way MDA communicates capability deliveries; better report information about MDA's use of undefinitized contract actions; and address the challenges MDA has encountered with certifying its test models and communicating limitations of those models. DOD partially concurred with the first recommendation and concurred with the other five. GAO continues to believe the recommendations are valid as discussed in the report.

For more information, contact Cristina Chaplain at (202) 512-4841 or chaplainc@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: DOD partially concurred with this recommendation. While DOD agreed with the intent of this recommendation as we note in GAO-18-324, DOD stated that the Director, MDA will determine which major integrated capabilities should be delivered via the TCD process. The department also noted that the agency developed a list of such capabilities that it will update annually. These actions are an improvement over the current process, but they do not meet the full intent of our recommendation. For example, the list of future TCDs that MDA produced is not inclusive of all future integrated capabilities. This poses the risk that not all integrated capabilities will be delivered to the warfighter with comprehensive information. We continue to believe that in order for the agency to meet the full intent of our recommendation, it should establish in policy, a clear, definitive standard for which capabilities require a TCD delivery.

    Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA revises MDA policies to require that all integrated capabilities--capabilities that require integration of two or more elements--be included in a Technical Capability Declaration. (Recommendation 1)

    Agency Affected: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering

  2. Status: Open

    Comments: DOD concurred with this recommendation and contends that its new Policy Memorandum 90 meets the intent of our recommendation. However, this new Policy Memorandum leaves open the possibility of continued inconsistent application of the TCD process. In order for the agency to meet the full intent of our recommendation, MDA should establish in policy, a clear, definitive standard for which capabilities require a TCD for delivery. We believe it necessary to wait until MDA delivers a TCD in accordance with the new parameters set out in the Policy Memorandum.

    Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA clarifies, in written policy, the exact requirements, process, and key milestones necessary to issue a Technical Capability Declaration, including a requirement that the Assessment Requirements Review be held in such a time frame that it can provide meaningful input to MDA's test plans. (Recommendation 2)

    Agency Affected: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering

  3. Status: Open

    Comments: DOD concurred with this recommendation and has taken steps to implement it. While we noted in GAO-18-324 that the BMDS Accountability Report for 2018, approved by the Director, MDA on March 9, 2018 has taken steps to implement this recommendation, we will leave this recommendation open to ensure that the BMDS Accountability Report accurately contains all undefinitized contract actions that the agency enters into through the fiscal year and whether the reporting contains the information necessary to satisfy our recommendation.

    Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA includes information on current undefinitized contract actions in the BMDS Accountability Report, including the not-to-exceed price, the definitization schedule, the amount of funds obligated for the undefinitized period, and any changes since the contract action was entered into. (Recommendation 3)

    Agency Affected: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering

  4. Status: Open

    Comments: DOD concurred with this recommendation and stated that MDA is actively working with the BMDS Operational Test Agency (BMDS OTA) to resolve any issues associated with, and the reporting of, modeling limitations. However, as we note in GAO-18-324, according to BMDS OTA officials, MDA has never completely provided the needed data on time and often missed numerous subsequent deadlines. We, therefore, believe it is premature to close out this recommendation but will continue to track MDA's progress on taking the necessary steps to implement this recommendation.

    Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA, ensures that models used for operational tests are validated and accredited for such assessments. To help achieve this, MDA should provide the BMDS Operational Test Agency all evidence previously agreed to and needed to accredit models before ground testing events, as specified in the BMDS OTA accreditation plan. (Recommendation 4)

    Agency Affected: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering

  5. Status: Open

    Comments: DOD concurred with this recommendation and stated that MDA has made significant progress over the last year in these efforts. However, until the BMDS Operational Test Agency can ensure that the status of the models used are included in subsequent delivery packages we believe it is premature to close this recommendation. We will continue to monitor MDA's progress in taking the necessary steps.

    Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA includes in capability delivery packages, such as the Technical Capability Declaration memos and Operational Capability Baseline change packages, the following: a. The verification, validation, and accreditation status of the models used in operational ground tests; and b. Modeling and simulation limitations that affect operational ground test results. (Recommendation 5)

    Agency Affected: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering

  6. Status: Open

    Comments: DOD concurred with this recommendation and are taking steps to implement this recommendation. We will continue to monitor the efforts necessary including finalizing and executing written agreements between MDA and any BMDS elements that are Service-operated but represented in BMDS performance assessments.

    Recommendation: The Under Secretary of Defense for Research and Engineering should ensure that the Director, MDA and the Secretaries of the Armed Services responsible for operating BMDS elements develop written agreements as soon as feasible for modeling and simulations technical and funding requirements for any BMDS elements that are service-operated but represented in BMDS performance assessments. (Recommendation 6)

    Agency Affected: Department of Defense: Office of the Under Secretary of Defense for Research and Engineering

 

Explore the full database of GAO's Open Recommendations »

Sep 20, 2018

Sep 10, 2018

Sep 6, 2018

Sep 5, 2018

Sep 4, 2018

Aug 16, 2018

Aug 15, 2018

Aug 10, 2018

Aug 8, 2018

Looking for more? Browse all our products here