Defense Intelligence: Comprehensive Plan Needed to Improve Stakeholder Engagement in the Development of New Military Intelligence System
Fast Facts
Intelligence collected on foreign militaries is critical for the U.S. military. The Defense Intelligence Agency's database for this intelligence doesn't meet current needs. The agency plans to replace it with a new system called the Machine-assisted Analytic Rapid-repository System, or "MARS."
The agency hasn't planned adequately for ongoing, robust engagement with intelligence consumers, producers, and contributors during development of the system. Such engagement could help identify and rectify deficiencies early.
We recommended that the agency develop a plan that details how it will involve all stakeholders in system development.
Highlights
What GAO Found
The Defense Intelligence Agency (DIA) intends for its new foundational military intelligence system, the Machine-Assisted Analytic Rapid-Repository System (MARS), to have capabilities to collect and synthesize data that significantly exceed those of its legacy system. DIA began developing MARS in 2018. DIA and stakeholders have identified policy, technical, and operational risks facing the program. For example, MARS needs to ingest a significant variety of data types (e.g., signals, text, imagery), which cannot be done using any one data model. DIA's use of Agile development processes helps to manage risk to some degree, because it emphasizes continuously delivering software that addresses users' priority needs and incorporating frequent user feedback to inform development. However, these processes do not address all MARS risks, such as those related to implementing new technologies. To address these and other risks, DIA has begun establishing a risk management process. It will be important for DIA to continue to manage risk throughout MARS's life cycle.
DIA has taken a number of actions to identify the needs of Department of Defense (DOD) and intelligence community organizations that will use MARS—including combatant commands, services, and intelligence agencies—but it lacks a comprehensive plan for engaging with these stakeholders (see fig.).
Machine-Assisted Analytic Rapid-Repository System (MARS) Stakeholder Roles
DIA has conducted workshops to identify initial requirements and town halls to disseminate MARS updates. However, stakeholders expressed mixed views on the quality and extent of the program's continual engagement with them, and stakeholders were unsure about how their feedback and input are helping DIA prioritize and select MARS features. Further, DIA's initial stakeholder engagement and test plan lacks key details called for in leading practices for system development and GAO's Agile Assessment Guide , such as how stakeholders will be engaged at different times and how their feedback will be compiled to support the prioritization of features. Without a comprehensive plan to guide stakeholder engagement, MARS capabilities could fall short of stakeholder expectations, and DIA could face greater challenges in mitigating risks to MARS.
Why GAO Did This Study
Foundational military intelligence—all-source intelligence collected by the Intelligence Community (IC) on other countries' militaries—is a critical element in planning for military operations. The DIA legacy system for processing such intelligence is unable to meet current needs, and DIA intends to replace it with MARS. MARS is expected to transform the way the IC approaches and generates foundational military intelligence. However, agencies can face a wide array of issues in developing a new system of this magnitude, including incorporating feedback from a large number of stakeholders.
A committee report accompanying the Intelligence Authorization Act for Fiscal Years 2018, 2019, and 2020 includes a provision for GAO to review MARS development. This report (1) describes the initial risks DIA and stakeholders have identified in the development of MARS and the actions DIA has taken to manage risk and (2) assesses how DIA is engaging potential stakeholders in the development of capabilities for the MARS program.
GAO reviewed related documentation; interviewed DOD and intelligence community officials about the risks to MARS development and how DIA is engaging stakeholders; and attended a stakeholder engagement meeting.
Recommendations
GAO recommends that the Office of the Director of National Intelligence (ODNI), in coordination with the Secretary of Defense, ensure that DIA develops a comprehensive plan for engaging MARS stakeholders. ODNI concurred with GAO's recommendation.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of the Director of National Intelligence | The Director of National Intelligence, in coordination with the Secretary of Defense, should ensure that the Defense Intelligence Agency develops a comprehensive plan that details how it will engage all MARS stakeholders, to include how specific stakeholders will be engaged at different times, how stakeholder feedback will support the prioritization of features, how stakeholder engagement and feedback will be monitored, and how user satisfaction will be measured. |
Closed – Implemented
In June 2021, the Defense Intelligence Agency issued a detailed stakeholder engagement plan that, among other things, includes the identification of the stakeholder base, how stakeholders will be engaged, and mechanisms to monitor stakeholder engagement. This stakeholder engagement plan meets the intent of our recommendation.
|