Artificial Intelligence: DOD Should Improve Strategies, Inventory Process, and Collaboration Guidance
Fast Facts
The Defense Department believes that artificial intelligence will transform warfare, and failure to adopt AI technology could hinder national security. DOD is making organizational changes and investing billions of dollars to incorporate AI technology.
We found that DOD's AI-related strategies could be more comprehensive, such as by including full descriptions of the resources needed for developing AI-enabled technologies. In addition, DOD has not yet issued guidance that clearly defines the roles and responsibilities of components that participate in AI activities.
Our 7 recommendations address these and other issues.
Highlights
What GAO Found
The 2018 Department of Defense (DOD) Artificial Intelligence (AI) Strategy defines AI as the ability of machines to perform tasks that normally require human intelligence. The strategy and associated plans include some, but not all, characteristics of a comprehensive strategy. For example, DOD's nine AI-related strategies and plans do not include full descriptions of resources and investments and risk associated with adoption of AI-enabled technologies (See fig.). Issuing guidance to include all characteristics of a comprehensive strategy in future AI-related strategies could help DOD be better positioned to help managers ensure accountability and responsible use of AI.
Assessment of DOD Artificial Intelligence-Related Strategies and Plans
DOD has begun to identify and report on its AI activities, but limitations exist in its AI baseline inventory, such as the exclusion of classified activities. DOD officials said these limitations will be addressed in subsequent phases of the AI inventory identification process. However, DOD has not yet developed a high-level plan or roadmap that captures all requirements and milestones. Such a plan would provide DOD with a high-level, end-to-end view of all the features necessary to accomplish the program's goal to provide a complete and accurate inventory of AI activities to Congress and to DOD decision makers.
DOD organizations collaborate on AI activities, but can more fully incorporate leading collaboration practices. DOD uses a variety of formal and informal collaborative mechanisms that GAO's prior work has identified, such as interagency groups. DOD has partially incorporated leading collaboration practices, such as identifying leadership. However, DOD officials told us they are in the process of developing guidance and agreements that clearly define the roles and responsibilities of DOD components that participate in AI activities. By finalizing and issuing such guidance, DOD could help ensure all participants agree upon responsibilities and decision making on AI efforts across the department.
Why GAO Did This Study
DOD strategies state that AI will transform the character of warfare, and failure to adopt AI technology could hinder the capability of warfighters to defend our nation. DOD is making organizational changes and investing billions of dollars to incorporate AI technology, such as establishing the Joint AI Center to accelerate the delivery of AI-enabled capabilities across DOD.
House Report 116-442 accompanying the National Defense Authorization Act for Fiscal Year 2021, includes a provision for GAO to assess DOD's resources, capabilities, and plans for AI technology. This report evaluates the extent to which (1) DOD's AI Strategy and associated plans include characteristics of a comprehensive strategy; (2) DOD has identified and reported AI activities across the department; and (3) DOD collaborates on its AI activities. GAO reviewed relevant laws and DOD strategies that outline plans and processes to manage AI across the department, interviewed officials, and conducted a department-wide survey. This is a public version of a sensitive report that GAO issued in February 2022. Information that DOD deemed sensitive has been omitted.
Recommendations
GAO is making seven recommendations, including for DOD to issue guidance to include all characteristics of a comprehensive strategy; develop a high-level plan or roadmap for its AI inventory process; and finalize and issue guidance and agreements that define roles and responsibilities for AI collaboration. DOD concurred with all seven of these recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | The Secretary of Defense should ensure that the Deputy Secretary of Defense issues guidance to the Chief Digital and AI Officer (once established) and the JAIC, military services, and relevant DOD organizations to include all characteristics of a comprehensive strategy in future AI strategies and associated plans and to consider the key governance practices outlined in the GAO AI accountability framework. (Recommendation 1) |
Open
DOD concurred with this recommendation. In June 2022, DOD officials stated that the department is aware of the AI accountability framework developed by GAO and said that the principles have been incorporated in the forthcoming DOD Responsible AI Implementation Pathway and are informing the development of other AI strategies in the Department. The DOD AI Responsible Strategy an Implementation Pathway was issued in June 2022. In June 2023, DOD officials stated that they anticipate completion of the DOD Data, Analytics, and AI Strategy by August 31, 2023, but provided no update regarding the issuance of relevant guidance. When we confirm the actions the department has taken in response to this recommendation, we will provide updated information.
|
| Department of Defense | The Secretary of Defense should ensure that the Deputy Secretary of Defense, in consultation with the Chief Digital and AI Officer (once established), the JAIC, and military departments, establish documented procedures, including timelines, for the periodic review of the DOD AI Strategy and associated military service annexes to assess the implementation of the strategy and whether any revision is necessary. (Recommendation 2) |
Open
DOD concurred with this recommendation. In June 2023, DOD officials stated that by September 30, 2023, they expect to publish the assessment plan for the DOD Data, Analytics, and AI Strategy, which was initially expected in December 2022. DOD officials told us that the strategy will be overseen by the Chief Digital and Artificial Intelligence Officer (CDAO), and include a plan for periodic assessment and revision. When we confirm the actions the department has taken in response to this recommendation, we will provide updated information.
|
| Department of Defense | The Secretary of Defense should ensure that the Chief Digital and AI Officer (once established) and the Director of the JAIC in collaboration with the Office of the Under Secretary of Defense Comptroller and other entities, as appropriate, develop a high-level plan or roadmap—aligned with the best practices of an integrated master schedule—that captures all requirements, activities, and milestones that supports the preparation of the department's AI portfolio inventory and budget data. (Recommendation 3) |
Open
DOD concurred with this recommendation. In June 2022, DOD officials stated that by the end of December 2022 the Chief Digital and Artificial Intelligence Officer (CDAO) will complete a high-level roadmap-aligned with the best practices of an integrated master schedule-that captures all requirements, activities, and milestones that supports the preparation of the Department's AI portfolio inventory and budget data, as well as strategy implementation more generally. In November 2022, DOD officials provided a slide outlining a high-level timeline for DOD AI portfolio management. We requested additional information to demonstrate alignment with best practices. As of August 2023, DOD has not provided specific information that we requested. When we confirm the actions the department has taken in response to this recommendation, we will provide updated information.
|
| Department of Defense | The Secretary of Defense should ensure that the Deputy Secretary of Defense issues guidance that defines outcomes and monitors accountability for AI-related activities and includes AI key performance indicators. (Recommendation 4) |
Open
DOD concurred with this recommendation. In June 2023, DOD officials told us that the Chief Digital and Artificial Intelligence Officer (CDAO) will include high-level outcomes in the DOD Data, Analytics, and AI Strategy and help components translate that guidance to specific efforts and projects through efforts in AI test and evaluation, and monitor accountability for AI-related activities through CDAO governance processes. Additionally, officials noted that Section 1513 of the FY23 NDAA also tasked DOD to address this matter. DOD officials told us that the CDAO expects to brief Congress on the progress made on this front by end of June 2024, following a June 2023 briefing on the way-ahead. When we confirm the actions the department has taken in response to this recommendation, we will provide updated information.
|
| Department of Defense | The Secretary of Defense should ensure that the Chief Digital and AI Officer (once established) and the Director of the JAIC issue a roadmap or a high-level plan that captures all requirements and milestones for developing and onboarding users to the Joint Common Foundation. (Recommendation 5) |
Open
DOD concurred with this recommendation. In January 2023, DOD officials told us they completed the merge of key areas of the Joint Common Foundation (JCF) and Advana and provided us an integrated roadmap for current and planned AI development capabilities. DOD officials stated that the roadmap documents the key areas of JCF that merged and migrated to Advana, including improvements to the customer intake and onboarding processes. We reviewed the integrated roadmap and requested additional documentation to ensure its sufficiency in addressing our recommendation. We have not received an update as of August 2023. When we confirm the actions the department has taken in response to this recommendation, we will provide updated information.
|
| Department of Defense | The Secretary of Defense should ensure that the Deputy Secretary of Defense develops a timeline and guidance that directs Chief Digital and AI Officer (once established), and the JAIC, military services, and other relevant entities to establish common terminology for AI related activities. (Recommendation 6) |
Open
DOD concurred with this recommendation. In June 2023, DOD officials told us that they estimate establishment of a standard terminology for AI related activities as a part of required updates to existing DOD issuances, directives, and policies that define AI terminology by August 31, 2023, which they initially expected in December 2022. They also said their review of the existing guidance will be complete by December 31, 2023. These officials also stated that the Joint Staff is publishing a handbook that includes this terminology, and there is an effort underway to explore how best to use the handbook and the Intelligence Community data management lexicon on AI terminology. When we confirm the actions the department has taken in response to this recommendation, we will provide updated information.
|
| Department of Defense | The Secretary of Defense should ensure that the Deputy Secretary of Defense, in coordination with the Chief Digital and AI Officer (once established) and the JAIC, finalize and issue guidance and agreements that define the roles and responsibilities of the military services and other DOD organizations for leadership and relevant participants collaborating on AI activities. (Recommendation 7) |
Open
DOD concurred with this recommendation. In June 2022, DOD officials stated that by the end of December 2022, the Chief Digital and Artificial Intelligence Officer (CDAO) will coordinate with the Deputy Secretary of Defense to ensure the necessary guidance and agreements are established for the Services and other DOD organizations. In June 2023, DOD officials stated that the CDAO expects to publish a CDAO Charter and corresponding Capstone Directive that will define the roles and responsibilities of DOD components as they pertain to collaborating on AI activities by August 31, 2023. When we confirm the actions the department has taken in response to this recommendation, we will provide updated information.
|