What We Found
The Department of Defense can better ensure that its sizeable weapon systems investment will help yield a decisive and sustained U.S. military advantage by following knowledge-based practices and developing a plan to monitor recent acquisition reforms.
Since our 2019 High-Risk Report, our assessment of the Department of Defense’s (DOD) performance against our five criteria remains unchanged.
Leadership commitment: met. DOD continues to demonstrate a strong commitment, at the highest levels, to improving the management of its weapon system acquisitions. However, to sustain the met rating for this criterion, DOD will need to ensure it follows through to complete the implementation of new initiatives. It will also need to address leadership-related recommendations.
Since March 2019, DOD leadership has recognized the evolving challenges the department faces in fielding weapon systems that meet warfighter needs and has consistently taken steps to address them.
In June 2019, we reported that DOD made progress in implementing reforms to restructure the oversight of major defense acquisition programs, including shifting decision-making authority for many programs from the Office of the Secretary of Defense to military departments.
In 2020, DOD reissued its foundational acquisition guidance, emphasizing speed and agility in the acquisition process. The new guidance includes six acquisition pathways based on the characteristics and risk profile of the system being acquired. DOD has also issued supplemental guidance for these pathways and the functions that support them, such as cybersecurity and test and evaluation.
The guidance includes an increased focus on software development and cybersecurity practices that DOD leadership and others have recognized as a particular risk area for the department’s weapons system programs.
DOD leadership has also continued to make progress in clearly defining roles and responsibilities for acquisition oversight. In June 2019, we reported that DOD needed continued leadership attention to address challenges with implementing acquisition oversight reforms, including disagreements between the Office of the Secretary of Defense and the military departments about acquisition oversight roles. Subsequently, the Deputy Secretary of Defense issued a memorandum in December 2019 to define roles for acquisition oversight.
In July 2020, the department issued charters for the Under Secretaries of Defense for Research and Engineering and Acquisition and Sustainment. These two new offices responsible for acquisition oversight were created in response to congressional direction. The charters should help to further clarify roles and responsibilities.
However, work still remains at both the Office of the Secretary of Defense and military department levels to complete the development and implementation of acquisition policies. According to officials from the Office of the Under Secretary of Defense for Acquisition and Sustainment, (1) the military departments will also need to update their policies to align with department-wide policies, and (2) the department will need to develop streamlined processes and tools to support the effective implementation of the newly-issued policies.
In June 2019 we reiterated the importance of recommendations we originally made in 2015 to clarify and strengthen roles and responsibilities at the enterprise level for making portfolio management decisions. These recommendations aim to ensure that DOD’s investments are strategy driven, affordable, and balance near- and long-term needs. We noted that these recommendations may take on more importance for DOD in light of the implementation of acquisition reforms that will further diffuse responsibility for initiating and overseeing acquisition programs, but DOD has yet to implement them.
Capacity: partially met. In reshaping its acquisition organization to emphasize speed and agility, DOD acknowledged the importance of the acquisition workforce and took steps to increase its hiring and training for that workforce. DOD has made sufficient progress in addressing overall acquisition workforce shortfalls such that we have removed that issue from our Contract Management high-risk area this year.
However, since our last High-Risk Report in 2019, we and others reported on capacity challenges related to weapon system acquisition specifically. In June 2019, we reported that DOD faced challenges in filling vacancies in the Offices of the Under Secretaries of Defense for Research and Engineering and Acquisition and Sustainment, as well as gaps in skill sets such as data analytics that are critical to acquisition oversight.
In June 2020, we reported that many major defense acquisition programs reported difficulty in hiring software development staff with the required expertise and in time to complete the required work. DOD has taken initial steps to implement a statutory requirement to establish software development and acquisition training and management programs, but, according to a review by the Defense Innovation Board in March 2020, implementation is still in a formative stage.
With regard to cybersecurity, DOD’s Director, Operational Test and Evaluation reported in December 2019 that the department lacked testing personnel with deep cybersecurity expertise and stated that without substantial improvements in cybersecurity test and evaluation, especially in the workforce, DOD risks lowering overall force readiness and lethality.
Multiple reports we published between August 2015 and April 2020 indicate that DOD needs to enhance its acquisition policies in a number of areas. These areas include (1) ensuring that program cost estimates better conform to leading practices, (2) improving reliability and sustainment planning early in the acquisition process, (3) strengthening coordination and processes for portfolio management, and (4) improving the science and technology management framework to apply leading practices and encourage innovation.
Action plan: partially met. Since our last High-Risk Report in 2019, DOD began implementing planned actions to improve acquisition outcomes, including developing and issuing guidance for six new acquisition pathways under its adaptive acquisition framework. However, the department has yet to develop detailed plans for how it will assess whether the new acquisition pathways achieve intended outcomes, including the applicability of metrics to each pathway.
Additionally, because of changes to its annual performance reporting, DOD may have less insight than it had in the past into root causes of cost or schedule growth to allow it to develop effective action plans. From 2013 to 2016, DOD assessed and reported publicly on its acquisition performance across its full portfolio of programs, including analysis of causative factors. However, DOD’s reporting from 2017 onward includes only limited analysis of program cost and schedule performance and does not analyze causative factors.
While DOD began implementing actions for certain software and cybersecurity challenges, it is still developing implementation plans and policies for others. In response to numerous recommendations made in 2018 by the Defense Science Board and in 2019 by the Defense Innovation Board related to the implementation of leading software practices, DOD made certain existing software development capabilities available enterprise-wide and created working groups to address related workforce issues.
However, DOD is still analyzing how it will address a Defense Science Board recommendation related to machine learning in defense systems, which the board identified as a complicating factor for software acquisitions. Similarly, while we reported in October 2018 that DOD began initiatives to better understand and address cybersecurity vulnerabilities, as of December 2020 according to DOD officials, DOD is still developing and implementing policies in support of its Risk Management Framework approach to cybersecurity in weapons system acquisition.
Monitoring: partially met. The Under Secretary of Defense for Acquisition and Sustainment stated her commitment to conduct data-driven oversight of acquisition programs. Nearly all of these programs are now managed at the military department level instead of the Office of the Secretary of Defense level in part due to a fiscal year 2016 statutory reform. DOD has made progress in developing its approach to this type of oversight, such as completing data strategies for some acquisition pathways.
In June 2020, the Office of the Under Secretary of Defense for Acquisition and Sustainment announced plans to adopt a data and analytics strategy to facilitate data-driven oversight, which the Under Secretary’s office and the military departments are developing together.
This effort could help to address disagreements that we reported on in June 2019 between the Office of the Secretary of Defense and military departments about the amount of program information that military departments should be required to provide to the Office of the Secretary of Defense for certain programs. DOD emphasized the importance of resolving these disagreements in a November 2020 report to Congress in which it noted that ensuring data transparency across the DOD components was a challenge to improving acquisition data.
The department also has yet to take several actions we have recommended in the past to improve the availability and quality of data needed for effective monitoring. For example, in June 2019, we recommended that DOD develop a plan to assess recent acquisition reforms and to identify the necessary data. However, DOD has yet to determine how it will monitor most of the reforms we reviewed.
We also reported on continued challenges with data reliability for one of DOD’s new acquisition pathways—middle-tier acquisition—that is intended to deliver capabilities to the warfighter within 2 to 5 years. In our June 2020 assessment of DOD’s weapon system programs, we observed inconsistent cost reporting and wide variation in schedule metrics across these programs. These issues pose oversight challenges for Office of the Secretary of Defense and military department leaders trying to assess performance of these programs.
We and others also identified challenges with regard to DOD’s efforts to monitor software development efforts. For example, we reported in June 2020 that some weapon system programs did not submit required reports on software development efforts needed to prepare acquisition and life-cycle cost estimates.
Additionally, the Defense Innovation Board reported on several deficiencies with DOD’s software development metrics in May 2019. A working group comprised of DOD and industry officials recommended approaches to monitoring software development efforts in April 2020, but it is too soon to tell whether these approaches will be effective.
Demonstrated progress: partially met. In 2019, we identified a cost avoidance totaling $136 billion in procurement funding DOD realized from 2013 to 2018 after reforming business case and cost estimate practices. In 2019 and 2020, we reported a statistical correlation of lower cost and schedule growth for major defense acquisition programs that consistently implemented specific knowledge-based acquisition practices, such as maturing critical technologies and conducting preliminary design reviews prior to starting development.
These analyses provide evidence that DOD can reduce its cost and schedule growth by consistently implementing knowledge-based acquisition practices. However, our June 2020 assessment of DOD’s weapon systems still shows DOD’s inconsistent implementation of knowledge-based acquisition practices, even among its newer programs.
Programs also show extensive cost and schedule growth from their initial cost and schedule baselines, much of which is unrelated to the increase in quantities purchased.
DOD took significant steps in the past few years to implement acquisition reforms and to issue new guidance. These steps aim to streamline the acquisition process to help deliver capabilities faster and to improve software approaches and cybersecurity practices. It is likely too early to see effects of these reforms on the cost, schedule, and performance of the department’s weapon system acquisition programs.
Until DOD determines its action plan and ensures the availability and quality of data needed for monitoring, DOD and Congress cannot be sure whether the new reforms and policies are leading to the intended results.
In June 2020, we reported that DOD expects to invest about $1.8 trillion to acquire 106 new weapon systems. Congress and DOD have long sought to improve how DOD acquires these systems, yet many programs continue to fall short of cost, schedule, and performance goals. We added this area to our High-Risk List in 1990.
These challenges occur in an era when programs are more software driven than ever before and face global cybersecurity threats. However, software development continues to be a stumbling block for programs, and DOD has made only limited progress in addressing cybersecurity vulnerabilities.
A number of other issues could also affect DOD’s ability to keep pace with evolving threats, such as the ability to develop innovative technologies and the capabilities of the defense industrial base.
DOD is implementing significant changes in an effort to improve weapon system outcomes. However, considerable work remains, and until it is completed, DOD’s ability to quickly deliver capabilities to the warfighter is hindered.
Since we added this area to our High-Risk List in 1990, we have made hundreds of related recommendations. As of December 2020, 114 recommendations remain open, 56 of which we made since the last High-Risk Report in March 2019. To show a continued commitment to improving its weapon systems outcomes, DOD should implement our open recommendations including the following:
- Improve DOD’s ability to manage its portfolio by (1) requiring annual enterprise-level portfolio reviews that incorporate requirements, acquisition, and budget processes; (2) directing appropriate department staff to collaborate on their data needs; and (3) incorporating lessons learned from military service portfolio reviews and portfolio management activities, such as using multiple risk and funding scenarios to assess needs and reevaluate priorities.
- Determine (1) the metrics needed to assess middle-tier acquisition and acquisition programs other than major defense acquisition programs’ cost and schedule performance; and (2) how reliable data will be collected and shared between the services and the department to facilitate oversight.
- Implement several recommendations to individual programs related to knowledge-based acquisition practices including (1) fully maturing critical technologies prior to starting development; (2) ensuring program cost estimates are fully compliant with best practices including cost risk assessments; and (3) employing reliability and sustainment planning early in a program’s development to ensure realistic reliability requirements and that sustainment cost targets are met.
- Implement numerous recommendations to individual military departments and DOD components related to improving acquisition cost, schedule, and performance.
- Implement guidance for software development that provides specific, required direction on when and how often to involve users early in the development process and to continue involving users through development of related program components.
- Implement leading practices for managing science and technology programs.