Software Development: DOD Faces Risks and Challenges in Implementing Modern Approaches and Addressing Cybersecurity Practices

What GAO Found

According to the Department of Defense's (DOD) fiscal year (FY) 2021 budget request, DOD spent $2.8 billion on the 29 selected major business information technology (IT) programs in FY 2019. The department also reported that it planned to invest over $9.7 billion on these programs between FY 2020 and FY 2022. In addition, 20 of the 29 programs reported experiencing cost or schedule changes since January 2019. Program officials attributed cost and schedule changes to a variety of reasons, including modernization changes and requirements changes or delays. Seventeen of the 29 programs also reported experiencing challenges associated with the early impacts of the COVID-19 pandemic, including the slowdown of contractors' software development efforts.

DOD and GAO's assessments of program risk identified a range of program risk levels and indicated that some programs could be underreporting risks. Specifically, of the 22 programs that were actively using a register to manage program risks, DOD rated nine programs as low risk, 12 as medium risk, and one as high risk. In contrast, GAO rated seven as low risk, 12 as medium risk, and three as high risk. In total, GAO found 10 programs for which its numerical assessments of program risk reflected greater risk than reported by DOD, while DOD had three programs with greater reported risk than GAO. DOD officials noted that differences in risk levels might be associated with a variety of factors, including different risk assessment approaches. However, the differences in risk level GAO identified highlight the need for DOD to ensure that it is accurately reporting program risks. Until the department does so, oversight of some programs could be limited by overly optimistic risk perspectives.

As of December 2020, program officials for the 22 major DOD business IT programs that were actively developing software reported using approaches that may help to limit cost and schedule risks. (See table.)

Selected Software Development and Cybersecurity Approaches That May Limit Risks and Number of Major DOD Business IT Programs That Reported Using the Approach

Source: GAO analysis of Department of Defense questionnaire responses. | GAO-21-351
^aThe Defense Innovation Board encourages more frequent delivery of working software to users for Agile and DevOps practices.

Program officials also reported facing a variety of software development challenges while implementing these approaches. These included difficulties finding and hiring staff, transitioning from waterfall to Agile software development, and managing technical environments. DOD's continued efforts to address these challenges will be critical to the department's implementation of modern software development approaches.

DOD has also made organizational and policy changes intended to improve the management of its IT acquisitions, such as taking steps to implement Agile software development and improve data transparency. In addition, to address statutory requirements, DOD has taken steps to remove the department's chief management officer (CMO) position. However, the department had not yet sufficiently implemented these changes. Officials from many of the 18 programs GAO assessed that reported using Agile development reported that DOD had implemented activities associated with Agile transition best practices to only some or little to no extent, indicating that the department had not sufficiently implemented best practices. For example, 12 of the 18 programs reported that DOD's life-cycle activities only supported Agile methods to some or little to no extent. Program officials also reported challenges associated with implementing Agile software development. The department has a variety of efforts underway to help with its implementation of Agile software development. DOD officials stated that the department's transition to Agile will take years and will require sustained engagement throughout DOD.

In addition, DOD has taken steps aimed at improving the sharing and transparency of data it uses to monitor its acquisitions. According to a November 2020 proposal from the Office of the Under Secretary for Acquisition and Sustainment, DOD officials are to develop data strategies and metrics to assess performance for the department's acquisition pathways. However, as of February 2021, DOD did not have data strategies and had not finalized metrics for the two pathways associated with the programs discussed in this report. Officials said they were working with DOD programs and components to finalize initial pathway metrics. They stated that they plan to implement them in fiscal year 2021 and continue to refine and adjust them over the coming years. Without important data from acquistion pathways and systems, DOD risks not having timely quantitative insight into program performance, including its acquisition reform efforts.

Finally, DOD's CMO position was eliminated by a statute enacted in January 2021. This position was responsible for key efforts associated with the department's business systems modernization, which has been on GAO's High Risk List since 1995. DOD plans to take steps to address the uncertainty associated with the recent elimination of the position.

Why GAO Did This Study

For fiscal year 2021, DOD requested approximately $37.7 billion for IT investments. These investments included major business IT programs, which are intended to help the department carry out key business functions, such as financial management and health care.

The National Defense Authorization Act for Fiscal Year 2019 included a provision for GAO to assess selected IT programs annually through March 2023. GAO's objectives for this review were to (1) summarize DOD's reported performance of its portfolio of IT acquisition programs and the reasons for this performance; (2) evaluate DOD's assessments of program risks; (3) summarize DOD's approaches to software development and cybersecurity and identify associated challenges; and (4) evaluate how selected organizational and policy changes could affect IT acquisitions.

To address these objectives, GAO selected 29 major business IT programs that DOD reported to the federal IT Dashboard (a public website that includes information on the performance of major IT investments) as of September 2020. GAO reviewed planned expenditures for these programs, from fiscal years 2019 through 2022, as reported in the department's FY 2021 budget request. It also aggregated program office responses to a GAO questionnaire that requested information about cost and schedule changes that occurred since January 2019 and the early impacts of COVID-19.

GAO also analyzed the risks of the 22 programs that were actively using central repositories known as risk registers to manage program risks. GAO used these registers to create program risk ratings, and then compared its ratings to those of the DOD chief information officer (CIO).

In addition, GAO aggregated DOD program office responses to the questionnaire that requested information about the software and cybersecurity practices used by 22 of the 29 IT programs that were actively developing software. GAO compared the responses to relevant guidance and leading practices.

GAO reviewed selected IT-related organizational and policy changes and reviewed reports and documentation related to the effects of these changes on IT acquisitions. GAO also aggregated program office responses to the questionnaire that requested information about DOD's implementation of these changes. This included information on DOD's implementation of best practices as part of its efforts to implement Agile software development. GAO met with relevant DOD officials to discuss each of the topics addressed in this report.