What GAO Found
Of the 15 selected Department of Defense (DOD) major automated information system (MAIS) programs, 13 had cost information available (2 did not, due to revisions to requirements and changes in scope). Of these 13 programs, 11 experienced changes in their cost estimates, including 7 that experienced increases ranging from 4 to 2,233 percent and 4 that experienced decreases ranging from 4 to 86 percent. Two programs remained unchanged in their cost goals. Additionally, of 14 programs that had schedule information available (1 did not due to revisions to requirements), 13 experienced schedule changes—including 12 that had slippages ranging from a few months to 6 years, and 1 that accelerated its schedule. One program remained on schedule. Further, of 11 programs that had system performance data available, 3 programs met their system performance targets, while 8 did not fully meet their targets.
The three programs selected for analysis of risk management demonstrated mixed progress in effectively defining and managing risks. Specifically, the Defense Health Agency's (DHA) Theater Medical Information Program – Joint Increment 2 had implemented key risk management practices. While the Navy's Global Combat Support System – Marine Corps program did not, among other things, update its risk tracking log during a 5-month period in 2013, the program recently updated its risks and mitigation plans, which should help the program to more effectively manage risks going forward. The Defense Logistics Agency's (DLA) Defense Agencies Initiative program had taken steps to implement selected risk management practices, including establishing a risk management board. However, the program was still in the early stages of identifying risks and had not yet identified a comprehensive set of program risks, nor consistently evaluated and categorized its risks. Until this program maintains a complete risk log and mitigation plans, and accurately evaluates and categorizes its risks, it will lack assurance that it is appropriately mitigating all identified risks.
The three programs also demonstrated mixed progress in implementing key requirements management and project monitoring and control best practices. Specifically, the Navy program had implemented all key requirements management best practices and the DLA program had recently taken steps to do so. However, while the DHA program had implemented many requirements management best practices, it had not maintained complete traceability between its requirements and work products. Additionally, the DHA program had not updated its capabilities baseline to reflect program scope changes. Until DHA implements these requirements management best practices, stakeholders will lack assurance that the system will have all intended functionality to meet users' needs. Regarding project monitoring and control, each of the programs lacked key practices. For instance, the DLA program had not tracked significant deviations in performance. Additionally, the Navy program did not always take timely corrective actions to address issues. Further, the DHA program did not use earned value management to track contractor performance in meeting planned cost targets, even though these data were being collected and certain contracts met DOD's threshold for its use; as such, the program had not effectively determined progress against the plan. Until the three programs implement these project monitoring best practices, they will be limited in their ability to manage the programs.
Why GAO Did This Study
The National Defense Authorization Act for Fiscal Year 2012 mandated that GAO select and assess DOD MAIS programs annually through March 2018. This report discusses the results of GAO's second annual assessment. Based on the act's requirements, GAO's objectives were to (1) describe the extent to which selected MAIS programs have changed their planned cost and schedule estimates and met performance targets; (2) assess selected MAIS programs' actions to manage risks; and (3) assess the extent to which selected MAIS programs used key information technology acquisition best practices.
To do so, GAO selected 15 of the 42 DOD MAIS programs based on several factors, including representation from multiple DOD components, and summarized the results of analyses of cost, schedule, and performance across the programs. Further, GAO selected 3 of the 15 programs (1 each from DHA, DLA, and Navy) and assessed them against best practices for risk management, requirements management, and project monitoring and control.
GAO recommends that DOD direct the programs to address respective weaknesses in their risk management, requirements management, and project monitoring and control practices. DOD concurred with six of GAO's recommendations and partially concurred with the remaining two. GAO maintains that it is important that the DHA program trace all capabilities to their associated requirements and update its capabilities baseline to reflect program scope changes.
Recommendations for Executive Action
|Department of Defense||To better ensure that the Defense Agencies Initiative (DAI) implements effective risk management and information technology (IT) acquisition best practices, the Secretary of Defense should direct the Director of the Defense Logistics Agency to direct the DAI program office to establish a comprehensive risk log that includes all up-to-date risks with evaluations and categorizations that comply with DLA's defined parameters; and associated mitigation plans.|
|Department of Defense||To better ensure that DAI implements effective risk management and IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Logistics Agency to direct the DAI program office to identify and document significant cost and schedule deviations from the program's plan.|
|Department of Defense||To help ensure that the Global Combat Support System-Marine Corps (GCSS-MC) implements effective project monitoring and control best practices, the Secretary of Defense should direct the Secretary of the Navy to direct the GCSS-MC program office to include corrective actions and time frames in future analyses of critical issues and monitor actions taken against those time frames.|
|Department of Defense||To improve the Theater Medical Information Program - Joint Increment 2 (TMIP-J) program's implementation of IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Health Agency to direct the TMIP-J program office to update the program's capabilities baseline to reflect program scope changes.|
|Department of Defense||To improve the TMIP-J program's implementation of IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Health Agency to trace all capabilities to their associated requirements in the requirements traceability matrix.|
|Department of Defense||To improve the TMIP-J program's implementation of IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Health Agency to implement earned value management in accordance with DOD's policy and train management staff with project oversight responsibilities on the proper use of earned value management.|
|Department of Defense||To improve the TMIP-J program's implementation of IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Health Agency to develop an authoritative listing of all interfaces currently included in the scope of the program.|
|Department of Defense||To improve the TMIP-J program's implementation of IT acquisition best practices, the Secretary of Defense should direct the Director of the Defense Health Agency to report to stakeholders the number of current and planned system users and sites and provide updates as needed.|