DOD Major Automated Information Systems: Improvements Can Be Made in Reporting Critical Changes and Clarifying Leadership Responsibility

GAO-16-336 Published: Mar 30, 2016. Publicly Released: Mar 30, 2016.
Jump To:
Skip to Highlights
Highlights

What GAO Found

All 18 major automated information system (MAIS) programs that experienced a critical change to program cost, schedule, or system performance targets submitted complete reports to Congress that contained all four statutory elements, but 16 programs did not meet the requirement to report to Congress within 60 days of the program manager's submission to the senior Department of Defense (DOD) official that led to the critical change determination. Of the 16 critical change reports that exceeded the 60-days to report, 10 of the programs took over 100 days. Officials said that 60 days is too short to perform a program evaluation. Since the reports were not always timely, Congress may not have the necessary information when it is needed to make decisions. Finally, the DOD did not demonstrate that it had an internal control to ensure that MAIS programs not in compliance with reporting requirements were restricted from obligating funds on major contracts as required by law.

All three MAIS programs GAO selected to review experienced changes in their cost and schedule estimates, and one program did not fully meet its technical performance targets (see table).

Table: Extent of Changes in Cost and Schedule Estimates, and the Results of Technical Performance Targets (from the First Acquisition Program Baseline Estimate)

Program

Cost increase

Schedule change

Met technical performance targets?

Army

program

$368 million (19%)

3 months (full deployment)

Navy

program

$1.66 billion (477%)

13 years 9 months (full deployment)a

Air Force

program

$129 million (9%)

6 months (milestone C)

 

Source: GAO analysis of data provided by DOD officials. | GAO-16-336

a Delay was attributed to a major change in project scope and restructuring of the program.

The three selected programs implemented all seven IT acquisition best practices for risk management, and most of the best practices were implemented for requirements management: the Army and Navy implemented three of five best practices and the Air Force implemented four of five best practices. For example, the Army program did not adequately manage requirements changes and ensure that deliverables were in alignment with requirements. Until the programs fully implement best practices for requirements management, management of development efforts will likely be impaired.

As of October 2015, all appropriate programs were represented on the Federal IT Dashboard (Dashboard) as required by the Office of Management and Budget (OMB); however, the organization responsible for performance of MAIS programs was not provided. Specifically, DOD's Chief Information Officer is shown as the responsible party because OMB requires this, but the Under Secretary of Defense for Acquisition, Technology, and Logistics (AT&L) has overall responsibility for the MAIS programs. Therefore, users of the Dashboard are unaware that AT&L is the responsible organization and, thus, public accountability of the MAIS programs is decreased.

Why GAO Did This Study

The National Defense Authorization Act for Fiscal Year 2012 includes a provision for GAO to select, assess, and report on DOD MAIS programs annually through March 2018. MAIS programs are intended to help the department sustain its key operations. This report: (1) evaluates DOD's implementation of statutory reporting requirements for MAIS programs experiencing a critical change; (2) describes the extent to which selected MAIS programs have changed their planned cost and schedule estimates, and met performance targets; (3) assesses the extent to which selected MAIS programs have used key IT acquisition best practices, including requirements and risk management; and (4) determines the extent to which MAIS programs are represented on the Dashboard. GAO compared information on programs with a critical change to the reporting requirements. GAO selected three programs based on factors, such as representation from each military service (Air Force, Army, and Navy), identified changes to cost, schedule, and performance, and assessed them against selected best practices. GAO traced the programs to the Dashboard and reviewed relevant processes.

Skip to Recommendations

Recommendations

GAO recommends, among others, that DOD examine the critical change reporting process and implement corrections for the reports' timeliness, and address weaknesses with requirements management, and add AT&L as a responsible organization for MAIS programs to the Dashboard. DOD concurred with all recommendations. OMB did not concur but GAO continues to believe that improved transparency is needed.

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of the Army To help improve the management of MAIS programs, the Secretary of the Army should direct the Tactical Mission Command program manager to develop a requirements management plan to document and manage its requirements process.
Closed – Not Implemented
The Department of Defense has provided documentation that the Army's Tactical Mission Command program has been formally discontinued. Action on this recommendation is no longer required.
Department of the Navy To help improve the management of MAIS programs, the Secretary of the Navy should direct the Common Aviation Command and Control System program manager to identify weaknesses in the requirements traceability process and take corrective actions to manage the traceability of requirements to the respective lower-level requirements, and periodically evaluate work products, including the requirements management plan, and update them in accordance with the requirements guidance.
Closed – Not Implemented
DOD concurred with this recommendation and stated in March 2016 that the Navy had corrected the data query issue that caused 11 requirements to be eliminated from the traceability matrix we reviewed. DOD also stated that the Navy had identified the weakness in the traceability process that led to 14 general requirements not being fully traced. However, as of June 2020, DOD had not provided documentation that supports that it identified the weakness in the requirements traceability process. In addition, It did not demonstrate that the program office had updated its requirements management guidance to address the weakness it identified. In May 2022, DOD informed GAO that the Common Aviation Command and Control System reached full deployment in December 2020 and is now in the sustainment phase. DOD also provided supporting documentation to validate this change in program status. GAO is closing the recommendation as not implemented because (1) the program did not demonstrate that it has addressed the recommendation, and (2) the recommendation is no longer applicable due to DOD having fully deployed the system.
Department of the Air Force To help improve the management of MAIS programs, the Secretary of the Air Force should direct the Defense Enterprise Accounting and Management System program manager to address weaknesses in its controls for ensuring that all software requirements are tested and validated before deployment of new software releases.
Closed – Implemented
The Air Force demonstrated that it has addressed the recommendation. Specifically, in February 2018, the Air Force developed a Developmental Test and Evaluation Integrated Test Plan for DEAMS Increment 1, which includes validating that all desired capabilities function properly and meet specific functional and technical requirements. The test plan includes employing a tool to document test script traceability to system requirements and design, and to track the execution of test scripts for functional and regression testing.
Office of Management and Budget To help improve the management of MAIS programs, the Director of OMB should instruct the Federal Chief Information Officer (CIO) to add the Under Secretary of Defense for AT&L as a responsible party to DOD's MAIS entries on the Federal IT Dashboard website, alongside the CIO, to publicly disclose the responsible party for the acquisition performance management of MAIS programs.
Closed – Not Implemented
The Office of Management and Budget (OMB) has not implemented the recommendation or identified plans to do so. Specifically, in 2016, in response to our draft report, OMB stated that it did not agree with our recommendation. In July 2020, OMB stated that it believed the IT Dashboard appropriately identified the DOD Chief Information Officer (CIO) as the principal staff assistant to the Secretary of Defense for Information Technology (IT), including national security systems and defense business systems, along with information resources management. We continue to maintain that the official with responsibility to the Secretary of Defense on matters related to acquisition, currently the Under Secretary of Defense of Acquisition and Sustainment, should also be identified on the IT Dashboard. Given that OMB has no plans to implement this recommendation, we are closing the recommendation as not implemented.
Department of Defense To help improve the management of MAIS programs, the Secretary of Defense should examine the MAIS critical change reporting process to identify root causes for delays and implement corrective actions for the timely delivery of critical change reports.
Closed – Not Implemented
The National Defense Authorization Act for Fiscal Year 2017 repealed the statutory requirement that DOD issue critical change reports to the Congress. Action on this recommendation is no longer required. Moreover, DOD has no plan to implement this recommendation
Department of Defense To help improve the management of MAIS programs, the Secretary of Defense should develop a mechanism for monitoring whether MAIS programs with late reports are restricted from obligating funds and in turn ensuring compliance with the Antideficiency Act.
Closed – Not Implemented
The National Defense Authorization Act for Fiscal Year 2017 repealed the statutory requirement that DOD issue critical change reports to the Congress. Action on this recommendation is no longer required. Moreover, DOD has no plan to implement this recommendation.

Full Report

GAO Contacts