DOD has faced challenges acquiring new weapon systems—such as aircraft, ships, and satellites—on time and on budget.
In 2018, DOD revised its process for reviewing and approving "capability documents," which identify new or enhanced capabilities in weapon systems.
But DOD's information system for tracking these documents doesn't have reliable data about how many programs have gone through the revised process, or how long it takes to review documents.
We recommended ensuring this system collects data that could help DOD evaluate its capability document process.
What GAO Found
The Department of Defense (DOD) sets the foundation of its weapon system acquisitions in documented requirements for new or enhanced capabilities. DOD's Joint Staff uses the Joint Capabilities Integration and Development System (JCIDS) process to manage the review and approval of capability requirements documents. The Joint Requirements Oversight Council (JROC) oversees the process. At congressional direction, the Joint Staff revised the process in November 2018, reducing the JROC's role to focus on documents addressing requirements of multiple departments, while increasing the role of military departments for their unique capability documents.
GAO found that the Joint Staff lacks reliable data on the total number of programs that have completed the revised process. In addition, GAO found that Joint Staff data for the time to validate selected capability documents were also unreliable. Capability documents move through the JCIDS process in the Joint Staff's Knowledge Management and Decision Support (KM/DS) information system. GAO found discrepancies between KM/DS data and data from those that submit documents, known as sponsors. Joint Staff officials stated that deficiencies with the KM/DS system are at the root of its data issues. A detailed plan addressing these deficiencies will better position the Joint Staff to assess if the revised process is achieving stated JCIDS objectives. See figure below.
The Joint Staff cannot assess the JCIDS process because it lacks reliable data and a baseline to measure timeliness. Joint Staff guidance provides a notional length of time of 103 days to review documents in the JCIDS process, but this is not evidence-based. Joint Staff officials stated they have not measured the actual length of time that documents take to go through the JCIDS process. GAO analysis and sponsor officials confirmed that none of the selected capability documents completed the process within 103 days. Sponsor officials noted that certain issues can add time to the review process and emphasized document quality over fast review and approval. However, without a data-driven baseline that reflects issues that affect the length of time to validate capability documents, Joint Staff officials are not able to assess JCIDS' efficiency and effectiveness.
Discrepancies between Joint Staff and Sponsor Validation Timeline Data
Note: One selected program is not included in the figure because the sponsor withdrew it from the process.
Why GAO Did This Study
In the National Defense Authorization Act for Fiscal Year 2017, Congress mandated revisions to the JCIDS process by modifying the scope of the JROC's responsibilities. The accompanying Senate Armed Services Committee report noted that these changes were, in part, to improve the timeliness of the JCIDS process.
House Armed Services Committee report 116-120 included a provision for GAO to review the revisions to the JCIDS process. This report examines (1) key revisions to the process, (2) how many programs have been through the revised process and how long it took, and (3) the Joint Staff's ability to assess the timeliness of the process.
GAO reviewed JCIDS policies and guidance, and interviewed relevant DOD officials. GAO also selected a nongeneralizable sample of 12 capability documents from across the Air Force, Army, and Navy. GAO analyzed data associated with these documents from the Joint Staff's KM/DS information system and compared it to data provided by military department officials to determine the Joint Staff's ability to assess the timeliness of the document review process.
GAO recommends that DOD (1) develop a plan for resolving the JCIDS information system operational deficiencies, and (2) establish a baseline based on reliable data and issues that affect the length of time to validate capability documents. DOD concurred with the recommendations.
Recommendations for Executive Action
|Department of Defense||The Secretary of Defense should ensure that the Vice Chairman of the Joint Chiefs of Staff develop a plan for resolving the deficiencies of the information system that facilitates the operation of the Joint Capabilities Integration and Development System (JCIDS), to include updating guidance for (a) assessing the system's effectiveness in providing reliable system data, (b) accounting for programs that have entered and exited the process, and (c) collecting and analyzing JCIDS' performance and timeliness metrics. (Recommendation 1)|
|Department of Defense||After resolving the information system data reliability issues for the Joint Capabilities Integration and Development System (JCIDS), the Secretary of Defense should ensure that the Vice Chairman of the Joint Chiefs of Staff establish a baseline based on reliable data and relevant issues for determining a length time to validate capability documents and a process for using that baseline to assess JCIDS' efficiency and effectiveness in meeting management objectives and making improvements. (Recommendation 2)|