Information Technology:

DOD Needs to Strengthen Management of Its Statutorily Mandated Software and System Process Improvement Efforts

GAO-09-888: Published: Sep 8, 2009. Publicly Released: Sep 8, 2009.

Additional Materials:

Contact:

Randolph C. Hite
(202) 512-6256
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Department of Defense's (DOD) acquisition of weapon systems and modernization of business systems have both been on GAO's list of high-risk areas since 1995. To assist DOD in managing software-intensive systems, Section 804 of the Bob Stump National Defense Authorization Act for Fiscal Year 2003 required the Office of the Secretary of Defense (OSD) and DOD component organizations, including the military departments, to undertake certain software and systems process improvement (SSPI) actions. As requested, GAO assessed (1) the extent to which DOD has implemented the process improvement provisions of the act, and (2) the impact of DOD's process improvement efforts. To do so, GAO analyzed relevant plans, policies, guidance, performance measures, and reports against statutory requirements and relevant guidance, and interviewed DOD officials.

OSD and the military departments have implemented a number of statutory requirements aimed at improving their processes for acquiring software-intensive systems. However, they have not satisfied all of their respective statutory requirements, or key aspects of relevant SSPI guidance. In particular, OSD has issued guidance calling for military departments and defense agencies to implement process improvement programs, revised guidance to emphasize contractor past performance in source selection decisions, and established a clearinghouse for software and system acquisition and development best practices, all of which are required by the statute. However, it has not implemented a requirement in the statute related to overseeing DOD component organization process improvement programs to ensure compliance with its guidance, and it has not satisfied a key aspect of relevant guidance pertaining to monitoring organizationwide process improvement efforts. According to OSD, process improvement is a component responsibility and thus it does not view oversight of component SSPI efforts as necessary. Without strong, central leadership over DOD's improvement efforts, OSD is not fulfilling key tenets of section 804 and relevant guidance associated with well-managed software process improvement programs, and has increased the risk that component process improvement efforts and their impacts are not being maximized. The military departments have established process improvement programs, although two did not do so within the time frame specified in the statute. Also, each has documented processes that address the four key software process areas cited in the statute, and have taken steps to ensure that key personnel have the appropriate level of software/system-related experience or training, and to develop process improvement performance metrics, as required by the statute. However, none is using these performance metrics for continuous process improvement, as provided for in the statute and relevant guidance. Also, while each has a process governing implementation of key acquisition requirements, these processes do not fully reflect the range of verification steps advocated in relevant guidance. Reasons cited for the state of the department's respective efforts include senior leadership turnover and not viewing all the statutory requirements as necessary. By not having fully implemented the statute and relevant guidance, the military departments are not positioned to maximize the potential of their process improvement efforts. Neither OSD nor the military departments have measured the impact of their collective or separate process improvement efforts. However, studies by GAO and others continue to identify system and software acquisition and development process weaknesses, as well as cost, schedule, and performance shortfalls, across a range of DOD software-intensive programs, thus suggesting that the potential value of these efforts has yet to be fully realized.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: The Department of Defense (DOD) reported that it has taken steps to improve aspects of its software and system process improvement efforts. For example, it has initiated its Business Capability Lifecycle framework, which is intended to rapidly deliver information technology capabilities by consolidating various oversight requirements and streamlining documentation requirements. The department also reported that it has established an information technology acquisition task force, which includes participation from across the Office of the Secretary of Defense and the military departments, and which is to engage with Congress, GAO, and stakeholders throughout the department and industry to further define and implement updated approaches to acquiring information technology systems. In addition, the department reported that, among other things, the Air Force has modified its Air Force Product Data Acquisition tool to emphasize software and include software guidance, lessons learned, template language for solicitations, and recommended Data Item Descriptions for software documentation and software deliverables. DOD also reported that, among other things, the Department of the Navy Space and Naval Warfare Systems Command has taken actions such as reviewing its systems engineering processes to reflect standards and best practices and to improve documented guidance. However, while these are important steps towards addressing our recommendation, DOD has yet to jointly develop a DOD-wide strategic plan, and supporting organizational component plans, for ensuring that all of the requirements in section 804 of the 2003 National Defense Authorization Act, as well as relevant guidance, are fully implemented.

    Recommendation: To strengthen DOD's management of its SSPI efforts, the Secretary of Defense should direct the ASD(NII)/CIO and the USD(AT&L), in concert with the military departments and defense agencies, to jointly develop a DOD-wide strategic plan, and supporting organizational component plans, for ensuring that all of the requirements in section 804 of the 2003 NDAA, as well as relevant SEI guidance, are fully implemented.

    Agency Affected: Department of Defense

  2. Status: Closed - Not Implemented

    Comments: The Department of Defense (DOD) has issued reports to Congress that discuss certain steps it has taken to improve aspects of its software and system process improvement efforts. For example, in its November 2010 report to Congress on identifying a new approach for delivering information technology capabilities, the department reported that it has established an information technology acquisition task force, which includes participation from across the Office of the Secretary of Defense and the military departments, that is to engage with Congress, GAO, and stakeholders throughout the department and industry to further define and implement updated approaches to acquiring information technology systems. In addition, in its March 2012 annual report on Developmental Test and Evaluation and Systems Engineering, the department reported, among other things, that the Air Force has modified its Air Force Product Data Acquisition tool to emphasize software and include software guidance, lessons learned, template language for solicitations, and recommended Data Item Descriptions for software documentation and software deliverables. However, without a DOD-wide strategic plan, and supporting organizational component plans, for ensuring that all of the requirements in section 804 of the 2003 National Defense Authorization Act, as well as relevant guidance, are fully implemented, DOD has not been able to report to the congressional defense committees on its progress in implementing such a plan and the impacts of doing so.

    Recommendation: To strengthen DOD's management of its SSPI efforts, the Secretary of Defense should direct the ASD(NII)/CIO and the USD(AT&L), in concert with the military departments and defense agencies, to jointly and periodically report to the congressional defense committees on their progress in implementing the plan and the impacts of doing so.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Sep 25, 2014

Sep 23, 2014

Jun 10, 2014

May 22, 2014

May 12, 2014

May 8, 2014

May 7, 2014

Apr 2, 2014

Looking for more? Browse all our products here