Information Technology: DOD's Acquisition Policies and Guidance Need to Incorporate Additional Best Practices and Controls

GAO-04-722 Published: Jul 30, 2004. Publicly Released: Jul 30, 2004.
Jump To:
Skip to Highlights
Highlights

The way in which the Department of Defense (DOD) has historically acquired its business systems has been cited as a root cause for its limited success in delivering promised system capabilities and benefits on time and within budget. In response, DOD recently revised its systems acquisition policies and guidance to incorporate best practices, including those pertaining to business systems. GAO was asked to determine whether DOD's revised systems acquisition policies and guidance (1) are consistent with industry best practices, including those pertaining to commercial component-based systems, and (2) provide the necessary controls to ensure that DOD component organizations adhere to the practices.

Skip to Recommendations

Recommendations

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Defense To improve DOD's ability to acquire business systems, the Secretary of Defense should direct the Under Secretary for Acquisition, Technology, and Logistics, in collaboration with the Assistant Secretary for Networks and Information Integration and the Director, Operational Test and Evaluation, to develop and implement an explicit plan for incorporating into the 5000 series the best practices and associated activities currently missing from the series. The plan should specify tasks to be performed, resources needed and assigned, and milestones for completing tasks.
Closed – Implemented
In response to our report, DOD agreed that incorporation of additional best practices in its acquisition policies and guidance should be undertaken or considered. Further, DOD has updated the acquisition guidebook to incorporate most of the missing practices that we identified in our report and the following 12 recommendations address. DOD's actions are generally consistent with our recommendation.
Department of Defense The progress against this plan should be tracked and reported as appropriate, and the plan, at a minimum, should incorporate best practices activities. Product line requirements--rather than just the requirements for the system being acquired--are an explicit consideration in each acquisition.
Closed – Implemented
DOD has developed an Enterprise Integration tool kit that endorses and supports the best practice of reusing reports, interfaces, and conversions that have been built or acquired by other programs. According to the Software Engineering Institute, system reuse is a key part of the product line requirements best practice. Under DOD's planned approach to consider product line requirements in acquisition, Enterprise Integration toolkit reuse covers such items as software modules or components, which developers are encouraged to use. DOD's actions are largely consistent with our recommendation.
Department of Defense The progress against this plan should be tracked and reported as appropriate, and the plan, at a minimum, should incorporate best practices activities. Acquisition project management activities are communicated to all stakeholders.
Closed – Implemented
DOD agreed that communication of acquisition management activities to all stakeholders is a best practice. DOD 5000 acquisition policies and procedures and related guidance now provide for program managers and acquisition officials to conduct acquisition oversight through integrated product teams as a way to communicate information. These policies and procedures are largely consistent with our recommendation.
Department of Defense The progress against this plan should be tracked and reported as appropriate, and the plan, at a minimum, should incorporate best practices activities. Acquisition reviews include the status of identified risks.
Closed – Implemented
DOD has updated the DOD 5000 acquisition policies and procedures and related guidance to include acquisition risk management, including references to a risk management guide that provides for identifying the status of risks at each phase and milestone of the acquisition process. Further, it describes what is to be done to manage and mitigate the risks. This is largely consistent with our recommendation.
Department of Defense The progress against this plan should be tracked and reported as appropriate, and the plan, at a minimum, should incorporate best practices activities. Modification of commercial components is discouraged and allowed only if justified by a thorough analysis of life-cycle costs and benefits.
Closed – Implemented
DOD has added this best practice activity to its DOD 5000 Acquisition guidance. In the guidebook, DOD added a section entitled, "Modifying Commercial off-the-Shelf Software (COTS)" which discusses ways to avoid modifying the COTS product. For example, the guidebook notes that modifying the core code of a COTS product should be avoided. Further, it adds that the business processes inherent in the COTS product should be adopted, not adapted, by the organization implementing the product. DOD not only discouraged the modification of commercial components but also noted that should the product need to be modified, a business process re-engineering approach should be considered. This is largely consistent with our recommendation.
Department of Defense The progress against this plan should be tracked and reported as appropriate, and the plan, at a minimum, should incorporate best practices activities. Modification or upgrades to deployed versions of system components are centrally controlled, and unilateral user release changes are precluded.
Closed – Implemented
DOD agreed that a best practice to modify deployed versions of system components should be incorporated. Accordingly, DOD's 5000 acquisition guidance cites the application of configuration management and incorporates configuration change controls. However, the guidance does not address whether the modifications are centrally controlled and whether unilateral user release changes are precluded.
Department of Defense The progress against this plan should be tracked and reported as appropriate, and the plan, at a minimum, should incorporate best practices activities. Acquisition decisions about commercial components are based on deliberate and thorough research, analysis, and evaluation of the components' interdependencies.
Closed – Implemented
According to DOD, acquisition decisions about commercial components based on research, analysis, and evaluation of components interdependencies is considered in the Joint Capabilities Integration and Development System (JCIDS) process and is to be addressed in the Information Support Plan (ISP). Further, DOD has incorporated a section on the ISP in the DOD 5000 acquisition guidebook. Specifically, the guidebook notes that the ISP is to identify and resolve implementation issues concerning acquisition programs, as well as provide managers with a method of identifying information-related dependencies. This is largely consistent with our recommendation.
Department of Defense The progress against this plan should be tracked and reported as appropriate, and the plan, at a minimum, should incorporate best practices activities. Acquisition plans provide for preparing users for the impact that the business processes embedded in the commercial components will have on their respective roles and responsibilities.
Closed – Implemented
DOD agreed that change management activities, such as those described in this recommendation, are best practices and it points to its Enterprise Integration tool kit as containing a change management roadmap that addresses organizational change, readiness, and preparing users for changes to their roles and responsibilities. This tool kit provides relevant change management information for the technology development phase of the acquisition process. As such, it is largely consistent with our recommendation.
Department of Defense The progress against this plan should be tracked and reported as appropriate, and the plan, at a minimum, should incorporate best practices activities. Changes affecting how users will be expected to use the system to execute their jobs are actively managed.
Closed – Implemented
DOD agreed that change management activities, such as those described in this recommendation, are best practices and it points to its Enterprise Integration tool kit as containing a change management roadmap that addresses organizational change, readiness, and preparing users for using the system to execute their jobs. This tool kit provides relevant change management information for the technology development phase of the acquisition process. As such, it is largely consistent with our recommendation.
Department of Defense The progress against this plan should be tracked and reported as appropriate, and that the plan, at a minimum, should incorporate systems best practices activities. Systems integration contractors are explicitly evaluated on their ability to implement commercial components.
Closed – Implemented
DOD 5000 acquisition policies and procedures and related guidance now cite and reference the use of information on contractors' past performance, which can include the ability to implement commercial components. This is largely consistent with our recommendation.
Department of Defense The progress against this plan should be tracked and reported as appropriate, and the plan, at a minimum, should incorporate best practices activities. Investment decisions throughout a system's life cycle are based on a continuous set of tradeoffs among capabilities available in commercial components (current and future), the architectural environment in which the system is to operate, defined system requirements, and existing cost/schedule constraints.
Closed – Implemented
DOD has incorporated a section on systems engineering in its DOD 5000 acquisition guidebook that discusses tradeoffs throughout a system's life cycle. In addition, the Joint Capabilities Integration and Development System require that three analyses be conducted before a capability is acceptable to enter the technology development phase of the acquisition process. These analyses are to be used at investment decision points in making tradeoffs among various available capabilities (including to those available in commercial components) relative to the architectural environment in which the system is to operate, defined system requirements, and existing cost/schedule constraints.
Department of Defense The progress against this plan should be tracked and reported as appropriate, and the plan, at a minimum, should incorporate best practices activities. Evaluation criteria are established for selecting among commercial component options that include both defined system requirements and vendor/commercial product characteristics.
Closed – Not Implemented
DOD stated that no changes were necessary to the DOD 5000 policies or guidebook because they call for a capability development document that describes system requirements. However, they do not cite the need to consider vendor/commercial product characteristics.
Department of Defense To ensure that the best practices provided for in DOD acquisition policies and guidance are appropriately followed, the plan should incorporate steps to include in DOD's acquisition policies a provision for measurement and verification of best practices.
Closed – Not Implemented
According to DOD, its existing oversight process includes the necessary compliance activities because statutory and regulatory compliance are referenced in DOD's integrated process team and milestone decision point process components of its DOD 5000 acquisition policies and guidance. However, they do not require measurement and verification. Rather, it makes such reviews discretionary.

Full Report