Best Practices:

Successful Application to Weapon Acquisitions Requires Changes in DOD's Environment

NSIAD-98-56: Published: Feb 24, 1998. Publicly Released: Feb 24, 1998.

Contact:

Jack L. Brock, Jr
(202) 512-4841
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Pursuant to a congressional request, GAO reviewed whether commercial practices offer ways to improve the Department of Defense's (DOD) process for transitioning weapons from development to production, focusing on: (1) a comparison of DOD's practices for preparing a weapon system for production with best commercial practices; (2) whether differences in commercial and DOD environments for developing new products affect practices; and (3) environmental changes that are key to the success of DOD initiatives for improving the transition of weapons from development to production.

GAO noted that: (1) commercial firms gained more knowledge about a product's technology, performance, and producibility much earlier in the product development process than DOD; (2) product development in commercial ventures was a clearly defined undertaking for which firms insisted on having the technology in hand to meet customer requirements before starting; (3) once underway, these firms demanded--and got--specific knowledge about a new product before production began; (4) the process of discovery--the accumulation of knowledge and the elements of unknowns--was completed for the best commercial programs well ahead of production; (5) in contrast, DOD programs allowed more technology development to continue into product development; (6) consequently, the programs proceeded with much less knowledge--and thus more risk--about required technologies, design capability, and producibility; (7) although DOD accepted more unknowns on its programs than commercial firms, it understated the risks present; (8) the commercial and defense environments created different incentives and elicited different behaviors from the people managing the programs; (9) specific practices took root and were sustained because they helped a program succeed in its environment--not because they were textbook solutions; (10) the success of commercial product developments was determined when production items were sold; (11) DOD programs began without needed technology in hand; rather, they were encouraged to include undeveloped technology; (12) because these programs ran much longer, production concerns did not play as big a role and were not as critical to success in the early stages; (13) the definition of success was more complicated in DOD; (14) during most of product development, success was defined as getting DOD and Congress as the customer, rather than the firm, to fund the development annually; (15) commercial practices for gaining knowledge and assessing risks can help produce better outcomes on DOD acquisitions; (16) indeed, DOD has several commercial-like initiatives under way; (17) at least two factors are critical to fostering an environment conducive to such practices: (a) program launch decisions must be relieved of the need to overpromise on technical performance and resource estimates; and (b) once a program is under way, it must become acceptable for program managers to identify unknowns as high risks so that they can be aggressively worked on earlier in development; and (18) currently, identifying a high risk on a DOD program is perceived as inviting criticism and the loss of funding.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Matters for Congressional Consideration

    Matter: If DOD takes steps to manage technology development efforts outside the bounds of individual weapon system programs, Congress may wish to consider providing the funds needed for such efforts.

    Status: Closed - Implemented

    Comments: In this case, the required action would be both continuous and dependent on what DOD does. In the past 2 years, DOD has not taken significant steps to manage more technology development outside the bounds of individual weapon system programs. In fact, the funds budgeted for such technology activities have declined in real terms over the last 2 years.

    Matter: Because of its critical role in creating the environment for what constitutes program success and which practices will work, Congress may wish to consider supporting the Secretary of Defense's efforts to create such an environment through changes to the acquisition process that provide program managers clear incentives for gaining sufficient knowledge at key points in weapon acquisition programs. The best commercial practices described in this report suggest what may constitute sufficient levels of knowledge.

    Status: Closed - Implemented

    Comments: Although the required action is continuous and could require the results of several congressional actions on programs, Congress' actions during the past 2 years are indicative of its response to the recommendation. Defense and oversight committees have held hearings that focused on the maturity of technology in programs that are early in development, which is the first of three knowledge points that GAO has reported on. Such hearings do create incentives for program managers to know more about key technologies early in their programs. However, these hearings have not yet altered the decisions on whether to fund or otherwise approve individual programs.

    Matter: Congress could help create the right incentives on individual programs by favorably considering DOD funding requests to mitigate high risks early in a program and cautiously considering late requests for funds to resolve problems that should have been addressed earlier.

    Status: Closed - Implemented

    Comments: In this case, the required action would be both continuous and dependent on what DOD does. It will take at least another year to see if DOD takes an action as recommended above to see Congress' response to the action.

    Recommendations for Executive Action

    Recommendation: On individual program decisions, the Secretary of Defense or his designee should send the signals that create incentives for acquisition managers to identify unknowns and ameliorate their risks early in development. Incentives could take the form of a decision to fully fund one program's efforts to mitigate a high risk identified early or requiring another program in which risks are revealed late to absorb the associated financial consequences.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: DOD completed action on this recommendation on May 6, 1998. It pointed to its year 2000 acquisition goals for reducing program cycle time and obviating program cost growth as action taken, but did not specify how this has been or would be implemented.

    Recommendation: To make the environment for DOD product developments more conducive to the techniques used by commercial firms, the Secretary of Defense should redefine the point for launching programs as the point at which technology development ends and product development begins. This recommendation is made without prejudice toward the necessity of technology development but rather with the intent that programs could be better managed if such development was conducted outside their purview. Thus, concomitant with defining the launch point later in the acquisition cycle must be the willingness of decisionmakers in DOD and Congress to support research and development efforts that will advance technology and establish the basis for determining which technologies and subsystems have the mettle to meet the performance, production, and precision estimating demands of product development. If extenuating circumstances necessitate including technology development in a program, this should be recognized as a departure from sound practices, accorded a high risk, and funded and paced accordingly.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: In its formal response, DOD stated that its engineering and manufacturing development phase is the Department's "Product Launch" equivalent. It also stated that it has taken steps to separate technology development from product development, but did not specify what steps have been taken. Finally, DOD stated that it was reviewing appropriate decision points for major acquisitions, but did not specify the results of those reviews or actions taken to reduce risk from technology during a major acquisition program. Since that response, DOD rewrote its 5000 series policies that guide the management of weapon system acquisitions. The policies redefine milestone decision points for major weapons that are consistent with moving the launch point for new programs to the point at which key technologies are mature.

    Recommendation: To close the gap between policy and practice, the Secretary of Defense should take steps to ensure that sound standards for the timing and quality of production-related knowledge are applied to individual weapon system programs and used as a basis for assessing production risks and for making tradeoffs. These standards, which can already be found to some extent in existing DOD guidance, should draw from commercial practices and could include the release of engineering drawings, identification of key production processes, demonstration of risky or new production processes, and achievement of statistical process control. Such standards could enable manufacturing representatives on integrated product teams to flag something as high risk if it would delay drawing release and the achievement of statistical process control beyond the standard of acceptability. Identifying the impact of such deficits in production knowledge could help program managers to say "no" to proposals to accept unknowns and could force tradeoffs in the design.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: DOD completed action on the recommendation on May 6, 1998. DOD pointed to the initiation of an Enterprise Metrics Program that led to establishing year 2000 goals for reducing program cycle-time and obviating program cost growth. It also pointed to numerous production and manufacturing courses available that outline key attributes that the program manager should use in assessing the extent of the program's effectiveness in designing for producibility. Finally, it has included a copy of this GAO report on its automated Acquisition Deskbook for use as guidance by program managers. These actions show movement toward improving practices, but by themselves do not represent steps that ensure that sound standards for the timing and quality of production-related knowledge are applied to weapon system programs and used as a basis for assessing risk and making tradeoffs.

    Jul 17, 2014

    Jul 14, 2014

    Jul 11, 2014

    Jul 9, 2014

    Jul 8, 2014

    Jun 30, 2014

    Jun 26, 2014

    Looking for more? Browse all our products here