Best Practices:

Better Management of Technology Development Can Improve Weapon System Outcomes

NSIAD-99-162: Published: Jul 30, 1999. Publicly Released: Jul 30, 1999.

Contact:

Paul L. Francis
(202) 512-4841
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Pursuant to a congressional request, GAO assessed how best practices offer improvements to the way the Department of Defense (DOD) incorporates new technology into weapon system programs, focusing on: (1) the impact of technology maturity on product outcomes; (2) best practices for managing new technologies and incorporating them into products; and (3) ways DOD can adapt these practices to get better outcomes on weapon system programs.

GAO noted that: (1) the experiences of DOD and commercial technology development cases GAO reviewed indicate that demonstrating a high level of maturity before new technologies are incorporated into product development programs puts those programs in a better position to succeed; (2) the technology readiness levels (TRL), as applied to the 23 technologies, reconciled the different maturity levels with subsequent product development experiences; (3) they also revealed when gaps occurred between a technology's maturity and the intended product's requirements; (4) for technologies that were successfully incorporated into a product, the gap was recognized and closed before product development began, improving the chances for successful cost and schedule outcomes; (5) the closing of the gap was a managed result; (6) it is a rare program that can proceed with a gap between product requirements and the maturity of key technologies and still be delivered on time and within costs; (7) two conditions were critical to closing the maturity gap; (8) one condition is that the right environment for maturing technologies exists; (9) key to this environment was making a science and technology organization, rather than the program or product development manager, responsible for maturing technologies to a high TRL; (10) when a maturity gap persisted, managers were given the flexibility to take the time to mature the technology or decrease product requirements so that they could use another, already mature technology; (11) both technology and product managers were supported with the disciplined processes, readily available information, readiness standards, and authority to ensure technology was ready for products; (12) this support enabled these managers to safeguard product development from undue technology risks; (13) on the other hand, immature technologies were sometimes incorporated into products for reasons such as inflexible performance requirements, increasing the likelihood of cost overruns and delays in product development; (14) product managers had little choice but to accept the technologies and hope that they would mature successfully; (15) however, the pressures of product development made for an environment less conducive to maturing technology; (16) DOD is likely to move technologies to product development programs before they are mature; and (17) the challenge will be whether the lessons learned from cases and initiatives offer an approach that has a DOD-wide application.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To help guard against the possibility that the more basic research and technology development activities would be compromised by having S&T organizations routinely take key technologies to TRL 6 or higher, the Secretary of Defense should extract lessons from the nonpenetrating periscope, the Advanced Amphibious Assault Vehicle, and the Army's Future Scout programs, and other advanced technology demonstrations and Advanced Concept Technology Demonstrations programs. Specifically, the Secretary of Defense should assess whether the resources needed to enable S&T organizations to play a leading role in the development of technologies and, in some cases, preliminary system design, detracted from or displaced more basic research and technology development programs.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: In commenting on a draft of this report, DOD stated that it agreed with much of GAO's position that the traditional path to new weapon system development is no longer affordable or necessary. DOD stated its Revolution in Business Affairs will enable the acquisition of new technologies to be more efficient and effective. DOD, in its revised acquisition policy, has emphasized the use of successful technology transitions methods such as advanced concept technology demonstrations, making them the norm instead of the exception.

    Recommendation: Science and technology (S&T) organizations will have to play a greater role in maturing technologies to higher levels and should be funded accordingly. Therefore, the Secretary of Defense should evaluate the different ways S&T organizations can play a greater role in helping technologies reach high levels of maturity before product development begins. For example, given that a technology has sufficient potential for application to a weapon system, at a minimum, a S&T organization should be responsible for taking a technology to TRL 6 before it is handed off to a program office at the program definition and risk reduction phase. During this phase, the program manager would be responsible for maturing the technology to TRL 7 before it is included in an engineering and manufacturing development program. In a situation where a single, design-pacing technology is to be developed for a known application--like the nonpenetrating periscope--a S&T organization should be required to mature that technology to TRL 7 before it is turned over to a product development manager. S&T organizations could play a similar role when a significant new technology is being prepared for insertion into an existing weapon system. When multiple new technologies are to be merged to create a weapon system, S&T organizations should be required to bring key technologies to TRL 6 and then become part of a hybrid organization with product developers to integrate the technologies and bring them to TRL 7 before handing full responsibility to a product development manager.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: In commenting on a draft of this report, DOD stated that it agreed with much of GAO's position that the traditional path to new weapon system development is no longer affordable or necessary. DOD stated its Revolution in Business Affairs will enable the acquisition of new technologies to be more efficient and effective. DOD has taken steps to separate technology and product development in their revised acquisition policy. A key part of this is using better ways to transition technologies into product development such as better use of advanced technology demonstrations and advanced concept technology demonstrations.

    Recommendation: The Secretary of Defense should find ways to ensure that the managers responsible for maturing the technologies and designing weapon systems before product development are provided the more flexible environment that is suitable for the discovery of knowledge, as distinct from the delivery of a product. Providing more flexibility will require the cooperation of requirements managers and resource managers so that rigid requirements or the threat of jeopardizing the funding planned to start product development will not put pressure on program managers to accept immature technologies. Such an environment may not be feasible if the program definition and risk reduction phase remains the effective launch point for an entire weapon system program.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: In commenting on a draft of this report, DOD stated that it agreed with much of GAO's position that the traditional path to new weapon system development is no longer affordable or necessary. DOD stated its Revolution in Business Affairs will enable the acquisition of new technologies to be more efficient and effective.

    Recommendation: The Secretary of Defense should: (1) establish the place at which a match is achieved between key technologies and weapon system requirements as the proper time for committing to the cost, schedule, and performance baseline for developing and producing that weapon system; and (2) require that key technologies reach a high maturity level--analogous to TRL 7--before making that commitment. This would approximate the launch point for product development as practiced by leading commercial firms.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: In commenting on a draft of this report, DOD stated that it agreed with much of GAO's position that the traditional path to new weapon system development is no longer affordable or necessary. DOD stated its Revolution in Business Affairs will enable the acquisition of new technologies to be more efficient and effective.

    Recommendation: To help ensure that new technologies are vigorously pursued and successfully moved into weapon system programs, the Secretary of Defense should adopt a disciplined and knowledge-based method for assessing technology maturity, such as TRLs, DOD-wide. This practice should employ standards for assessing risks of handoff to program managers that are based on a technology's level of demonstration and its criticality to meeting the weapon system's requirements.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: In commenting on a draft of this report, DOD stated that it agreed with much of GAO's position that the traditional path to new weapon system development is no longer affordable or necessary. DOD stated that its Revolution in Business Affairs will enable the acquisition of new technologies to be more efficient and effective.

    Recommendation: The Secretary of Defense should empower managers of product development programs to refuse to accept key technologies with low levels of demonstrated maturity. The Secretary of Defense can encourage this behavior through supportive decisions on individual programs, such as by denying proposals to defer the development of key technologies and by favoring proposals to lengthen schedules or lessen requirements to reduce technological risk early.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: In commenting on a draft of this report, DOD stated that it agreed with much of GAO's position that the traditional path to new weapon system development is no longer affordable or necessary. DOD stated its Revolution in Business Affairs will enable the acquisition of new technologies to be more efficient and effective. Since then, in May 2003, DOD revised its acquisition policies and included guidance for beginning an acquisition program as follows: "...Technology developed in S&T shall have been demonstrated in a relevant environment or, preferably in an operational environment to be considered mature enough to use for product development." It also now requires an approved Technology Development Strategy before an acquisition program can begin.

    Jul 31, 2014

    Jul 30, 2014

    Jul 28, 2014

    Jul 17, 2014

    Jul 14, 2014

    Looking for more? Browse all our products here