Department of Energy:

Major Construction Projects Need a Consistent Approach for Assessing Technology Readiness to Help Avoid Cost Increases and Delays

GAO-07-336: Published: Mar 27, 2007. Publicly Released: Mar 27, 2007.

Additional Materials:

Contact:

Eugene E. Aloise
(202) 512-6870
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Department of Energy (DOE) spends billions of dollars on major construction projects that help maintain the nuclear weapons stockpile, conduct research and development, and process nuclear waste so that it can be disposed of. Because of DOE's long-standing project management problems, GAO determined the extent to which (1) DOE's major construction projects are having cost increases and schedule delays and the major factors contributing to these problems and (2) DOE ensures that project designs are sufficiently complete before construction begins to help avoid cost increases and delays. We examined 12 DOE major projects with total costs of about $27 billion, spoke with federal and contractor officials, and reviewed project management documents.

Of the 12 DOE major projects GAO reviewed, 9 exceeded their original cost or schedule estimates, principally because of ineffective DOE project oversight and poor contractor management. Specifically, 8 of the 12 projects experienced cost increases ranging from $79.0 million to $7.9 billion, and 9 of the 12 projects were behind schedule by 9 months to more than 11 years. Project oversight problems included, among other things, inadequate systems for measuring contractor performance, approval of construction activities before final designs were sufficiently complete, ineffective project reviews, and insufficient DOE staffing. Furthermore, contractors poorly managed the development and integration of the technology used in the projects by, among other things, not accurately anticipating the cost and time that would be required to carry out the highly complex tasks involved. Even though DOE requires final project designs to be sufficiently complete before beginning construction, it has not systematically ensured that the critical technologies reflected in these designs have been demonstrated to work as intended (technology readiness) before committing to construction expenses. Specifically, only one of the five DOE project directors with projects that have recently begun or are nearing construction had systematically assessed technology readiness. The other four directors also told us that they have or will have completed prior to construction, 85 to 100 percent of their projects' final design, but they had not systematically assessed technology readiness. Proceeding into construction without also demonstrating a technology's readiness can lead to cost increases and delays. For example, one technology to be used in DOE's Waste Treatment and Immobilization Plant was not sufficiently demonstrated--that is, shown to be technologically ready for its intended application--before construction began. Consequently, the technology did not perform as expected, which resulted in about $225 million in redesign costs and schedule delays of more than 1 year. To help avoid these problems, the National Aeronautics and Space Administration (NASA) pioneered and the Department of Defense (DOD) has adopted for its projects a method for measuring and communicating technology readiness levels (TRL). Using a scale from one (basic principles observed) through nine (total system used successfully in project operations), TRLs show the extent to which technologies have been demonstrated to work as intended in the project. DOE project directors agreed that such an approach would help make technology assessments more transparent and improve stakeholder communication prior to making critical project decisions, such as authorizing construction.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: DOE issued its Technology Readiness Assessment Guide on October 12, 2009--DOE G 413.3-4. Consistent with our recommendation, this guide provides a disciplined and consistent approach to assess technology readiness, including the use of consistent metrics and terminology and oversight protocols for reviewing technology readiness. As explained in the purpose section of the guide, the guide presents a tailored version of the NASA and DOD technology assessment model. As a result of these actions, this recommendation is closed.

    Recommendation: To improve decision making and oversight for major DOE construction projects, including how project technology readiness is measured and reported, the Secretary of Energy should evaluate and consider adopting a disciplined and consistent approach to assessing TRLs for projects with critical technologies that includes developing comprehensive standards for systematically measuring and communicating the readiness of project technologies. At a minimum, these standards should (1) specify consistent metrics for determining technology readiness departmentwide, (2) establish terminology that can be consistently applied across projects, and (3) detail the oversight protocols to be used in reporting and reviewing TRLs. In preparing these standards, DOE should consider lessons learned from NASA and DOD, and its own experience in measuring technology readiness. If DOE's evaluation results in the decision to adopt these standards, it should incorporate them into DOE Order 413.3A and Manual 413.3-1, and provide the appropriate training to ensure their proper implementation.

    Agency Affected: Department of Energy

  2. Status: Closed - Implemented

    Comments: DOE issued its Technology Readiness Assessment Guide on October 12, 2009--DOE G 413.3-4. DOE has tailored technology assessment model used by NASA and DOD in two important ways that create a lower standard to achieve compared to best practices. First, the guide suggests approval of a project at the preliminary design and commitment phase (i.e., critical decision-2) with critical technologies that have reached a readiness level of 6 rather than level 7 that we recommended. The guide also suggests approval of a project at the beginning of construction (i.e., critical decision-3) with critical technologies that have reached a readiness level of 6 rather than, as we recommended, at least level 7 or, if possible, level 8. Second, the DOE revised the definitional language of the technology readiness level 7 from requiring technology be demonstrated to work in an "operational" environment to requiring that the technology be demonstrated in a "relevant" environment. The "relevant" environment standard is used in TRL 6, thus DOE has lowered the standard for meeting technology development best practices, creating a situation where the department may be at higher risk for cost increases and schedule delays due to unacknowledged technology development risks. Nonetheless, the technology readiness guide is a significant commitment on the part of DOE intended to address technology issues and overall is responsive to our recommendation. As a result, this recommendation is closed.

    Recommendation: To improve decision making and oversight for major DOE construction projects, including how project technology readiness is measured and reported, the Secretary of Energy should evaluate and consider adopting a disciplined and consistent approach to assessing TRLs for projects with critical technologies that includes directing DOE Acquisition Executives to ensure that projects with critical technologies reach a level of readiness commensurate with acceptable risk--analogous to TRL 7--before deciding to approve the preliminary design and commit to definitive cost and schedule estimates, and at least TRL 7 or, if possible, TRL 8 before committing to construction expenses.

    Agency Affected: Department of Energy

  3. Status: Closed - Implemented

    Comments: DOE explained that the assessment of technology readiness and risk factors will be included in the Acquisition Executive's documented decision process, and will be appropriately communicated to interested Congressional committees and members. The issuance of the technology readiness guide and the commitment to include a technology assessment as part of the documented acquisition process is responsive to our recommendation. As a result, this recommendation is closed.

    Recommendation: To improve decision making and oversight for major DOE construction projects, including how project technology readiness is measured and reported, the Secretary of Energy should evaluate and consider adopting a disciplined and consistent approach to assessing TRLs for projects with critical technologies that includes informing the appropriate committees and Members of Congress of any DOE decision to approve definitive cost and schedule estimates, or to begin construction, without first having ensured that project technologies are sufficiently ready (at TRL 7 or 8). This information should include specific plans for mitigating technology risks, such as developing backup technologies to offset the effects of a potential technology failure, and appropriate justification for accepting higher technological risk.

    Agency Affected: Department of Energy

 

Explore the full database of GAO's Open Recommendations »

Oct 20, 2014

Oct 1, 2014

Sep 22, 2014

Sep 10, 2014

Aug 11, 2014

Jul 17, 2014

Jul 11, 2014

Jun 23, 2014

Jun 9, 2014

Jun 5, 2014

Looking for more? Browse all our products here