DOE and NNSA Project Management:
Analysis of Alternatives Could Be Improved by Incorporating Best Practices
GAO-15-37: Published: Dec 11, 2014. Publicly Released: Dec 11, 2014.
What GAO Found
GAO identified 24 best practices for analysis of alternatives (AOA)—a process that is a key first step in capital asset acquisition. The process entails identifying, analyzing, and selecting a preferred alternative to best meet the mission need by comparing the operational effectiveness, costs, and risks of potential alternatives. Because no single set of best practices for AOAs was broadly recognized by government and private-sector entities, GAO developed a set of practices by reviewing AOA policies and guidance used by seven public and private-sector entities with experience in the AOA process, and verified these practices with subject matter experts. These best practices include, among other things, defining functional requirements based on mission need, conducting the AOA without a predetermined solution, including the status-quo alternative, and conducting an independent review of the entire AOA process. These practices can be applied to a wide range of activities, projects, and programs.
The Department of Energy's (DOE) requirements and guidance—found in the agency's orders and associated guides—for conducting an AOA do not conform to the 24 best practices GAO identified. Therefore, DOE does not have assurance that applying these requirements and guidance may lead to reliable AOAs. GAO's review of DOE's requirements for AOAs found that they conform to only 1 of the 24 best practices: the practice of defining functional requirements based on mission need. GAO's review of DOE's requirements combined with associated guidance—which includes nonmandatory approaches for meeting requirements—found that they conform to 9 of the 24 best practices. For example, DOE's guidance suggests identifying and considering at least three viable alternatives, including the status quo—a best practice that is not included in the requirements. Federal standards for internal control related to risk assessment call for agency management to decide on actions to mitigate identified risks. Without developing a reliable AOA process, DOE may not be successful in mitigating the risk it has identified related to this process.
DOE's requirements and guidance for AOAs also apply to the National Nuclear Security Administration (NNSA), a separately organized agency in DOE. For three recent NNSA projects that GAO reviewed, NNSA did not conform to most of the best practices GAO identified for conducting AOAs, therefore, raising concerns about the reliability of these AOAs. Specifically, for the three projects' AOAs, NNSA conformed to 6, 8, and 11 of the 24 identified best practices, respectively. For all three projects, NNSA consistently followed the one DOE requirement that conformed to a best practice—to define functional requirements based on mission need. DOE and NNSA officials acknowledge that unreliable AOAs are a risk factor for major cost increases and schedule delays for NNSA projects. As GAO has previously reported, NNSA has spent billions of dollars designing and partially constructing projects with an estimated cost of $750 million or more, only to later reassess alternatives. Without a process to develop reliable AOAs, NNSA may continue on this path and continue to have limited assurance that it is selecting alternatives that best meet its mission needs and will not result in major cost increases and schedule delays in the future.
Why GAO Did This Study
NNSA manages numerous programs that require the design and construction of one-of-a-kind facilities. NNSA's selection of preferred alternatives for these projects is governed by DOE requirements and guidance related to conducting an AOA. In recent years, NNSA has incurred substantial cost increases and schedule delays for such projects.
GAO was mandated to review the AOA process applied by NNSA. This report (1) identifies and describes AOA best practices, (2) determines the extent to which DOE requirements and guidance for conducting an AOA conform to AOA best practices, and (3) determines the extent to which NNSA conformed to best practices and followed certain DOE requirements and guidance in conducting the AOAs for recent NNSA projects.
To do this work, GAO examined relevant AOA guidance from the public and private sectors and DOE's AOA requirements and guidance, sought input from AOA experts, and interviewed agency officials. GAO reviewed three AOAs that NNSA completed or was scheduled to complete between November 2010 and September 2014.
What GAO Recommends
GAO recommends that DOE incorporate best practices into its AOA requirements. DOE agreed with this recommendation.
For more information, contact David C. Trimble at (202) 512-3841 or email@example.com.
Recommendation for Executive Action
Comments: DOE agreed with GAO's recommendation. In its written comments, DOE stated that DOE's order for project management (DOE 0 413.3B) will be assessed for revision following the issuance of the revision to DOE-STD1189, Integration of Safety into the Design Process, currently scheduled for November 2016. DOE stated that it will consider the AOA best practices when the policy is updated. In the interim, DOE stated that it has strengthened guidance in the area of AOA, and that in fiscal year 2015 it will develop and issue an AOA Handbook that will reflect the best practices for AOAs. In June 2015, the Secretary issued a memorandum on project management policies that, among other things, addressed this recommendation and DOE's written commitment. It directed the department to complete a DOE 413-series guide on the AOA process incorporating GAO best practices and required that AOAs be conducted and documented in a manner consistent with the guide when it is complete. In May 2016, DOE updated the project management order (DOE O 413.3B). We are pleased that the update stated that AOAs will be consistent with published GAO best practices. In addition, as of October 2016, according to a DOE official, the department expects to complete in December 2016 the AOA guide that DOE committed to in its written response to our report and in the June 2015 memorandum. The AOA guide will be critical to ensuring implementation of the new AOA requirement by providing DOE procedures for conducting reliable AOAs.
Recommendation: To minimize the risk of developing unreliable AOAs and incurring major cost increases and schedule delays on projects, the Secretary of Energy should direct DOE's Office of Acquisition and Project Management to update its project management order requirements to incorporate best practices for conducting an AOA.
Agency Affected: Department of Energy