High-Performance Computing: NNSA Could Improve Program Management Processes for System Acquisitions
Fast Facts
The National Nuclear Security Administration maintains the nuclear stockpile. To analyze the performance, safety, and reliability of nuclear weapons, it acquires high-performance computing systems for simulations. The latest system is expected to be complete in 2024.
Is NNSA following best practices to acquire this $600 million system?
NNSA could have done a better job considering alternatives. For example, NNSA's analysis of alternatives was not conducted independently, as its policies require. The contractor that manages the acquisition program conducted the analysis.
Our recommendations address this and other issues we found.
U.S. Air Force missile maintainers working on an intercontinental ballistic missile
Highlights
What GAO Found
The National Nuclear Security Administration's (NNSA) analysis of alternatives (AOA) process for its $600 million El Capitan HPC acquisition did not fully follow agency policy that states that AOA processes should be consistent with GAO best practices, where possible, and any deviations must be justified and documented. According to GAO best practices, a reliable AOA process should meet four characteristics: it should be comprehensive, well documented, unbiased, and credible. As seen in the table, the AOA process for El Capitan partially met one of these characteristics and minimally met the other three. NNSA did not justify or document the deviations from these best practices, as required by NNSA policy. GAO also found that the AOA process was conducted by the contractor that manages the El Capitan acquisition program, contrary to agency policy and guidance stating that AOAs should be conducted by an independent entity. Without following AOA best practices where possible; justifying and documenting any deviations; and ensuring AOA processes are conducted by an independent entity, as required, NNSA cannot be assured of a reliable assessment of options for meeting critical mission needs.
Extent to Which the National Nuclear Security Administration (NNSA) Met the Characteristics of a Reliable Analysis of Alternatives (AOA) Process
|
AOA characteristic |
GAO assessment |
Example of deviation |
|
Comprehensive |
Partially met |
Cost estimates are incomplete and did not follow best practices. |
|
Well documented |
Minimally met |
The alternatives' descriptions are not detailed enough for a robust analysis. |
|
Unbiased |
Minimally met |
NNSA had a predetermined solution, acquiring an HPC system, before performing the AOA process. |
|
Credible |
Minimally met |
The selection criteria appear to have been written for the preferred alternative. |
Source: GAO analysis of NNSA information. | GAO-21-194
GAO found that, in the second year of the El Capitan acquisition program's 5-year acquisition life cycle, NNSA has fully implemented selected key practices related to program monitoring and control. However, NNSA has only partially implemented key practices related to requirements management. Specifically, El Capitan program officials did not update and maintain acquisition program documents to include current requirements. NNSA officials stated that once the program developed its program plan early in the program's life cycle, they did not require the program to update and maintain that program plan. However, NNSA's own program management policy requires programs to update program documents throughout the duration of the program. Without updating and maintaining El Capitan program documents to include current requirements, NNSA officials may be limited in their ability to ensure that all mission requirements are met.
Why GAO Did This Study
NNSA is responsible for maintaining the nation's nuclear stockpile. To analyze the performance, safety, and reliability of nuclear weapons, it acquires high-performance computing (HPC) systems to conduct simulations. The latest system, El Capitan, is expected to be fully deployed by March 2024.
The committee report accompanying the Energy and Water Development and Related Agencies Appropriations Act, 2019, includes a provision for GAO to review NNSA's management of its Advanced Simulation and Computing program. This report examines, among other things, (1) the extent to which NNSA's AOA process for the El Capitan acquisition met best practices and followed agency policy and guidance and (2) the extent to which NNSA is implementing selected acquisition best practices in carrying out the El Capitan acquisition program. GAO reviewed documents and interviewed NNSA officials and laboratory representatives involved in carrying out the AOA and acquisition processes.
Recommendations
GAO recommends that NNSA (1) ensure that future HPC acquisition programs follow AOA best practices, where possible, and justify and document any deviations consistent with policy; (2) ensure that an independent entity conducts future AOA processes; and (3) update and maintain acquisition program documents to include El Capitan requirements for the ongoing acquisition. NNSA generally concurred with these recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| National Nuclear Security Administration | The Administrator of NNSA should take steps to ensure that, for future HPC acquisitions, the ASC program follows GAO best practices for AOA processes, where possible, and justifies and documents any deviations, as required by agency policy. (Recommendation 1) |
Open
In it's agency comments, NNSA indicated it would follow analysis of alternatives (AOA) best practices only for capital acquisition projects; the agency does not consider the bulk of a high-performance computing (HPC) purchase as a capital acquisition. We found the comment to be unresponsive to our recommendation. For the El Capitan acquisition, NNSA's AOA did not meet best practices, which could have the effect of reducing, rather than enhancing, the credibility of future NNSA HPC efforts. The report accompanying the FY 2022 National Defense Authorization Act requires NNSA to develop a HPC roadmap, but it is not yet complete. NNSA is currently planning for another HPC acquisition within the next 1-2 years.
|
| National Nuclear Security Administration | The Administrator of NNSA should ensure that the ASC program's future AOA processes are performed by an entity independent of the contractor organization managing and executing the program. (Recommendation 2) |
Open
NNSA agreed with this recommendation, but it is unclear if the agency will follow through given that it does not consider HPCs capital acquisitions and does not believe AOA best practices need to be followed for programs that are not capital acquisitions. We will continue to monitor future HPC acquisitions. The report accompanying the FY 2022 National Defense Authorization Act requires NNSA to develop a HPC roadmap. NNSA has not yet completed this document. We will analyze that document when available.
|
| National Nuclear Security Administration | The Administrator of NNSA should update and maintain its acquisition program documents to include current El Capitan requirements and clearly document the relationship between El Capitan's mission and functional requirements. (Recommendation 3) |
Open
NNSA agreed with this recommendation and officials stated they verified the traceability between six El Capitan mission needs documented in the Acquisition Management Plan and the functional requirements that are documented in the Statement of Work (e.g., specific requirements for system memory, power consumption, and availability). However, the agency has not yet demonstrated that it documented this relationship in the program documentation. In January 2023, officials stated that they expect to update program documentation once the system is delivered and reiterated this position in August 2023. To meet the intent of this recommendation, we would expect that the program documentation be updated to reflect this traceability, as well as any changes, throughout the course of development rather than after the system was delivered. NNSA said that traceability will be apparent in the next High Performance Computer acquisition. Unrealized El Capitan scope will automatically flow into subsequent scopes of work.
|