National Nuclear Security Administration:

Actions Needed to Clarify Use of Contractor Assurance Systems for Oversight and Performance Evaluation

GAO-15-216: Published: May 22, 2015. Publicly Released: Jun 10, 2015.

Additional Materials:

Contact:

David C. Trimble
(202) 512-3841
trimbled@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The National Nuclear Security Administration (NNSA) has not fully established policies or guidance for using information from contractor assurance systems (CAS) to conduct oversight of management and operating (M&O) contractors. As a result, NNSA does not have standards for ensuring that contractors are overseen consistently. For example, at the headquarters level, NNSA has not provided guidance beyond its general framework for assessing the maturity of contractors' CAS to determine whether information from CAS is sufficiently reliable for federal oversight purposes. In the absence of headquarters level policy, NNSA field offices—federal offices located at contractor operated sites and responsible for day-to-day oversight of M&O contractors—have established their own procedures for using information from CAS to conduct oversight, but these procedures also are not always complete and differ among field offices. For example, five of NNSA's seven field offices reported having complete procedures for assessing CAS maturity, but these procedures describe different processes and rating scales for conducting such assessments. The other two field offices reported not having such procedures. NNSA had designed a process for validating field offices' oversight approaches, including the extent to which these approaches use information from CAS, but NNSA discontinued this process after determining that it had not been effective. Discontinuing this process without replacing it with another form of validation eliminated the internal control activity NNSA designed to assure the effectiveness and consistency of oversight approaches across the nuclear security enterprise, including the appropriate use of information from CAS.

NNSA also has not established policies or guidance specific to using information from CAS to evaluate M&O contractor performance. Neither NNSA policy nor NNSA's Handbook published in 2013 to guide the performance evaluation process includes information on how or to what extent NNSA officials should use information from CAS in evaluating M&O contractors' performance. Some field office officials told GAO they developed their own procedures on performance evaluation. GAO reviewed these procedures and found they were not sufficiently detailed for using information from CAS to evaluate contractors' performance.

NNSA has not determined whether it has sufficient, qualified personnel to implement its framework for using information from CAS for oversight or for performance evaluation. NNSA officials GAO interviewed were unable to identify any studies that had been completed that assessed this question. Field office officials have raised concerns that staffing levels and the mix of staff skills may not be adequate to conduct appropriate oversight in the future and that this may result in overreliance on information from CAS without the ability to ensure that this information is sufficiently mature. In 2013, concerned about their capacity to fully support all oversight requirements, field offices called on NNSA headquarters to initiate a review of field office staffing resources needed to implement the oversight and performance evaluation framework and whether a staffing model including shared technical staff among field offices could address these concerns. NNSA began this review, but NNSA headquarters officials said it was not completed, the data reviewed are now too old to be useful, and that the agency has no plans to complete it. These officials said that they plan on discussing staffing issues with senior leadership in 2015.

Why GAO Did This Study

NNSA is responsible for overseeing the work of seven M&O contractors that execute its programs across eight nuclear security enterprise sites and evaluating their performance. NNSA directed its M&O contractors in 2011 to implement CAS—systems designed and used by contractors to assure their own performance—that NNSA could also leverage for oversight purposes and thereby improve efficiency. To determine the extent to which to use contractor-generated information made available through CAS, NNSA is to apply a framework for evaluating the risk of contractors' activities, their past performance, and their CAS maturity.

GAO was asked to review NNSA's implementation of its framework for using CAS. This report examines the extent to which NNSA has fully established policies and guidance for using information from CAS to (1) oversee M&O contractors and (2) evaluate M&O contractors' performance; and whether NNSA has determined it has sufficient, qualified personnel to implement its framework for using information from CAS for these two purposes. To conduct its work, GAO surveyed all NNSA field offices and analyzed key policies and guidance on NNSA's use of information from CAS.

What GAO Recommends

GAO is recommending, among other things, that NNSA develop guidance on using information from CAS to oversee and evaluate M&O contractors, reinstitute a process for evaluating oversight effectiveness, and study staffing needs. NNSA agreed with these recommendations.

For more information, contact David C. Trimble at (202) 512-3841 or trimbled@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Priority recommendation

    Comments: We recommended that NNSA establish comprehensive policies and guidance, beyond a general framework, for using information from contractor assurance systems (CAS) to conduct oversight of management and operating (M&O) contractors, clarifying whether CAS is to cover mission-related activities and describing how to conduct assessments of risk, CAS maturity, and the level of the contractor's past performance. NNSA agreed with the recommendation and has taken an important step to revise its policy. However, NNSA needs to take additional action. Specifically, NNSA approved a revised corporate site governance policy in August 2016, and NNSA further revised its policy in October 2019. The revised policy improves on the agency's prior policy by clarifying one element in our recommendation that CAS is to cover mission-related activities. However the policy is still a general framework and NNSA has not established associated implementing guidance. Specifically, NNSA needs to develop guidance for NNSA headquarters' and field offices' procedures to use information from CAS and appropriately balance use of information from CAS with other more direct activities to oversee M&O contractors. As of April 2020, this recommendation remains open.

    Recommendation: To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should establish comprehensive NNSA policies and guidance, beyond a general framework as included in NAP-21, for using information from CAS to conduct oversight of M&O contractors, clarifying whether CAS is to cover mission-related activities, and describing how to conduct assessments of risk, CAS maturity, and the level of the contractor's past performance.

    Agency Affected: Department of Energy: National Nuclear Security Administration

  2. Status: Open

    Comments: NNSA concurred with the recommendation. As of October 2019, NNSA field offices have generally adopted updated procedures for assessment practices. However, these updated procedures generally do not appear to go beyond the broad general framework described in NNSA's August 2016 revised corporate site governance policy. As such, additional actions are needed as the procedures may not support consistent assessment practices. We will provide updated information as we review these field office procedures and future updates.

    Recommendation: To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should work with field office managers to establish field office procedures consistent with headquarters policy and guidance to support assessment practices for determining appropriate oversight approaches.

    Agency Affected: Department of Energy: National Nuclear Security Administration

  3. Status: Closed - Implemented

    Comments: NNSA concurred with this recommendation and issued Supplemental Directive (SD) 226.1B, which provides an approach for reviewing the effectiveness of field offices' oversight approaches. The SD requirements include a peer review process and provide attributes that, if followed, may result in improved field oversight activities. According to NNSA, the agency has completed three peer reviews--pilot peer reviews at the Nevada National Security Site in July 2017 and the Lawrence Livermore National Laboratory in November 2017, and another peer review at the National Nuclear Security Administration Production Office (Y-12 National Security Complex/Pantex Plant) in April 2018--with additional peer reviews scheduled for later in 2018 (Sandia) and 2019 (Kansas City Plant, Los Alamos, Savannah River).

    Recommendation: To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should reestablish a process for reviewing the effectiveness of field offices' oversight approaches, including their use of information from CAS.

    Agency Affected: Department of Energy: National Nuclear Security Administration

  4. Status: Open

    Comments: NNSA concurred with the recommendation. NNSA's revised Supplemental Directive addresses the use of CAS information in evaluating M&O contractor performance, NNSA issued a revised performance management policy (NAP-4C), and NNSA field offices have generally adopted updated procedures outlining their governance systems. However, these actions have not gone further than generally stating that those responsible for evaluating M&O contractor performance should use information from CAS "as appropriate." We will provide updated information on the extent to which other new policy, guidance, or procedures address the use of information from CAS for performance evaluations .

    Recommendation: To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should revise NNSA policy, guidance, and procedures on performance evaluation to fully address how and under what circumstances those responsible for evaluating M&O contractors' performance should use information from CAS for this purpose.

    Agency Affected: Department of Energy: National Nuclear Security Administration

  5. Status: Closed - Implemented

    Comments: NNSA concurred with this recommendation and has assessed its staffing needs through a study by the U.S. Office of Personnel Management (OPM) and a study by NNSA's Office of Cost Estimating and Program Evaluation (CEPE). According to NNSA, it is actively pursuing a staffing strategy for defensible and sustainable oversight based on the results of these studies. Specifically, the studies informed staffing plans to identify positions needed for mission success that NNSA directed each program, functional, and field office to develop in October 2018. The staffing plans are currently being reviewed by the Staffing Plan Review Group, which is charged to examine position title, function, occupational series, grade, supervisory status, geographic locality and mission requirement and identify if additional staff are needed above the levels in the OPM and CEPE studies. The Staffing Plan Review Group will present its findings to the Administrator near the end of 2019.

    Recommendation: To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should assess NNSA's staffing needs to determine whether it has sufficient, qualified personnel to conduct oversight activities consistent with comprehensive policies and guidance, including the use of information from CAS.

    Agency Affected: Department of Energy: National Nuclear Security Administration

 

Explore the full database of GAO's Open Recommendations »

Jul 24, 2020

Jun 24, 2020

Jun 9, 2020

May 13, 2020

May 12, 2020

Apr 29, 2020

  • energy icon, source: Art Explosion

    Priority Open Recommendations:

    Department of Energy
    GAO-20-285PR: Published: Apr 22, 2020. Publicly Released: Apr 29, 2020.

Apr 17, 2020

Mar 11, 2020

Mar 4, 2020

Feb 26, 2020

  • energy icon, source: Art Explosion

    Science & Tech Spotlight:

    Nuclear Microreactors
    GAO-20-380SP: Published: Feb 26, 2020. Publicly Released: Feb 26, 2020.

Looking for more? Browse all our products here