National Nuclear Security Administration:

Actions Needed to Clarify Use of Contractor Assurance Systems for Oversight and Performance Evaluation

GAO-15-216: Published: May 22, 2015. Publicly Released: Jun 10, 2015.

Additional Materials:

Contact:

David C. Trimble
(202) 512-3841
trimbled@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The National Nuclear Security Administration (NNSA) has not fully established policies or guidance for using information from contractor assurance systems (CAS) to conduct oversight of management and operating (M&O) contractors. As a result, NNSA does not have standards for ensuring that contractors are overseen consistently. For example, at the headquarters level, NNSA has not provided guidance beyond its general framework for assessing the maturity of contractors' CAS to determine whether information from CAS is sufficiently reliable for federal oversight purposes. In the absence of headquarters level policy, NNSA field offices—federal offices located at contractor operated sites and responsible for day-to-day oversight of M&O contractors—have established their own procedures for using information from CAS to conduct oversight, but these procedures also are not always complete and differ among field offices. For example, five of NNSA's seven field offices reported having complete procedures for assessing CAS maturity, but these procedures describe different processes and rating scales for conducting such assessments. The other two field offices reported not having such procedures. NNSA had designed a process for validating field offices' oversight approaches, including the extent to which these approaches use information from CAS, but NNSA discontinued this process after determining that it had not been effective. Discontinuing this process without replacing it with another form of validation eliminated the internal control activity NNSA designed to assure the effectiveness and consistency of oversight approaches across the nuclear security enterprise, including the appropriate use of information from CAS.

NNSA also has not established policies or guidance specific to using information from CAS to evaluate M&O contractor performance. Neither NNSA policy nor NNSA's Handbook published in 2013 to guide the performance evaluation process includes information on how or to what extent NNSA officials should use information from CAS in evaluating M&O contractors' performance. Some field office officials told GAO they developed their own procedures on performance evaluation. GAO reviewed these procedures and found they were not sufficiently detailed for using information from CAS to evaluate contractors' performance.

NNSA has not determined whether it has sufficient, qualified personnel to implement its framework for using information from CAS for oversight or for performance evaluation. NNSA officials GAO interviewed were unable to identify any studies that had been completed that assessed this question. Field office officials have raised concerns that staffing levels and the mix of staff skills may not be adequate to conduct appropriate oversight in the future and that this may result in overreliance on information from CAS without the ability to ensure that this information is sufficiently mature. In 2013, concerned about their capacity to fully support all oversight requirements, field offices called on NNSA headquarters to initiate a review of field office staffing resources needed to implement the oversight and performance evaluation framework and whether a staffing model including shared technical staff among field offices could address these concerns. NNSA began this review, but NNSA headquarters officials said it was not completed, the data reviewed are now too old to be useful, and that the agency has no plans to complete it. These officials said that they plan on discussing staffing issues with senior leadership in 2015.

Why GAO Did This Study

NNSA is responsible for overseeing the work of seven M&O contractors that execute its programs across eight nuclear security enterprise sites and evaluating their performance. NNSA directed its M&O contractors in 2011 to implement CAS—systems designed and used by contractors to assure their own performance—that NNSA could also leverage for oversight purposes and thereby improve efficiency. To determine the extent to which to use contractor-generated information made available through CAS, NNSA is to apply a framework for evaluating the risk of contractors' activities, their past performance, and their CAS maturity.

GAO was asked to review NNSA's implementation of its framework for using CAS. This report examines the extent to which NNSA has fully established policies and guidance for using information from CAS to (1) oversee M&O contractors and (2) evaluate M&O contractors' performance; and whether NNSA has determined it has sufficient, qualified personnel to implement its framework for using information from CAS for these two purposes. To conduct its work, GAO surveyed all NNSA field offices and analyzed key policies and guidance on NNSA's use of information from CAS.

What GAO Recommends

GAO is recommending, among other things, that NNSA develop guidance on using information from CAS to oversee and evaluate M&O contractors, reinstitute a process for evaluating oversight effectiveness, and study staffing needs. NNSA agreed with these recommendations.

For more information, contact David C. Trimble at (202) 512-3841 or trimbled@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Priority recommendation

    Comments: We recommended that NNSA establish comprehensive policies and guidance, beyond a general framework, for using information from contractor assurance systems (CAS) to conduct oversight of management and operating (M&O) contractors, clarifying whether CAS is to cover mission-related activities and describing how to conduct assessments of risk, CAS maturity, and the level of the contractor's past performance. NNSA agreed with the recommendation and has taken an important step to revise its policy. However, NNSA needs to take additional action. Specifically, NNSA approved a revised corporate site governance policy in August 2016. The revised policy improves on the agency's prior policy by clarifying one element in our recommendation that CAS is to cover mission-related activities. However the policy is still a general framework and NNSA has not established associated implementing guidance. Specifically, NNSA needs to develop guidance for NNSA headquarters' and field offices' procedures to use information from CAS and appropriately balance use of information from CAS with other more direct activities to oversee M&O contractors.

    Recommendation: To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should establish comprehensive NNSA policies and guidance, beyond a general framework as included in NAP-21, for using information from CAS to conduct oversight of M&O contractors, clarifying whether CAS is to cover mission-related activities, and describing how to conduct assessments of risk, CAS maturity, and the level of the contractor's past performance.

    Agency Affected: Department of Energy: National Nuclear Security Administration

  2. Status: Open

    Comments: Agency concurred with GAO's recommendation. Per DOE, NNSA will require field offices to develop new or modify existing procedures as appropriate to support the new requirements. The estimated completion date for these activities is September 30, 2016. DOE has not completed these activities by the estimated date and has not issued a revised date as of October 20, 2016.

    Recommendation: To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should work with field office managers to establish field office procedures consistent with headquarters policy and guidance to support assessment practices for determining appropriate oversight approaches.

    Agency Affected: Department of Energy: National Nuclear Security Administration

  3. Status: Open

    Comments: Agency concurred with recommendation. NNSA's revised Supplemental Directive (SD) 226.1B provides an approach to evaluate the effectiveness of the field oversight activities. The SD requirements include a peer review process and provide attributes that when followed, may result in improved field oversight activities. As of April 2017 The first peer review, which will pilot the review process and help assess implementation, will likely occur in June 2017 and will be conducted at the Nevada National Security Site. The next review will be in November at the Lawrence Livermore National Laboratory. GAO will monitor the results of the pilot to determine whether and how it will be permanently implemented.

    Recommendation: To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should reestablish a process for reviewing the effectiveness of field offices' oversight approaches, including their use of information from CAS.

    Agency Affected: Department of Energy: National Nuclear Security Administration

  4. Status: Open

    Comments: Agency concurred with recommendation. While NNSA's revised Supplemental Directive addresses the use of CAS information in evaluating M&O contractor performance, NNSA has not yet developed guidance or procedures for how to use information from contractor assurance systems in its performance evaluation process.

    Recommendation: To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should revise NNSA policy, guidance, and procedures on performance evaluation to fully address how and under what circumstances those responsible for evaluating M&O contractors' performance should use information from CAS for this purpose.

    Agency Affected: Department of Energy: National Nuclear Security Administration

  5. Status: Open

    Comments: Agency concurred with recommendation. Action in progress. NNSA will assess staffing needs and develop a staffing strategy for defensible and sustainable oversight. This strategy will implement the corporate policy and implementing guidance, while adhering to the constraints of the National Defense Authorization Act staffing ceiling. NNSA initially estimated it would complete this activity by December 31, 2016, but it has not yet been completed.

    Recommendation: To improve the internal control environment for oversight using information from CAS and develop a consistent approach to the use of information from CAS in M&O contractor oversight and performance evaluation across the nuclear security enterprise, the Administrator of NNSA should assess NNSA's staffing needs to determine whether it has sufficient, qualified personnel to conduct oversight activities consistent with comprehensive policies and guidance, including the use of information from CAS.

    Agency Affected: Department of Energy: National Nuclear Security Administration

 

Explore the full database of GAO's Open Recommendations »

Nov 14, 2017

Nov 1, 2017

Oct 26, 2017

Sep 8, 2017

Sep 5, 2017

May 26, 2017

May 25, 2017

May 24, 2017

May 17, 2017

May 3, 2017

Looking for more? Browse all our products here