Recommendation: The Director of NSF should assess the agency's large facilities oversight workforce to identify any project management competency gaps, develop a plan to address any gaps and time frames for doing so, and monitor progress in closing them. (Recommendation 1)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation and, as of September 2019, had awarded a contract for a proficiency assessment and workforce gap analysis. NSF anticipated finishing the proficiency assessment and workforce gap analysis by the second quarter calendar year 2020 and planned to consider options to address any identified gaps, such as identifying workforce development training opportunities. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

Recommendation: The Director of NSF should establish criteria for the project management expertise of award recipients for large facilities projects and incorporate the criteria in project requirements and external panel reviews. (Recommendation 2)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation and, as of November 2019, had drafted but not finalized new language for the Major Facilities Guide and related supplemental award terms and conditions for major facilities. The new language would require award recipients to document their plans for meeting project management competencies. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

Recommendation: The Director of NSF should ensure, through a requirement or other means, that award recipients for large facilities projects provide information to NSF on any lessons learned or best practices. (Recommendation 3)

Agency: National Science Foundation
Status: Open

Comments: NSF agreed with this recommendation and, as of November 2019, had drafted but not finalized supplemental award terms and conditions for major facilities to require recipients to participate in NSF's process for identifying and sharing lessons learned on projects - for example, by sending appropriate staff to the annual major facilities workshop that NSF hosts to provide a collaborative forum for continuous learning and information-sharing or by presenting lessons learned or good practices at the workshop. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

Recommendation: The Secretary of Energy, in coordination with DOE's Office of Project Management and NNSA's Office of Acquisition and Project Management, should develop requirements that clearly define how and where project management lessons learned for capital asset projects should be documented and saved to make them readily accessible across the department. (Recommendation 1)

Agency: Department of Energy
Status: Open

Comments: DOE concurred with the recommendation and as of July 2020, continues to work on developing a policy memorandum in advance of revising DOE Order 413.3B, Program and Project Management for the Acquisition of Capital Assets, to identify the project management lessons learned and outline the kinds of information the repository will collect. We will monitor DOE's effort and evaluate the action once it is complete.

Recommendation: The Secretary of Energy, in coordination with DOE's Office of Project Management and NNSA's Office of Acquisition and Project Management, should develop requirements for sharing project management lessons learned for capital asset projects from the beginning of a project (i.e., prior to the start of construction at CD-3) routinely and in a timely manner to improve DOE's ability to identify and evaluate problematic practices and positive experiences. (Recommendation 2)

Agency: Department of Energy
Status: Open

Comments: DOE concurred with the recommendation and as of July 2020, continues to work on developing a policy memorandum in advance of revising DOE Order 413.3B, Program and Project Management for the Acquisition of Capital Assets, to have those conducting peer reviews elicit project management lessons learned with department-wide applicability and submit any discerned, as well as replace the order's definition of project management lessons learned with a standard industry definition. We will monitor DOE's efforts and evaluate the actions once they are complete.

Recommendation: The Secretary of Energy, in coordination with DOE's Office of Project Management and NNSA's Office of Acquisition and Project Management, should develop requirements for evaluating the effectiveness of corrective actions taken in response to project management problems for capital asset projects, with a focus on those lessons that necessitate the need for department-wide improvements. (Recommendation 3)

Agency: Department of Energy
Status: Open

Comments: DOE concurred with the recommendation and as of July 2020, plans to revise the Project Management Risk Committee charter by assigning it the responsibility to qualitatively evaluate the effectiveness of corrective actions taken in response to project management lessons learned from projects with a total project cost greater than $750 million having department-wide implications. This action, however, may not fully address the recommendation, in part because there may be some lessons learned with applicability department-wide from projects that do not meet this cost threshold. We will monitor DOE's effort and evaluate the action once it is complete.

Recommendation: The Administrator of NNSA should require an independent entity to validate that contractor EVM systems used for LEPs meet the EVM national standard. (Recommendation 1)

Agency: Department of Energy: National Nuclear Security Administration
Status: Open

Comments: In October 2018, NNSA stated that the integrated baseline reviews, internal reviews of earned value management (EVM) systems used by life extension programs (LEP), and other controls over data integration provide a practical and cost-beneficial approach to the validation of contractor EVM systems. However, in early 2019, NNSA conducted a lessons learned study to more fully assess the cost, benefit, mission impacts, and feasibility of implementing our recommendation for possible application to future LEPs. According to NNSA documentation, based on the results of the lessons learned study, NNSA concluded that the effort and expense needed to validate contractor EVM systems against the EVM national standard could be better used providing resources to improve the agency's ability to establish work breakdown structures, schedules, and integration of scope, cost, and schedule. As a result, as of September 2019, NNSA stated that its actions meet the intent of our recommendation. We disagree. As we stated in our report, without requiring an independent entity to validate that contractor EVM systems meet the EVM national standard, NNSA may not have assurance that its LEPs are obtaining reliable EVM data for managing their programs and reporting their status. We will continue to monitor NNSA's activities to address this recommendation.

Recommendation: The Administrator of NNSA should require an independent entity to conduct surveillance reviews of contractor EVM systems used for LEPs to ensure that they maintain compliance with the EVM national standard through program completion. (Recommendation 2)

Agency: Department of Energy: National Nuclear Security Administration
Status: Open

Comments: In October 2018, NNSA stated that it conducts ongoing surveillance through integrated baseline reviews, internal reviews of EVM systems used by LEPs, and other assessments, which consider national standards. However, in early 2019, NNSA conducted a lessons learned study to more fully assess the cost, benefit, mission impacts, and feasibility of implementing our recommendation for possible application to future LEPs. According to NNSA documentation, based on the results of the lessons learned study, NNSA concluded that the effort and expense needed to conduct surveillance reviews of contractor EVM systems to ensure compliance with the EVM national standard could be better used providing resources to improve the agency's ability to establish work breakdown structures, schedules, and integration of scope, cost, and schedule. As a result, as of September 2019, NNSA stated that its actions meet the intent of our recommendation. We disagree. As we stated in our report, without requiring an independent entity to conduct surveillance reviews of contractor EVM systems through program completion, NNSA may not have assurance that its LEPs are obtaining reliable EVM data for managing their programs and reporting their status. We will continue to monitor NNSA's activities to address this recommendation.

Recommendation: The Administrator of NNSA should require its programs to ensure that LEP critical technologies meet specific technology readiness level benchmarks at decision points, or otherwise document with program executive approval their rationale for not meeting these benchmarks. (Recommendation 3)

Agency: Department of Energy: National Nuclear Security Administration
Status: Open

Comments: In October 2018, NNSA stated that it has already incorporated specific benchmarks for technology readiness levels at decision points. As an example, it stated that it recommends a technology readiness level of 5 at the beginning of phase 6.3 for an LEP. As a result, NNSA stated that its actions meet the intent of our recommendation. We disagree. As we stated in our report, it is important for NNSA to establish a requirement, not just a recommendation, that LEP critical technologies meet specific technology readiness level benchmarks at decision points. We will continue to monitor NNSA's activities to address this recommendation.

Recommendation: To improve NASA management and oversight of its spaceflight projects, and to improve the reliability of project EVM data, the NASA Administrator should direct the appropriate offices to modify the NASA Procedural Requirements (NPR) 7120.5 to require projects to implement a formal surveillance program that: (1) Ensures anomalies in contractor-delivered and in-house monthly earned value management reports are identified and explained, and report periodically to the center and mission directorate's leadership on relevant trends in the number of unexplained anomalies. (2) Ensures consistent use of work breakdown structures (WBS) for both the EVM report and the schedule. (3) Ensures that lower level EVM data reconcile to project level EVM data using the same WBS structure. (4) Improves underlying schedules so that they are properly sequenced using predecessor and successor dependencies and are free of constraints to the extent practicable so that the EVM baseline is reliable.

Agency: National Aeronautics and Space Administration
Status: Open
Priority recommendation

Comments: NASA partially agreed with this recommendation, stating that the reliability and utility of the EVM data needed to be improved but that it did not plan to implement a formal surveillance plan due to resource constraints. Since initially commenting on the report, however, in December 2018, NASA included an initiative in its Corrective Action Plan-a plan put in place in response to recent programmatic performance and NASA's designation on GAO's High-Risk List-to enhance EVM implementation. In June 2019, NASA issued EVM guidance that covered several items, including enhancing in-house and contracted earned value management surveillance and requiring EVM reporting at Baseline Performance Review. NASA officials reported that its near-term plans are well-defined to address the reliability of project EVM data, but they have expressed concerns about funding challenges and cultural resistance. To fully implement this recommendation, NASA will need to take action and provide documentary support for several of its identified planned next steps to enhance EVM surveillance. Without implementing proper surveillance, NASA may be utilizing unreliable EVM data in its analyses to inform its cost and schedule decision making.