National Science Foundation:

Revised Policies on Developing Costs and Schedules Could Improve Estimates for Large Facilities

GAO-18-370: Published: Jun 1, 2018. Publicly Released: Jun 1, 2018.

Additional Materials:

Contact:

John Neumann
(202) 512-3841
neumannj@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

One of the ways the National Science Foundation supports research is by funding construction of major research equipment and facilities, such as telescopes and ships with construction costs of $70 million or more.

We looked at whether such projects have stayed on track since the agency's 2008 policy to manage construction cost overruns. We found that 5 of 7 projects had cost or schedule increases.

We also looked at NSF's procedures for construction projects and found that they met many but not all best practices for estimating costs and schedules.

We recommended that NSF better incorporate estimating best practices into its policies.

NSF's Daniel K. Inouye Solar Telescope on Maui, Hawaii is more than 80% complete. It has experienced a 16% cost increase and a 31% schedule delay.

This is a photo of a telescope building on top of a mountain and above clouds.

This is a photo of a telescope building on top of a mountain and above clouds.

Additional Materials:

Contact:

John Neumann
(202) 512-3841
neumannj@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The National Science Foundation's (NSF) procedures for overseeing large facilities construction projects met many best practices for cost estimating but not those for developing project schedules. Specifically, NSF's procedures fully or substantially met 7 of 12 best practices in GAO's cost estimating guide and partially or minimally met others (such as conducting a sensitivity analysis to understand which variables most affect the cost estimate). In addition, they minimally met or did not meet 6 of 10 best practices in GAO's schedule development guide (such as establishing the durations of all activities). Further, while NSF reviews recipients' construction cost and schedule estimates for large facilities, the agency's policies did not incorporate procedures on how NSF officials are to ensure that those estimates meet best practices. An agency's procedures support the creation of reliable cost and schedule estimates when they fully or substantially meet the best practices in GAO's cost and schedule guides. NSF officials said that the agency's approach was to reference rather than repeat best practices in GAO's cost guide, except where agency-specific clarifications were needed. Nevertheless, without policies on how to apply all relevant best practices specifically to NSF's large facilities projects, recipients may develop cost or schedule estimates that are not reliable.

Of the seven projects NSF had funded that were covered by its policy to manage cost overruns, five had experienced cost or schedule increases since starting construction. In particular, two projects—the National Ecological Observatory Network and Daniel K. Inouye Solar Telescope—had both cost and schedule increases that were partly due to permitting challenges that could not be estimated at the time of the projects' proposals, according to NSF officials. Both of these projects also reduced their scopes. NSF required the reductions to the observatory network's scope under the agency's policy for managing cost overruns, whereas the reduction to the telescope project was a separate action. Three other projects had only schedule increases, with no increase in costs, and the remaining two projects had experienced neither cost nor schedule increases as of December 2017.

Cost and Schedule Performance of Large Facilities Projects Covered by NSF's Policy for Managing Cost Overruns, as of December 2017

Project name

Percentage complete

Cost change

Schedule change

Scope reductions

Alaska Region Research Vessel

100

-

Ocean Observatories Initiative

100

-

Advanced Laser Interferometer Gravitational Wave Observatory

99

-

-

National Ecological Observatory Network

94

Daniel K. Inouye Solar Telescope

81

Large Synoptic Survey Telescope

50

-

-

-

Regional Class Research Vessels

4

-

-

-

Legend: - = no cost or schedule increase or scope reductions since starting construction; = cost decreased; ▲ = cost or schedule increased; ✔ = scope reduced.

Source: GAO analysis of National Science Foundation (NSF) documents.  |  GAO-18-370

Why GAO Did This Study

NSF uses cooperative agreements and contracts to fund construction of science and engineering research infrastructure, such as telescopes. In 2008, NSF established a policy to manage cost overruns and strengthen oversight of these large facilities projects, which typically have construction costs greater than $70 million. Among other things, the policy generally requires a project's scope to be reduced before its NSF-authorized cost may be increased.

Senate Report 114-239 and House Report 114-605 included provisions for GAO to review projects funded within NSF's Major Research Equipment and Facilities Construction account used to fund construction of large facilities projects. GAO (1) examined the extent to which NSF's procedures for estimating construction costs and schedules of large facilities met best practices and (2) described the construction cost and schedule performance of NSF's large facilities projects since implementation of its policy to manage cost overruns. GAO compared NSF's procedures documented in its policies with best practices in GAO's cost and schedule guides, analyzed documents for the seven projects covered by NSF's policy to manage cost overruns, and interviewed NSF officials.

What GAO Recommends

GAO recommends that NSF revise its policies for estimating and reviewing the costs and schedules of large facilities projects to better incorporate best practices in GAO's cost and schedule guides. NSF agreed with GAO's recommendations.

For more information, contact John Neumann at (202) 512-3841 or neumannj@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: NSF addressed this recommendation by updating its Major Facilities Guide (previously named the Large Facilities Manual) in September 2019 to fully incorporate the five cost estimating best practices in GAO's cost guide that we previously found were minimally or partially met. For example, whereas NSF's procedures previously required a sensitivity analysis but did not describe how one is to be conducted, its updates to the Major Facilities Guide now also describe how sensitivity analysis should be applied to major facilities' cost estimates.

    Recommendation: The Director of NSF should revise the agency's policies for estimating the costs of large facilities projects, and for reviewing those costs, to better incorporate the best practices in GAO's cost guide. (Recommendation 1)

    Agency Affected: National Science Foundation

  2. Status: Open

    Comments: NSF agreed with this recommendation and, as of November 2019, had updated its internal guidance to include a new section related to schedule reviews. The updated guidance states that the NSF Large Facilities Office will lead analysis of the schedule for each proposed major facilities project, which will include a technical evaluation by the sponsoring office. As further steps to implement this recommendation, NSF planned to develop (1) a new section of the Major Facilities Guide on schedule development, estimating, and analysis and (2) new internal guidance on including project schedules as part of external panels' oversight reviews. NSF anticipated completing these actions by mid-fiscal year 2020. We will continue to monitor and provide updates on NSF's actions to address this recommendation.

    Recommendation: The Director of NSF should revise the agency's policies for developing schedules for large facilities projects, and for reviewing those schedules, to better incorporate the best practices in GAO's schedule guide. (Recommendation 2)

    Agency Affected: National Science Foundation

 

Explore the full database of GAO's Open Recommendations »

Apr 3, 2020

Jan 7, 2020

Dec 9, 2019

Dec 5, 2019

Dec 4, 2019

Nov 21, 2019

Sep 16, 2019

Looking for more? Browse all our products here