Census Bureau Needs to Improve Its Life-Cycle Cost Estimating Process
GAO-16-628: Published: Jun 30, 2016. Publicly Released: Jun 30, 2016.
- Highlights Page:
- Full Report:
- Accessible Version:
- Related WatchBlog Post:
- Related WatchBlog Post:
What GAO Found
Since 2012, the U.S. Census Bureau (Bureau) has taken significant steps to improve its capacity to carry out an effective cost estimate; however, its October 2015 cost estimate for the 2020 Census does not fully reflect characteristics of a high-quality estimate and cannot be considered reliable. To reflect these characteristics, an organization must meet or substantially meet four best practices. Overall, GAO found the cost estimate partially met the characteristics of two best practices (comprehensive and accurate) and minimally met the other two (well-documented and credible). One reason why GAO's overall assessment is low is because the estimate is not well-documented. Improving cost estimation practices will increase the reliability of the Bureau's cost estimate, which will in turn help improve decision making, budget formulation, progress measurement, course correction when warranted, and accountability for results.
Best practices state a risk and uncertainty analysis should be performed to determine the level of risk associated with the cost estimate. The Bureau carried out such an analysis only for a portion of estimated costs for fiscal years 2018 to 2020. According to Bureau officials, they scoped the analysis narrowly to those 3 years when most of the census costs occur. GAO found that, as a result, the Bureau's risk and uncertainty analysis (modeled costs) covered $4.6 billion, only about 37 percent of the $12.5 billion total estimated life-cycle cost, and less than one-half of the total estimated cost of the census during future fiscal years.
Components of Total 2020 Census Life-Cycle Cost Estimate
Note: All figures are in constant 2020 dollars.
The Bureau has risk identification processes, which identify a broad range of risks that could affect the cost of the 2020 Census. Yet this awareness of risk is not leveraged in the Bureau's cost estimation. The cost estimation team did not consult risk registers or examine specific risks directly for inclusion in the cost model or risk and uncertainty analysis. It was not known what risks, if any, had been accounted for in other data in the cost model. As a result, neither the Bureau nor GAO are able to determine with confidence what risks the Bureau is prepared to mitigate or address within its $12.5 billion cost estimate. Improving control over how risk is accounted for will improve confidence that the Bureau's budgeted contingencies are at appropriate levels.
Why GAO Did This Study
In October 2015, the Bureau estimated that with its new approach it can conduct the 2020 Census for $12.5 billion, $5 billion less than the $17.8 billion it estimated it would cost to repeat the design and methods of the 2010 Census. Reliable cost estimates can help an agency manage large complex activities like the 2020 Census, as well as help Congress make funding decisions and provide oversight.
GAO was asked to evaluate the reliability of the Bureau's life-cycle cost estimate. Among other objectives, this report assesses the extent to which (1) the Bureau's life-cycle cost estimate met GAO's best practices for cost estimation and (2) the Bureau identified and accounted for key risks facing the 2020 Census. To meet these objectives, GAO reviewed documentary and testimonial evidence from Bureau officials responsible for developing the 2020 Census cost estimate. GAO used its cost assessment guide ( GAO-09-3SP ) and Standards for Internal Control in the Federal Government ( GAO-14-704G ) as criteria.
What GAO Recommends
GAO is making three recommendations including that the Secretary of Commerce direct the Bureau to take specific steps to ensure its cost estimate meets the characteristics of a high-quality estimate and improve control over how risk and uncertainty are accounted for in cost estimation. The Department of Commerce agreed with GAO's recommendations and provided additional context that was incorporated, as appropriate.
For more information, contact Robert Goldenkoff at (202) 512-2757 or firstname.lastname@example.org.
Recommendations for Executive Action
Comments: Commerce agreed with this recommendation. The Bureau should provide a cost estimate more current than the October 2015 estimate and ensure that the estimate is comprehensive, well-documented, accurate, and credible. In doing this the Bureau should consult the GAO's cost assessment guide (GAO-09-3SP) and Standards for Internal Control in the Federal Government (GAO-14-704G). High-quality estimates will: explicitly consider all life-cycle costs and assumptions, offer a clear step-by-step account of the methods and data sources used to compile the estimate, ensure the proper estimation techniques are used, reconcile any variances between actual and estimated costs, and allow cross-checking with independent cost estimates as verification of results. In August 2016, the Bureau laid out its action plan to implement this recommendation. The Bureau planned to develop a Cost Estimation Enhancement Plan that would mature the 2020 Census cost estimate and its associated processes via a series of 3-month sprints. According to the Bureau, the areas targeted for improvement were (I) Documentation, (2) Process, (3) Cost Estimate, and (4) Cost Integration. The Bureau's action plan reported the following deliverables: Incorporating the Decennial Census Management Division program work breakdown structure into the 2020 Census Cost Estimate (target completion was Q4 FY 2016); developing a formal basis of estimate document to address the cost elements, process flow, and calculations for the 2020 Census Cost Estimate (Q2 FY 2017); internal communication and training efforts to ensure these changes are widely shared and communicated (Q2 FY 2017); engaging with internal stakeholders to increase the amount of source and derivation documentation for estimates/model parameters currently based on expert judgment (Q4 FY 2016); developing a formal BOE document to address how 2020 Census program risk and uncertainty are dealt with in the 2020 Census Cost Estimate (Q2 FY 2017); and regularly comparing the results of the independent cost estimate conducted by the Office of Cost Estimation, Analysis and Assessment to the 2020 Census Cost Estimate and investigate/reconcile any significant differences (Q3 FY 2017). As of July 2017, we await this and other documentation from Bureau that may address this recommendation.
Recommendation: To help ensure the Bureau produces a reliable cost estimate for the 2020 Census, the Secretary of Commerce and Under Secretary for Economic Affairs should direct the Census Bureau to take the following steps to meet the characteristics of a high-quality estimate: (1) Comprehensive--among other practices, ensure the estimate includes all life-cycle costs and documents all cost-influencing assumptions. (2) Well-documented--among other practices, ensure that its planned documentation plan captures the source data used; contains the calculations performed and the estimating methodologies used for each element; and describes step by step how the estimate was developed. (3) Accurate--among other practices, ensure the estimating technique for each cost element is used appropriately and that variances between planned and actual cost are documented, explained, and reviewed. (4) Credible--among other practices, ensure the estimate includes a sensitivity analysis, major cost elements are cross-checked to see whether results are similar, and an independent cost estimate is conducted to determine whether other estimating methods produce similar results.
Agency Affected: Department of Commerce
Comments: Commerce agreed with this recommendation. The Bureau should implement processes for controlling and changing cost assumptions. These processes should include methods for evaluating the justification for any changes and documentation requirements that clearly show the information changed and the basis for the change. In August 2016, Bureau officials laid out their action plan to address this recommendation. The action plan described developing a Decennial Census Cost Estimation and Analysis Process and supporting policy to improve the maturity levels in this area and mentioned developing a draft internal communication and training plan for staff--target date is Q2 FY 2017. As of July 2017, we await this and other documentation from Bureau that may address this recommendation.
Recommendation: To further ensure the credibility of data used in cost estimation, the Secretary of Commerce and Under Secretary for Economic Affairs should direct the Census Bureau to establish clear guidance on when information for cost assumptions can and should be changed as well as the procedures for documenting such changes and traceable sources for information being used.
Agency Affected: Department of Commerce
Comments: Commerce agreed with this recommendation. The Bureau should ensure that its budget for contingencies reflects an accurate accounting of risk and uncertainty. In doing this, the Bureau should improve controls over risk and uncertainty accounting, ensure that risk accounting informs any relevant budgets and cost estimates, and institutionalize these controls by providing clear methods for their use. In August 2016, the Bureau laid out its action plan to implement this recommendation, describing that it would ensure regular review of 2020 Census program risks that would have high cost impacts if they occur and ensure estimates of these impacts are accounted for and documented in each iteration of the life-cycle cost estimates--target date is Q2 FY 2017. As of July 2017, we await documentation from Bureau that may address this recommendation.
Recommendation: To ensure Bureau and congressional confidence that the Bureau's budgeted contingencies are at appropriate levels, the Secretary of Commerce and Under Secretary for Economic Affairs should direct the Census Bureau to improve control over how risk and uncertainty are accounted for and communicated with the Bureau's decennial cost estimation process, such as by implementing and institutionalizing processes or methods for doing so with clear guidance.
Agency Affected: Department of Commerce