2010 Census:

Data Collection Operations Were Generally Completed as Planned, but Long-standing Challenges Suggest Need for Fundamental Reforms

GAO-11-193: Published: Dec 14, 2010. Publicly Released: Dec 14, 2010.

Additional Materials:

Contact:

Robert N. Goldenkoff
(202) 512-2757
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Although the U.S. Census Bureau (Bureau) generally completed the field data collection phase of the 2010 Census consistent with its operational plans, at $13 billion, 2010 was the costliest census in the nation's history. Moving forward, it will be important to both refine existing operations as well as to reexamine the fundamental approach to the census to better address long-standing issues such as securing participation and escalating costs. As requested, this report reviews (1) the conduct of nonresponse follow-up (NRFU), where enumerators collect data from households that did not return their census forms, (2) the implementation of other field operations critical to a complete count, and (3) potential reexamination areas that could help produce a more cost-effective 2020 Census. The report is based on GAO's analysis of Bureau data and documents, surveys of local census office managers, and field observations.

Nationally, the Bureau was well positioned to implement NRFU and subsequent field operations. The Bureau achieved a mail response rate of 63 percent, which was within its expectations, and recruited nearly 3.8 million total applicants for census jobs, which was 104 percent of its staffing goal. Moreover, the Bureau completed NRFU under budget, reportedly spending $1.59 billion on the operation, about $660 million (29 percent) less than the Bureau initially estimated. Most of the Bureau's local census offices (LCO) also completed NRFU ahead of the 10-week allotted time frame. Despite these operational successes, the Bureau encountered some notable challenges. For example, the pace of NRFU may have fostered a culture that tended to emphasize speed over quality, as those LCOs with higher percentages of less-complete questionnaires were more likely to have completed NRFU in 53 days or less (the average time LCOs took to complete NRFU). The Bureau also had to overcome issues with critical information technology (IT) systems. For example, performance problems with the IT system used to manage NRFU led to processing backlogs. Although the Bureau developed workarounds for the issue, it hindered the Bureau's ability to fully implement quality-assurance procedures as planned. The Bureau generally completed other follow-up operations designed to improve the accuracy of the data consistent with its plans. One of these activities was the vacant/delete check (VDC), where enumerators verified housing units thought to be vacant or nonexistent. The Bureau completed VDC two days ahead of schedule, but encountered duplicate addresses on the address list used for the operation, which could indicate a more systemic problem with the quality of the Bureau's address list. While it will be important to refine existing census-taking activities--many of which have been in place since 1970--results of prior censuses point to the fact that simply improving current methods will not bring about the reforms needed to control costs and maintain accuracy. The cost of conducting the census has, on average, doubled each decade since 1970. At the same time, because of demographic and attitudinal trends, securing a complete count has become an increasing challenge. As a result, a fundamental reexamination of the nation's approach to the census will be needed for a more cost-effective enumeration in 2020. Potential focus areas include new data collection methods; the tenure of the Census Director; and ensuring the Bureau's approaches to human-capital management, knowledge sharing, and other internal functions are aligned toward delivering more cost-effective outcomes. The Bureau recognizes that fundamental changes are needed and has already taken some important first steps, including developing a strategic plan. To help ensure the Bureau's efforts stay on track and to avoid problems it had in planning for prior censuses, it will be important for the Bureau to issue a comprehensive operational plan that includes performance goals, milestones, cost estimates, and other critical information that could be updated regularly. GAO recommends that the Census Director refine NRFU and other field follow-up efforts by, among other things, emphasizing quality as much as speed during NRFU and by incorporating best practices in its IT acquisition-management policy. To help ensure reform efforts stay on track, the Bureau should develop an operational plan that integrates performance, budget, and other information. The Department of Commerce generally agreed with GAO's findings and recommendations.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To help enhance the Census Bureau's performance and accountability, improve the transparency of the planning process, gauge whether the Bureau is on-track toward a more cost-effective 2020 Census, and foster greater public dialog about the census, the Bureau should develop an operational plan or road map for 2020 that integrates performance, budget, methodological, schedule, and other information that would be updated as needed and posted on the Bureau's Web site and other social media outlets, and develop a mechanism that allows for and harnesses input from census stakeholders and individuals.

    Agency Affected: Department of Commerce: Bureau of the Census

    Status: Open

    Comments: In its 'Business Plan for 2020' the Bureau has taken steps to develop a road map for 2020 that integrates performance, budget, methodological, schedule, and other information. In November 2012 the Bureau provided information regarding its new Life-cycle and Budget Planning team which they state will also address this recommendation. While the Census Bureau continues to work on developing an operational plan for the 2020 Census that integrates performance, budget, methodologies, schedule, and other information, it has not yet posted information to a website or social media outlet that allows for input from stakeholders and individuals. As of February 7, 2014, we continue to monitor the Bureau's progress in implementing this recommendation.

    Recommendation: To refine its approach to recruiting, the Census Bureau should evaluate current economic factors that are associated with and predictive of employee interest in census work, such as national and regional unemployment levels, and use these available data to determine the potential temporary workforce pool and adjust its recruiting approach.

    Agency Affected: Department of Commerce: Bureau of the Census

    Status: Open

    Comments: In Oct. 2011 the Bureau completed a 2010 Census Program for Evaluations and Experiments (CPEX) assessment on 2010 Recruiting and Hiring. The assessment included part of this recommendation verbatim. However, we are waiting for information on whether the approach for 2020 recruiting and hiring will adopt our recommendation. According to the Bureau, its research is focused on prioritized cost drivers and, as resources for additional research become available later in the decade, it will ramp up research in this and other areas. As of February 7, 2014, we continue to monitor the Bureau's progress in implementing this recommendation.

    Recommendation: To help ensure that the Census Bureau's procedures for NRFU result in the collection of high-quality data, the Bureau's procedures for the timely completion of NRFU should emphasize the collection of high-quality data and proper enumeration techniques as much as speed.

    Agency Affected: Department of Commerce: Bureau of the Census

    Status: Closed - Implemented

    Comments: During the research and testing phase for the 2020 Census, the Bureau has multiple efforts designed to improve the quality of data collection. These include evaluations, assessments, and quality profiles to measure the effectiveness and impact on data quality of the 2010 Census operations, and multiple research and testing projects which include components focused on improving data quality during field operations. Based on research they are developing procedures to ensure the collection of high-quality NRFU data. For example, the Improving Quality Control Team is researching methods to predict falsification at the enumerator level as well as, developing procedures to monitor LCOs and detect LCO-wide procedural violations and incidents of falsification.

    Recommendation: To improve the fingerprinting process of temporary workers, the Census Bureau should revise or modify training so that field staff are provided with numerous practice opportunities for collecting fingerprints prior to each operation.

    Agency Affected: Department of Commerce: Bureau of the Census

    Status: Open

    Comments: The 60-day letter concurs with this recommendation. The letter notes that the Bureau continuously worked to improve the fingerprinting process throughout the 2010 Census, increasing the duration of training from two to four hours, and increasing time for "hands-on" practice. As of November 2011, the Bureau was developing research and testing projects related to fingerprinting field staff for 2020. However, specific changes to the 2020 fingerprinting process will not be available until after research and testing are complete. Training modification will be addressed once a process of collecting fingerprints has completed. As of February 7, 2014, we continue to monitor the Bureau's progress in implementing this recommendation.

    Recommendation: To ensure that the Census Bureau improves its ability to manage future IT acquisitions, the Bureau should immediately establish and enforce a system-acquisition management policy that incorporates best practices in system- and software-acquisition management.

    Agency Affected: Department of Commerce: Bureau of the Census

    Status: Closed - Implemented

    Comments: Consistent with our recommendation, in December 2012 the Bureau's Enterprise System Development Lifecycle (SDLC) 2.0 was approved by the Census IT Project Review Governance Board. This plan includes the agile software development model. As a result, the SDLC should be more applicable and able to better guide a variety of software development projects at the bureau.

    Recommendation: To help ensure the Census Bureau compiles an accurate address list for VDC operations in 2020, the Bureau should research how duplicate addresses were inadvertently included in the VDC workload.

    Agency Affected: Department of Commerce: Bureau of the Census

    Status: Open

    Comments: The 60-day letter concurs with this recommendation, and notes that the Bureau is currently developing the research and development program for assessing the 2010 Census and planning the 2020 Census. The Census Bureau is in the process of creating a 2010 Census analysis file, which will facilitate research such as that included in this recommendation. As of February 7, 2014, we continue to monitor the Bureau's progress in implementing this recommendation.

    Aug 7, 2014

    Jul 30, 2014

    Jul 29, 2014

    Jul 22, 2014

    Jun 17, 2014

    Jun 11, 2014

    Jun 10, 2014

    May 28, 2014

    May 21, 2014

    May 12, 2014

    Looking for more? Browse all our products here