2020 Census:

Bureau Needs to Better Leverage Information to Achieve Goals of Reengineered Address Canvassing

GAO-17-622: Published: Jul 20, 2017. Publicly Released: Jul 20, 2017.

Additional Materials:

Contact:

Robert Goldenkoff
(202) 512-2757
goldenkoffr@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

For the 2020 Census, the Census Bureau originally planned to phase in technology and third-party data sources to build its address list—reducing the need to go door-to-door. The Bureau estimated it could update 75% of its address list this way, reducing costs by $900 million.

However, the Bureau has not evaluated the accuracy of its approach, which involves comparing aerial images to data in the master address list. It also suspended the operation's second phase—resolving mismatches—without detailing the cost, value, or accuracy of the new approach.

We made 3 recommendations to help the Bureau tackle these issues prior to the 2020 Census.

The Bureau’s initial plan for in-office and in-field canvassing to update its Master Address File

Flow chart illustrating Bureau's two-phased in-office address canvassing approach.

Flow chart illustrating Bureau's two-phased in-office address canvassing approach.

Additional Materials:

Contact:

Robert Goldenkoff
(202) 512-2757
goldenkoffr@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

To achieve cost savings, the Census Bureau (Bureau) decided in September 2014 to use a reengineered approach for building its address list for the 2020 Census without having to go door-to-door (or “in-field”) across the country, as it has in prior decennial censuses. Rather, some areas (known as “blocks”) might only need a review of their address and map information using computer imagery and third-party data sources – what the Bureau calls “in-office” procedures. The Bureau planned to use a two-phase, in-office address canvassing operation to update 75 percent of all housing units in its master address list, and initially estimated this approach could reduce 2020 Census costs by $900 million.

The Bureau has not completed evaluations of results from tests and activities it carried out in 2016 to assess the effectiveness of in-office address canvassing. Specifically, the Bureau has not completed its evaluations designed to evaluate the accuracy of in-office address canvassing. Similarly, citing budget constraints, the Bureau also cancelled 2017 fieldwork that could have provided additional data to evaluate the reengineered approach. Further, the Bureau has changed the design of the in-office address canvasing operation with limited information on the cost and quality implications of the changes. Citing budget uncertainty, in March 2017 the Bureau suspended its second phase of in-office address canvassing, which was designed to resolve address coverage issues identified in the first phase. Bureau officials indicated that this decision would increase the in-field address canvassing workload from 25 percent to at least 30 percent, but they could not provide details of the cost or quality tradeoffs of this decision. Using its remaining evaluations to determine cost and quality implications would better position the Bureau to justify future decisions for its reengineered approach.

GAO has previously reported on the need for the Bureau to rethink its approach to testing and evaluating the census. The Bureau agreed, and its early plans called for the use of incremental, small-scale testing throughout the decade. As the Bureau begins planning for future census operations, small-scale testing by the Bureau may be less susceptible to budgetary constraints and allow it to better demonstrate the effectiveness of its operational innovations.

The Bureau manages the day-to-day production of in-office address canvassing using measures that do not fully reflect the effectiveness of the approach in reducing fieldwork. For example, the Bureau regularly tracks the number of blocks for which it has conducted an initial round of review. However, the workload the Bureau identifies as completed includes blocks that have been put on hold and that await availability of better information before being reviewed again. Other assumptions about how quickly the workflow proceeds have not been met. In testing the Bureau's productivity assumptions using actual production data through March 2017, GAO found that even small differences in measurement can have significant impacts on estimates of the remaining workload and the time to complete it. Without measuring how much in-office work is truly completed and how much fieldwork has been reduced, the Bureau risks underestimating the time and resources needed to complete the in-office operation, and may require more in-field address canvassing—at greater costs—than the Bureau currently projects.

Why GAO Did This Study

With a lifecycle cost of $12.3 billion (in 2020 dollars), the 2010 Census was the most expensive in U.S. history. Reengineering how the Bureau builds its address list is one of the ways the Bureau intends to reduce the per–housing unit cost of the 2020 count.

GAO was asked to evaluate the Bureau's reengineered approach for 2020 address canvassing. This report (1) describes the Bureau's design for 2020 address canvassing, (2) evaluates the extent to which the Bureau assessed the cost and quality implications of its reengineered address canvassing approach, and (3) assesses the status of the Bureau's efforts to reduce the in-field address canvassing workload. GAO reviewed relevant design and testing documentation and interviewed cognizant Bureau officials. GAO also reviewed Bureau address canvassing production and payroll data in December 2016 and March 2017.

What GAO Recommends

GAO recommends that the Bureau (1) use its remaining evaluations before 2020 to determine cost and quality implications and to justify future decisions for its reengineered approach; (2) plan and execute smaller, more flexible tests needed to support key address canvassing design decisions in future census operations; and (3) use productivity measures that track the progress of the in-office address canvassing and its effectiveness in reducing fieldwork. The Department of Commerce had no disagreements with GAO's findings and recommendations. In addition, the Secretary noted that the Department was conducting its own review.

For more information, contact Robert Goldenkoff at (202) 512-2757 or goldenkoffr@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Secretary of Commerce should direct the Under Secretary of the Economics and Statistics Administration and the Director of the U.S. Census Bureau to use the Bureau's evaluations before 2020 to determine the implications of in-office address canvassing on the cost and quality of address canvassing, and use this information to justify decisions related to its re-engineered address canvassing approach.

    Agency Affected: Department of Commerce

  2. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: Early in the next decennial cycle, the Secretary of Commerce should direct the Under Secretary of the Economics and Statistics Administration and the Director of the U.S. Census Bureau to plan and execute more flexible, and perhaps smaller, address canvassing test and evaluation activity needed to support key design decisions having significant effect on the cost and quality of the census.

    Agency Affected: Department of Commerce

  3. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Secretary of Commerce should direct the Under Secretary of the Economics and Statistics Administration and the Director of the U.S. Census Bureau to use productivity measures that track the progress in completing in-office address canvassing workload and the effectiveness of in-office address canvassing in reducing fieldwork in order to make informed decisions on allocating resources to current and future address canvassing workload so that the operation is completed in a timely and cost-effective manner.

    Agency Affected: Department of Commerce

 

Explore the full database of GAO's Open Recommendations »

Oct 12, 2017

Oct 11, 2017

Oct 10, 2017

Sep 29, 2017

Sep 28, 2017

Sep 27, 2017

Looking for more? Browse all our products here