2010 Census:

Census Bureau's Decision to Continue with Handheld Computers for Address Canvassing Makes Planning and Testing Critical

GAO-08-936: Published: Jul 31, 2008. Publicly Released: Sep 2, 2008.

Additional Materials:

Contact:

Mathew J. Scire
(202) 512-6794
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The U.S. Census Bureau (Bureau) had planned to rely heavily on automation in conducting the 2010 Census, including using handheld computers (HHC) to verify addresses. Citing concerns about escalating costs, in March 2008 the Secretary of Commerce announced a redesign of the key automation effort. GAO was asked to (1) analyze Bureau and contractor data showing how HHCs operated and their impact on operations, and (2) examine implications the redesign may have on plans for address canvassing in the 2010 Census. GAO reviewed Bureau and contractor data, evaluations, and other documents on HHC performance and staff productivity; interviewed Bureau and contractor officials; and visited the two dress rehearsal sites to observe and document the use of the HHCs in the field.

Census and contractor data highlight problems field staff (listers) experienced using HHCs during the address canvassing dress rehearsal operation in 2007. Help desk logs, for example, revealed that listers most frequently reported issues with transmission, the device freezing, mapspotting (collecting mapping coordinates), and difficulties working with large blocks. When problems were identified, the contractor downloaded corrected software to the HHCs. Nonetheless, help desk resources were inadequate. The Bureau acknowledged that issues with the use of technology affected field staff productivity. After address canvassing, the Bureau established a review board and worked with its contractor to create task teams to analyze and address Field Data Collection Automation (FDCA) performance issues. Although the Bureau recognized that technology issues affected operations, and the contractor produced data on average transmission times, the Bureau and its contractor did not fully assess the magnitude of key measures of HHC performance. GAO previously recommended the Bureau establish specific quantifiable measures in such areas as productivity and performance. Also, the FDCA contract calls for the contractor to provide near real-time monitoring of performance metrics through a "dashboard" application. This application was not used during the census dress rehearsal. The Bureau has developed a preliminary list of metrics to be included in the dashboard such as daily measures on average transmission duration and number of failed transmissions, but has few benchmarks for expected performance. For example, the Bureau has not developed an acceptable level of performance on total number of failed transmissions or average connection speed. Technology issues and the Bureau's efforts to redesign FDCA have significant implications for address canvassing. Among these are ensuring that FDCA solutions for technical issues identified in the dress rehearsal are tested, the help desk adequately supports field staff, and a solution for conducting address canvassing in large blocks is tested. In June 2008, the Bureau developed a testing plan that includes a limited operational field test, but the plan does not specify the basis for determining the readiness of the FDCA solution for address canvassing and when and how this determination will occur.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: The Bureau specified the benchmarks for measures used to evaluate the HHC performance during address canvassing as we recommended. This was documented in an excel spreadsheet which laid out both performance requirements and benchmarks, such as: The Field Data Collection Automation (FDCA) shall distribute software and data updates to all FDCA MCE devices currently in operation in field in less than 24 hours for a 500K bytes download and in less than 48 hours for a complete software download; and The recharge time for an HHC device shall be less than 8 (TBD) hours.

    Recommendation: To ensure that the Bureau addresses key challenges facing its implementation of the address canvassing operation for the 2010 Census, the Secretary of Commerce should direct the Bureau to specify the benchmarks for measures used to evaluate the HHC performance during address canvassing.

    Agency Affected: Department of Commerce

  2. Status: Closed - Implemented

    Comments: The Census Bureau acted on our recommendation to specify how data collection in large blocks will be conducted in parallel with the address canvassing operation, and how this dual-track will be tested in order to ensure it will function as planned. On September 3, 2008 the Bureau laid out its testing plans that described how the large block canvassing operation would be conducted ("Census 2010: 2010 Census Address Canvassing Large Block Testing Plan" by James Hartman and Robert Colosi).

    Recommendation: To ensure that the Bureau addresses key challenges facing its implementation of the address canvassing operation for the 2010 Census, the Secretary of Commerce should direct the Bureau to specify how data collection in large blocks will be conducted in parallel with the address canvassing operation, and how this dual-track will be tested in order to ensure it will function as planned.

    Agency Affected: Department of Commerce

  3. Status: Closed - Implemented

    Comments: The Bureau followed GAO's recommendation that the Secretary of Commerce direct the Bureau to specify the basis for determining the readiness of the Field Data Collection Automation (FDCA) solution for address canvassing and when and how this determination will occur--when the Bureau would say that the contractor's solution meets its operational needs. They did so by specifying readiness criteria associated with the HHCs for address canvassing and established that the basis for determining the readiness criteria would be an assessment of each of the core requirements supporting the deployment and conduct of address canvassing. The desired performance requirements for handheld computers were established and final agreement was reached with Harris (the contractor) in September 2008. The review of Harris' technical performance was done on January 21, 2009 to determine their readiness to deploy on February 3, 2009. The Bureau said that Harris was ready and gave concurrence to go into operation on February 3.

    Recommendation: To ensure that the Bureau addresses key challenges facing its implementation of the address canvassing operation for the 2010 Census, the Secretary of Commerce should direct the Bureau to specify the basis for determining the readiness of the FDCA solution for address canvassing and when and how this determination will occur--when the Bureau would say that the contractor's solution meets its operational needs.

    Agency Affected: Department of Commerce

  4. Status: Closed - Implemented

    Comments: During the December 2008 Address Canvassing Operational Field Test, the contractor used the enterprise management system to monitor performance in their Network Operations Center/Security Operations Center in Reston, VA--this system is part of the dashboard where connectivity and system performance was monitored.

    Recommendation: To ensure that the Bureau addresses key challenges facing its implementation of the address canvassing operation for the 2010 Census, the Secretary of Commerce should direct the Bureau to use the dashboard to monitor performance of the HHCs in the operational field test of address canvassing.

    Agency Affected: Department of Commerce

 

Explore the full database of GAO's Open Recommendations »

Sep 20, 2018

Sep 17, 2018

Sep 12, 2018

Sep 6, 2018

Sep 5, 2018

Aug 30, 2018

Aug 27, 2018

Aug 15, 2018

Jul 30, 2018

Jul 26, 2018

Looking for more? Browse all our products here