2010 Census:
Census Bureau's Decision to Continue with Handheld Computers for Address Canvassing Makes Planning and Testing Critical
GAO-08-936: Published: Jul 31, 2008. Publicly Released: Sep 2, 2008.
Additional Materials:
- Highlights Page:
- Full Report:
- Accessible Text:
Contact:
(202) 512-6794
contact@gao.gov
Office of Public Affairs
(202) 512-4800
youngc1@gao.gov
The U.S. Census Bureau (Bureau) had planned to rely heavily on automation in conducting the 2010 Census, including using handheld computers (HHC) to verify addresses. Citing concerns about escalating costs, in March 2008 the Secretary of Commerce announced a redesign of the key automation effort. GAO was asked to (1) analyze Bureau and contractor data showing how HHCs operated and their impact on operations, and (2) examine implications the redesign may have on plans for address canvassing in the 2010 Census. GAO reviewed Bureau and contractor data, evaluations, and other documents on HHC performance and staff productivity; interviewed Bureau and contractor officials; and visited the two dress rehearsal sites to observe and document the use of the HHCs in the field.
Census and contractor data highlight problems field staff (listers) experienced using HHCs during the address canvassing dress rehearsal operation in 2007. Help desk logs, for example, revealed that listers most frequently reported issues with transmission, the device freezing, mapspotting (collecting mapping coordinates), and difficulties working with large blocks. When problems were identified, the contractor downloaded corrected software to the HHCs. Nonetheless, help desk resources were inadequate. The Bureau acknowledged that issues with the use of technology affected field staff productivity. After address canvassing, the Bureau established a review board and worked with its contractor to create task teams to analyze and address Field Data Collection Automation (FDCA) performance issues. Although the Bureau recognized that technology issues affected operations, and the contractor produced data on average transmission times, the Bureau and its contractor did not fully assess the magnitude of key measures of HHC performance. GAO previously recommended the Bureau establish specific quantifiable measures in such areas as productivity and performance. Also, the FDCA contract calls for the contractor to provide near real-time monitoring of performance metrics through a "dashboard" application. This application was not used during the census dress rehearsal. The Bureau has developed a preliminary list of metrics to be included in the dashboard such as daily measures on average transmission duration and number of failed transmissions, but has few benchmarks for expected performance. For example, the Bureau has not developed an acceptable level of performance on total number of failed transmissions or average connection speed. Technology issues and the Bureau's efforts to redesign FDCA have significant implications for address canvassing. Among these are ensuring that FDCA solutions for technical issues identified in the dress rehearsal are tested, the help desk adequately supports field staff, and a solution for conducting address canvassing in large blocks is tested. In June 2008, the Bureau developed a testing plan that includes a limited operational field test, but the plan does not specify the basis for determining the readiness of the FDCA solution for address canvassing and when and how this determination will occur.
Recommendations for Executive Action
Status: Closed - Implemented
Comments: The Bureau followed GAO's recommendation that the Secretary of Commerce direct the Bureau to specify the basis for determining the readiness of the Field Data Collection Automation (FDCA) solution for address canvassing and when and how this determination will occur--when the Bureau would say that the contractor's solution meets its operational needs. They did so by specifying readiness criteria associated with the HHCs for address canvassing and established that the basis for determining the readiness criteria would be an assessment of each of the core requirements supporting the deployment and conduct of address canvassing. The desired performance requirements for handheld computers were established and final agreement was reached with Harris (the contractor) in September 2008. The review of Harris' technical performance was done on January 21, 2009 to determine their readiness to deploy on February 3, 2009. The Bureau said that Harris was ready and gave concurrence to go into operation on February 3.
Recommendation: To ensure that the Bureau addresses key challenges facing its implementation of the address canvassing operation for the 2010 Census, the Secretary of Commerce should direct the Bureau to specify the basis for determining the readiness of the FDCA solution for address canvassing and when and how this determination will occur--when the Bureau would say that the contractor's solution meets its operational needs.
Agency Affected: Department of Commerce
Status: Closed - Implemented
Comments: The Census Bureau acted on our recommendation to specify how data collection in large blocks will be conducted in parallel with the address canvassing operation, and how this dual-track will be tested in order to ensure it will function as planned. On September 3, 2008 the Bureau laid out its testing plans that described how the large block canvassing operation would be conducted ("Census 2010: 2010 Census Address Canvassing Large Block Testing Plan" by James Hartman and Robert Colosi).
Recommendation: To ensure that the Bureau addresses key challenges facing its implementation of the address canvassing operation for the 2010 Census, the Secretary of Commerce should direct the Bureau to specify how data collection in large blocks will be conducted in parallel with the address canvassing operation, and how this dual-track will be tested in order to ensure it will function as planned.
Agency Affected: Department of Commerce
Status: Closed - Implemented
Comments: The Bureau specified the benchmarks for measures used to evaluate the HHC performance during address canvassing as we recommended. This was documented in an excel spreadsheet which laid out both performance requirements and benchmarks, such as: The Field Data Collection Automation (FDCA) shall distribute software and data updates to all FDCA MCE devices currently in operation in field in less than 24 hours for a 500K bytes download and in less than 48 hours for a complete software download; and The recharge time for an HHC device shall be less than 8 (TBD) hours.
Recommendation: To ensure that the Bureau addresses key challenges facing its implementation of the address canvassing operation for the 2010 Census, the Secretary of Commerce should direct the Bureau to specify the benchmarks for measures used to evaluate the HHC performance during address canvassing.
Agency Affected: Department of Commerce
Status: Closed - Implemented
Comments: During the December 2008 Address Canvassing Operational Field Test, the contractor used the enterprise management system to monitor performance in their Network Operations Center/Security Operations Center in Reston, VA--this system is part of the dashboard where connectivity and system performance was monitored.
Recommendation: To ensure that the Bureau addresses key challenges facing its implementation of the address canvassing operation for the 2010 Census, the Secretary of Commerce should direct the Bureau to use the dashboard to monitor performance of the HHCs in the operational field test of address canvassing.
Agency Affected: Department of Commerce
Explore the full database of GAO's Open Recommendations
»
Mar 2, 2021
-
High-Risk Series:
Dedicated Leadership Needed to Address Limited Progress in Most High-Risk AreasGAO-21-119SP: Published: Mar 2, 2021. Publicly Released: Mar 2, 2021. -
High-Risk Series:
Dedicated Leadership Needed to Address Limited Progress in Most High-Risk AreasGAO-21-383T: Published: Mar 2, 2021. Publicly Released: Mar 2, 2021. -
High-Risk Series:
Dedicated Leadership Needed to Address Limited Progress in Most High-Risk AreasGAO-21-384T: Published: Mar 2, 2021. Publicly Released: Mar 2, 2021.
Feb 3, 2021
-
Fixed-Price-Incentive Contracts:
DOD Has Increased Their Use but Should Assess Contributions to OutcomesGAO-21-181: Published: Feb 3, 2021. Publicly Released: Feb 3, 2021.
Jan 29, 2021
-
Federal Real Property:
Additional Documentation of Decision Making Could Improve Transparency of New Disposal ProcessGAO-21-233: Published: Jan 29, 2021. Publicly Released: Jan 29, 2021.
Jan 19, 2021
-
Federal Rulemaking:
Selected EPA and HHS Regulatory Analyses Met Several Best Practices, but CMS Should Take Steps to Strengthen Its AnalysesGAO-21-151: Published: Dec 17, 2020. Publicly Released: Jan 19, 2021.
Jan 13, 2021
-
Department of Energy Contracting:
Improvements Needed to Ensure DOE Assesses Its Full Range of Contracting Fraud RisksGAO-21-44: Published: Jan 13, 2021. Publicly Released: Jan 13, 2021.
Dec 16, 2020
-
Data Governance:
Agencies Made Progress in Establishing Governance, but Need to Address Key MilestonesGAO-21-152: Published: Dec 16, 2020. Publicly Released: Dec 16, 2020.
Dec 9, 2020
-
2020 Census:
The Bureau Concluded Field Work but Uncertainty about Data Quality, Accuracy, and Protection RemainsGAO-21-206R: Published: Dec 9, 2020. Publicly Released: Dec 9, 2020.
Dec 3, 2020
-
2020 Census:
Census Bureau Needs to Assess Data Quality Concerns Stemming from Recent Design ChangesGAO-21-142: Published: Dec 3, 2020. Publicly Released: Dec 3, 2020.
Looking for more? Browse all our products here