Additional Actions Could Strengthen Future Census Test Designs
GAO-14-26: Published: Oct 25, 2013. Publicly Released: Nov 21, 2013.
What GAO Found
The Census Bureau (Bureau) generally followed most key practices for a sound study plan in designing the three initial field tests. However, some practices were only partially followed. For example, the test designs varied for four practices related to design process management. Good management of the design process can help managers identify factors that can affect the quality of a test, such as potential risk of delays and challenges to the lines of communication. For example, the Bureau generally followed one of the practices for design process management--identifying clear reporting relationships--for only one of the test designs. The Bureau partially followed this practice for another test design, and did not follow it for the third.
The Bureau has already begun incorporating lessons learned from its initial field test designs. These lessons include obtaining internal expert review, and conducting reviews after each test to learn additional lessons. The Bureau has also recognized the importance of keeping design team leaders informed about key design elements. Yet the Bureau has not finalized planned revisions to the team leader handbook, which could help implement this lesson.
Additionally, the Bureau is realigning field test governance structures to improve communication and accountability. It has already taken such steps as identifying one point of contact for each test. However, GAO found that the Bureau needs to set timelines and milestones to formalize other restructuring proposals for managing field tests, such as creating a field test management team. Having a formalized proposal and guidance revisions will better position the Bureau to improve accountability, communication, and the monitoring of its test design processes. While lessons the Bureau identified should help it better design future field tests, it has not consistently documented these lessons learned. Documenting lessons can help reduce the risk of repeating prior deficiencies that may lead to test development delays, and can reinforce lessons learned. Given the long time frames involved in planning the census, documentation is essential to ensure lessons are incorporated into future tests.
Why GAO Did This Study
The Bureau is continuing its early testing efforts to prepare for the decennial. These tests must be well designed to produce useful information about how to implement the 2020 Census. The Bureau has completed the designs of three field tests. GAO was asked to monitor the Bureau's testing for the 2020 Census.
This report (1) determines the extent to which the Bureau followed key practices for a sound study plan in designing the earliest 2020 Decennial Census field tests, and (2) identifies what lessons were learned that may help improve future tests. To meet these objectives, GAO first selected 25 key practices for a sound research plan after reviewing its program evaluation literature. GAO then compared Bureau field test design documents for its three initial tests to these practices. GAO also examined where the Bureau had not followed key practices, identified actions needed to address them, and interviewed officials about lessons learned.
What GAO Recommends
GAO recommends that the Secretary of Commerce (1) finalize field test management revisions in the team leader handbook, (2) set a timeline and milestones for formalizing proposed field test management restructuring and guidance revisions, and (3) document lessons learned from designing initial field tests. The Department of Commerce concurred with GAO's findings and recommendations, and provided minor technical comments, which were included in the final report.
For more information, contact Robert Goldenkoff at (202) 512-2757 or firstname.lastname@example.org.
Recommendation for Executive Action
Status: Closed - Implemented
Comments: In response to our audit, the Bureau took steps to improve its management of field tests. Pursuant to a set schedule, in October 2016, the Bureau completed a reorganization of its decennial directorate staff and according to Bureau officials realigned the way it had historically done business by conducting a close examination of the core functions, operational needs, and resource requirements to inform the new structure. In the same timeframe the Bureau expanded its reliance on a shared document system, which, with the reorganization, updated its management and guidance of field tests and replaced the need for the prior handbook. In May 2017, Bureau officials provided a charter for their Decennial Research Objectives and Methods Working Group which provides guidance on the formulation of research programs, including their detailed plans and reporting. Minutes of meetings for the working group indicate the group is focused on leveraging lessons from earlier tests to later ones. These steps will help improve field tests designs.
Recommendation: To improve the Bureau's process of designing its census field tests for the 2020 Census, the Secretary of Commerce should require the Under Secretary for Economic Affairs who oversees the Economics and Statistics Administration, as well as the Director of the U.S. Census Bureau, to (1) finalize planned revisions that focus on field test management in the team leader handbook, (2) set a timeline and milestones for formalizing proposed field test management restructuring and guidance revisions, and (3) document lessons learned from designing initial field tests.
Agency Affected: Department of Commerce