Skip to main content

Census 2000: Design Choices Contributed to Inaccuracy of Coverage Evaluation Estimates

GAO-05-71 Published: Nov 12, 2004. Publicly Released: Dec 13, 2004.
Jump To:
Skip to Highlights

Highlights

Evaluations of past censuses show that certain groups were undercounted compared to other groups, a problem known as "coverage error." To address this, the Census Bureau included in its 2000 Census design the Accuracy and Coverage Evaluation Program (A.C.E.) to (1) measure coverage error and (2) use the results to adjust the census, if warranted. However, the Bureau found the A.C.E. results inaccurate and decided not to adjust or plan for adjustment in 2010. Congress asked GAO to determine (1) factors contributing to A.C.E.'s reported failure to accurately estimate census coverage error, and (2) the reliability of the revised coverage error estimates the Bureau subsequently produced. To do this, GAO examined three sets of Bureau research published in March 2001, October 2001, and March 2003 and interviewed Bureau officials.

Recommendations

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Commerce As the Bureau plans for its coverage evaluation of the next national head count in 2010 and to ensure that coverage evaluation results the Bureau disseminates are as useful as possible to Congress and other census stakeholders, the Secretary of Commerce should direct that to avoid creating any unnecessary blind spots in the 2010 coverage evaluation, as the Bureau plans for its coverage evaluation in 2010, it should take into account how any significant future design decisions relating to census (for example, residence rules, efforts to detect and reduce duplicates, or other procedures) or A.C.E. (for example, scope of coverage, and changes in search areas, if applicable), or their interactions, could affect the accuracy of the program.
Closed – Implemented
According to the Department of Commerce, the developmental efforts for the Census Coverage Measurement (CCM) program have been evolving over the years leading up to the 2010 census, and won't be complete until early 2010. The documentation of these research efforts and design choices are occurring on a flow basis as various stages of work are completed. To ensure that all design changes to the Census are known to all affected teams, the Bureau has created Operational Integration Teams (OIT) around major programs, like CCM, comprising subject matter experts and representatives of each census division with responsibility for some part of the program. The OIT monitors implementation and discusses all changes needing to be made affecting its program, ensuring two-way communication to and from the OIT and the divisions with their work affected. Representatives of the CCM OIT have cited as examples of issues they have addressed: redefining and applying residence rules properly; introducing the Coverage Follow-Up operation before CCM Person Interviewing; and workarounds to obtaining map spots on military bases. The creation of these OIT teams and their prominent role in both developing and monitoring the design of census and CCM operations, combine to address this recommendation.
Department of Commerce As the Bureau plans for its coverage evaluation of the next national head count in 2010 and to ensure that coverage evaluation results the Bureau disseminates are as useful as possible to Congress and other census stakeholders, the Secretary of Commerce should direct that, in the future, the Bureau should clearly report in its evaluation of A.C.E. how any significant changes in the design of census and/or A.C.E. might have affected the accuracy of the coverage error estimates.
Closed – Implemented
According to the Department of Commerce, specific dates on when Commerce will have a final evaluation plan for Census Coverage Measurement (CCM) have not been determined. Bureau officials have stated that they are open to conducting additional evaluations that seem warranted or appropriate as they learn more during the census as well. Nonetheless, the Bureau has committed to carrying out a series of evaluations of each of the major CCM operations and will be looking at specific changes in the design of the programs and their estimation methods. Officials mentioned the following topics as subjects of their planned evaluation reviews: changes in search areas, consistency of residence rules, changes in the CCM Person Interview operation, errors attributable to reliance on administrative records, and similarities in the wording of forms for Coverage Follow-Up and CCM Person Interviewing. Planning to evaluate such specific changes that might impact the accuracy of estimates of census coverage implements this recommendation.
Department of Commerce As the Bureau plans for its coverage evaluation of the next national head count in 2010 and to ensure that coverage evaluation results the Bureau disseminates are as useful as possible to Congress and other census stakeholders, the Secretary of Commerce should direct that, in the future, the Bureau plan to not only identify but report, where feasible, the potential range of impact of any significant methodological limitation on published census coverage error estimates. When the impact on accuracy is not readily quantifiable, the Bureau should include clear statements disclosing how it could potentially affect how people interpret the accuracy of the census or A.C.E.
Closed – Implemented
Bureau executives and Census Coverage Measurement (CCM) program managers said that they were well aware of the need for being clear about the impact of limitations on evaluation results. They also noted the various challenges in itemizing all possible non-sampling errors and how expensive it can be trying to measure all of them. Within these constraints, however, the Bureau has developed an approach to improve such measurement where feasible, including a candid discussion of how the limitations could affect the utility of the evaluation results and specific information on serious flaws they uncover in their CCM methodologies. Although the CCM results are to be released after the 2010 Census, because the Bureau has a mechanism in place to improve the transparency of the limitation of the evaluation results, the Bureau's efforts address this recommendation.

Full Report

GAO Contacts

Office of Public Affairs

Topics

CensusComputer matchingData collectionData integrityErrorsEvaluation methodsProgram evaluationStatistical dataHispanic AmericansData errors