This is the accessible text file for GAO report number GAO-02-562 entitled '2000 Census: Refinements to Full Count Review Program Could Improve Future Data Quality' which was released on July 3, 2002. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States General Accounting Office: GAO: Report to Congressional Addressees: July 2002: 2000 Census: Refinements to Full Count Review Program Could Improve Future Data Quality: Contents: Letter: Results in Brief: Scope and Methodology: Background: The Numbers and Kinds of Issues Identified During Full Count Review: The Bureau Resolved Few Issues Prior to Releasing Public Law Data: Lessons Learned That Can Inform Future Data Clearance Reviews: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Disposition of Data Issues Identified During Full Count Review by State: Appendix II: Comments From the Department of Commerce: Appendix III: GAO Products on the Results of the 2000 Census and Lessons Learned for a More Cost-Effective Census in 2010: Tables: Table 1: Types of Issues Flagged During Full Count Review: Table 2: Disposition of Data Issues Identified During Full Count Review: [End of section] United States General Accounting Office: Washington, DC 20548: July 3, 2002: The Honorable Henry A. Waxman: Ranking Minority Member: Committee on Government Reform: House of Representatives: The Honorable William Lacy Clay, Jr. The Honorable Carolyn B. Maloney: House of Representatives: Demographic Full Count Review was one of a number of quality assurance programs the Bureau of the Census developed for the 2000 decennial headcount to help ensure the completeness and accuracy of census data. Analysts were to identify, investigate, and document suspected data discrepancies or “issues” in order to clear census data files and products for subsequent processing or public release. Bureau reviewers were to determine whether and how to correct the data by weighing quality improvements against time and budget constraints. According to bureau officials, because the bureau lacked sufficient staff to conduct Full Count Review on its own, it contracted out some of the analysts’ work to members of the Federal-State Cooperative Program for Population Estimates (FSCPE), an organization composed of state demographers that, since its inception in 1973, has worked with the bureau to ensure accurate state and local population estimates. The bureau believed that FSCPE members’ knowledge of the demographic characteristics of their respective states could help the bureau review data files and products, including politically sensitive apportionment and redistricting data files known as public law data. [Footnote 1] This letter responds to your request to review the FSCPE members’ participation in the Full Count Review program. As agreed with your offices, we provide information on (1) the number and kinds of data issues identified by FSCPE members and bureau analysts, (2) how the bureau used the information developed during Full Count Review, and (3) lessons learned from the conduct of Full Count Review that can help the bureau plan for the 2010 Census. This report is the latest in a series of reviews that examine the results of key census-taking operations and highlight opportunities for reform (see app. III for a list of products issued to date). Results in Brief: FSCPE members documented 1,402 data issues, which was about 29 percent of the 4,809 issues identified by both FSCPE and bureau analysts during Full Count Review. Of the 4,809 issues, 1,599 (33 percent) dealt with “group quarters,” where the location or population counts for prisons, nursing homes, dormitories, and other group living facilities differed from what analysts expected. Of the 1,599 group quarters issues, FSCPE members identified 567 (35 percent). Discrepancies relating to housing unit counts, population data, and demographic characteristics accounted for 1,150 issues (24 percent), 375 of which (33 percent) were identified by FSCPE members. The bureau was unable to classify 2,060 issues (43 percent) because of insufficient documentation. According to bureau officials, 5 of the 4,809 issues identified during Full Count Review were corrected prior to the December 31, 2000, release of apportionment data, and the April 1, 2001, release of redistricting data. All five were group quarters issues where the bureau had the correct population count for each facility but placed them in the wrong locations. The “misplaced” group quarters included a military base, a federal medical center, and multiple facilities at two prisons and a college. The bureau was able to correct these issues because FSCPE members identified them early in the Full Count Review process and thoroughly documented them so that they did not require additional research or field verification. Because the bureau lacked the time and field staff from its regional offices to do any further investigative work, the 4,804 remaining issues went unresolved prior to the release of the redistricting data. As a result, uncertainties surround the accuracy of the census data for the affected localities. Overall, of the 4,809 issues identified during Full Count Review, 4,267 (89 percent) were not subjected to further investigation by the bureau in large part because of insufficient documentation. The bureau plans to review 537 issues as part of a subsequent effort called the Count Question Resolution (CQR) program, which the bureau developed to respond to challenges to the census data brought by state, local, or tribal governments (see app. I for the disposition of Full Count Review data issues by state). The bureau’s preliminary plans for the 2010 Census include a Full Count Review program. Our examination identified several areas where improvements are warranted. Foremost among these is that it will be important for the bureau to investigate and resolve a larger number of issues prior to the release of the public law data. We found three factors that limited the bureau’s ability to do so for the 2000 Census. First, bureau officials said that correcting individual issues was beyond the scope of the Full Count Review program. They noted that the program was developed in February 1999, just 14 months prior to Census Day 2000 and, as a result, the bureau was unable to test the program, or integrate it with other census operations and units that could have investigated the issues and made corrections. Second, the bureau’s requirements for documenting data issues were not clearly defined. As a result, there was considerable variation in the quality of the documentation analysts used to support issues, and in a number of cases, the bureau had difficulty determining the precise nature of an issue, or if in fact an issue even existed. A third, and related item that limited the bureau’s ability to resolve a larger number of issues was the fact that the bureau had no mechanism for managing its workload. Unlike the CQR program where the bureau requires specific documentation before committing resources to investigate local issues, the Full Count Review program had no filter for screening submissions. Had the bureau first categorized issues based on the quality and precision of the documentation provided, the bureau could have prioritized its investigative workload and perhaps reconciled a larger number of data issues. Another area where there is room for improvement concerns the consistency and clarity of the Full Count Review program’s objectives. For example, training materials noted that one purpose of the Full Count Review program was to document issues and “fix what can be fixed.” However, this conflicts with statements from bureau managers that correcting issues was outside the scope of the Full Count Review program. The different messages may have raised FSCPE members’ expectations that the bureau was going to correct a larger number of data issues than it actually did. That the apportionment and redistricting data were released with around 4,800 unresolved data issues of unknown validity, magnitude, and impact is cause for concern. The bureau did not fully exploit the Full Count Review program’s potential, and missed an opportunity to verify and possibly improve the quality of the public law data. To help develop the Full Count Review program into a more effective tool for enhancing data quality in the future, as well as to make better use of FSCPE members’ input, we recommend that the Secretary of Commerce explore ways of reconciling a larger number of data issues prior to the release of public law data. Specific steps include: (1) planning the Full Count Review program early in the census cycle and testing procedures under conditions as close to the actual census as possible; (2) integrating the Full Count Review program with other census organizational units and operations to ensure the bureau has sufficient time and field support to investigate issues; (3) developing clear guidelines on the minimum documentation needed for the bureau to investigate individual data issues; (4) categorizing issues on the basis of the quality and precision of the documentation, and investigating first those issues that are best documented and thus more easily resolved, and; (5) exploring the feasibility of using staff from the bureau’s regional offices to help investigate data issues in the field prior to the release of public law data. The Secretary of Commerce should also ensure that the bureau clarifies and consistently communicates to FSCPE members the objectives of the Full Count Review program and how the bureau plans to use the information derived from it. The Secretary of Commerce forwarded written comments from the Bureau of the Census on a draft of this report (see app. II). The bureau concurred with all of our recommendations and had no comments on them. The bureau also provided minor technical corrections that we incorporated in our report as appropriate. Scope and Methodology: To obtain information on the number and kinds of issues identified by the FSCPE and bureau analysts and to determine how the bureau used the information developed during the Full Count Review program, we analyzed the work papers submitted by FSCPE members and other participants in the Full Count Review program. We also analyzed data from the bureau’s Count Review Information System, a database that the bureau used to track issues flagged during the review process. We did not independently verify the information it contained. To identify lessons learned for future improvements, we examined bureau training manuals, statements of work, process models, and other documents that described the objectives, processes, and decision-making criteria. We also reviewed the results of a survey the bureau conducted of FSCPE members that asked them to rate their experience with Full Count Review processes and tools, bureau staff, and the overall effectiveness of the Full Count Review program. In addition, we interviewed managers in the bureau’s Population Division and other officials responsible for implementing the Full Count Review program, as well as three FSCPE members. We performed our audit in Washington, D.C., and the bureau’s headquarters in Suitland, Maryland, between May 2001 and April 2002. Our work was done in accordance with generally accepted government auditing standards. On April 26, 2002, we requested comments on a draft of this report from the Secretary of Commerce. The Secretary forwarded the bureau’s written comments on June 11, 2002 (see app. II). We address them in the “Agency Comments and Evaluation” section of this report. Background: Accurate census results are critical because the data are used to reapportion seats in the House of Representatives and for congressional redistricting. Moreover, census data remain an important element in allocating federal aid to state and local governments. With billions of dollars at stake, the data are scrutinized intensely for accuracy. To help ensure the accuracy of census data, the bureau conducted a number of quality assurance programs throughout the course of the census. One such program was the Full Count Review program, which was designed to rapidly examine, rectify if possible, and clear census data files and products for subsequent processing or public release. The bureau expected data analysts to identify data discrepancies, anomalies, and other data “issues” by checking the data for its overall reasonableness, as well as for its consistency with historical and demographic data, and other census data products. The Full Count Review program ran from June 2000 through March 2001. According to bureau officials, because the bureau could not complete the Full Count Review workload without a costly staff increase, some of the analysts’ work was contracted to members of the FSCPE, an organization composed of state demographers that works with the bureau to ensure accurate state and local population estimates. The bureau contracted with 53 FSCPE members who reviewed data for 39 states and Puerto Rico. Bureau employees reviewed data for the 11 remaining states and the District of Columbia without FSCPE representation in Full Count Review. Bureau and FSCPE analysts were to ensure that (1) group quarters were correctly placed or “geocoded” on census maps, and that their population counts and demographic characteristics appeared reasonable and (2) population counts of other areas were in line with population estimates. They were to describe each issue flagged and provide supporting documentation derived from bureau resources and/or resources of the respective state government. Additionally, bureau officials stated that staff from the regional offices reviewed demographic data from the 50 states, Puerto Rico, and the District of Columbia. They focused on identifying inconsistent demographic characteristics and did not necessarily concentrate on any one particular state or locality. The bureau reimbursed state governments for wages and expenses FSCPE members incurred. A separate set of employees from the bureau’s Population Division assessed issues identified by Full Count Review analysts based on (1) the adequacy of the documentation supporting each issue, and (2) whether or not they believed the issue to be resolvable through follow- up research by the bureau. Those issues deemed to have adequate documentation were classified as a “group quarters,” “housing unit,” or “household” or “other” issue. Bureau officials told us that the remaining issues could not be categorized because the nature of the issue could not be determined from the documentation. The Numbers and Kinds of Issues Identified During Full Count Review: Bureau data show that after reviewing census data for 39 states and Puerto Rico, FSCPE members identified a total of 1,402 issues, or about 29 percent of the 4,809 issues collectively flagged during Full Count Review (see table 1). Since the bureau has yet to resolve most of these issues, it is not known whether they are necessarily errors. Table 1: Types of Issues Flagged During Full Count Review: Type of issue: Group quarters; Issues identified by FSCPE members: 567; Issues identified by bureau personnel: 1,032; Total: 1,599. Type of issue: Housing unit; Issues identified by FSCPE members: 203; Issues identified by bureau personnel: 276; Total: 479. Type of issue: Household; Issues identified by FSCPE members: 134; Issues identified by bureau personnel: 154; Total: 288. Type of issue: Other; Issues identified by FSCPE members: 38; Issues identified by bureau personnel: 345; Total: 383. Type of issue: No type assigned; Issues identified by FSCPE members: 460; Issues identified by bureau personnel: 1,600; Total: 2,060. Type of issue: Total; Issues identified by FSCPE members: 1,402; Issues identified by bureau personnel: 3,407; Total: 4,809. Note: 201 issues were placed in more than one category. Source: GAO analysis of U.S. Census Bureau data. [End of table] Table 1 also shows that group quarters issues were those most frequently identified by the bureau, accounting for 1,599 of the 4,809 issues identified (33 percent). Group quarters issues relate to suspected discrepancies in the population counts and locations of prisons, dormitories, nursing homes, and similar group living arrangements. Analysts also identified 479 housing unit issues (10 percent of the total), and 288 household issues (6 percent of the total). With housing unit issues, the count of occupied housing units differed from what analysts expected while household issues had population data for occupied residences that differed from what analysts expected. There were also 383 issues (8 percent) that the bureau classified as “other”. They contained questions concerning the demographic characteristics of the data such as age, race, and gender. The bureau was unable to classify 2,060 issues (43 percent). Bureau officials told us that in these cases, analysts did not provide sufficient documentation for the bureau to determine the nature of the issue. According to bureau officials, bureau analysts identified a larger number of issues than FSCPE members—and a far larger number of issues for which the bureau could not assign a type—because bureau analysts used an automated process that compared data from the 2000 Census to independent benchmarks such as the 1990 Census, and flagged any anomalies. This process alerted bureau officials that there were data discrepancies, but did not indicate their nature. By comparison, FSCPE members compared census data to administrative records and other data, and were better able to document specific issues. Examples of the three issue categories and how they were found include: Group quarters issues: Analysts noticed that the group quarters population count in a particular census tract of a large midwestern city appeared to be too high, while a neighboring tract had a correspondingly low group quarters population count. By comparing state administrative records to information obtained from bureau resources, analysts determined that bureau data had placed college dormitories in the wrong tract. Housing unit issues: An urban area had a large amount of redevelopment since the 1990 census. As part of this, several condominiums and apartment complexes were built which substantially increased the number of housing units in a particular census tract. However, when the analyst compared population data from the 1990 Census and 2000 Census, the 2000 Census did not appear to reflect this increase, and it was flagged. Household issues: Data from the 2000 Census appeared to accurately reflect the large amount of new house construction that had taken place within a specific census tract. However, because the population count differed from that indicated by other data sources, the analyst flagged it as an issue to avoid undercounting the population. The Bureau Resolved Few Issues Prior to Releasing Public Law Data: Bureau officials told us that they used the Full Count Review program to identify systemic errors such as those that could be produced by software problems. None were found. The officials noted that the bureau generally did not use the Full Count Review program to resolve individual issues. According to bureau officials, the bureau corrected data for 5 of the 4,809 issues prior to the December 31, 2000, release of reapportionment data and the April 1, 2001, release of redistricting data. According to bureau officials, FSCPE members identified the five issues, all of which involved group quarters that were placed in the wrong locations, but the population counts were correct. They included (1) a military base in Nevada, (2) 10 facilities at a college in Wisconsin, (3) 9 facilities at a prison in New York City, (4) 14 facilities at a Washington prison, and (5) a federal medical center in Massachusetts. Bureau officials said that the bureau was able to correct these issues for two reasons. First, FSCPE analysts found them early in the Full Count Review program, while the bureau was processing a key geographic data file and was thus able to incorporate the corrections before the data were finalized. Second, the FSCPE analysts had thoroughly documented the issues and recommended how the bureau should correct the errors. The five errors did not require additional research or field verification. Bureau officials told us that they lacked the time to research the remaining issues, as well as field staff to inspect purported discrepancies prior to the release of the public law data. As a result, the bureau missed an important opportunity to verify and possibly improve the quality of the data, and instead the apportionment and redistricting data were released with more than 4,800 unresolved issues. Until these issues are resolved, uncertainties will surround the accuracy of the census data for the affected localities. Some of the issues might be resolved under the CQR program, which the bureau designed to respond to challenges to housing unit and group quarters population counts received from state, local, or tribal governments. [Footnote 2] However, as shown in table 2, of the 4,804 issues remaining after Full Count Review, 1,994 (42 percent) were referred to CQR, and of these, 537 (11 percent) were accepted for further investigation. The remaining 1,457 issues referred to CQR did not meet the bureau’s documentation requirements and consequently, the bureau took no further action on them (see app. 1 for the disposition of Full Count Review data issues by state). Table 2: Disposition of Data Issues Identified During Full Count Review: Action: Identified during Full Count Review; Number of issues: 4,809. Action: Corrected during Full Count Review; Number of issues: 5. Action: Identified during Full Count Review but not referred to CQR; Number of issues: 2,810. Action: Referred to CQR; Number of issues: 1,994. Action: Accepted by CQR; Number of issues: 537. Action: Rejected by CQR; Number of issues: 1,457. Source: GAO analysis of U.S. Census Bureau data. [End of table] Lessons Learned That Can Inform Future Data Clearance Reviews: The overall results of the Full Count Review program and FSCPE members’ participation appear to be mixed. On the one hand, the bureau reported that the Full Count Review program was successful in that it met a number of performance goals. For example, the bureau reported that the Full Count Review program was comprehensive in its review of geography and content, and was completed in time to release the public law data on schedule. Moreover, between January and February 2001, the bureau surveyed the 40 entities that participated in Full Count Review and the results suggest that most FSCPE members were satisfied with their Full Count Review experience. For example, respondents indicated that they were generally satisfied with such aspects of the program as its processes and technical tools, bureau staff, and the overall effectiveness of the review in terms of positioning states to use and understand census data. In addition, bureau officials believe the Full Count Review program benefited from FSCPE members’ local demographic knowledge. Nevertheless, our review of the Full Count Review program highlighted several areas where there is room for future improvement. It will be important for the bureau to address these shortcomings as its preliminary plans call for a similar operation as part of the 2010 Census. According to bureau officials, the bureau plans to include a Full Count Review program in census tests it expects to conduct later in the decade. Foremost among the areas in need of improvement is resolving, to the extent practical, a larger number of data issues prior to the release of apportionment data by December 31 of the census year, and redistricting data by April 1 of the following year. We found three factors that limited the bureau’s ability to do so. First, according to bureau officials, resolving individual issues was outside the scope of the Full Count Review program. They explained that the program was poorly integrated with other census operations and units that could have investigated the issues and corrected the data if warranted. This was because the Full Count Review program, with FSCPE participation, was not conceived until February 1999, which was extremely late in the census cycle, coming just 14 months before Census Day, April 1, 2000. The timing of the decision stemmed from the Supreme Court’s January 1999 ruling [Footnote 3] that prohibited the bureau from using statistical sampling for purposes of congressional apportionment (the bureau originally planned a “one-number” census that would have integrated the results of a sample survey with the traditional census to provide one adjusted set of census numbers). Faced with the larger workload of reviewing two sets of data—adjusted and unadjusted—the bureau decided to enlist the help of FSCPE members in order to meet the deadlines for releasing the public law data. Additionally, the bureau’s decision came after the 1998 dress rehearsal for the 2000 Census, which meant that the bureau had no opportunity to test the Full Count Review program in an operational environment. Bureau officials explained that if more time or staff were available in the future, it would be possible to correct a larger number of individual issues prior to the release of the public law data. They noted that field staff would be needed to help verify issues, and the effort would require close coordination with several bureau units. A second factor that affected the bureau’s ability to correct a larger number of issues was that the bureau’s requirements for documenting data issues were not clearly defined. For example, the training materials we examined did not provide any specific guidance on the type of evidence analysts needed to support data issues. Instead, the training materials told analysts to supply as much supporting information as necessary. This could help explain the variation that we observed in the quality of the documentation analysts provided. Indeed, while some analysts provided only minimal data, others supported issues with state and local administrative records, historical data, photographs, and maps. In some cases, the bureau had difficulty determining the precise nature of an issue or if in fact an issue even existed. In contrast, the CQR program provides comprehensive guidelines on the documentation required for making submissions. The guidance available on the bureau’s CQR web site notes that before the bureau will investigate concerns raised by government and tribal officials, such officials must first supply specific information. The guidance then details the information needed to support boundary corrections, geocoding and coverage corrections, and group quarters population corrections. A third, and related factor that affected the bureau’s ability to resolve a larger number of issues stemmed from the fact that the bureau had no mechanism for managing the Full Count Review workload. Unlike the CQR program, where the bureau required local governments to provide specific documentation before it would commit resources to investigate local data issues, the Full Count Review program had no filter for screening submissions based on the quality of the documentation. Better guidance on documenting issues for the Full Count Review program could make the bureau’s follow-up investigations more efficient. Another area where there is room for improvement concerns the consistency and clarity in which the bureau communicated the objectives of the Full Count Review program and how the bureau planned to use analysts’ input. For example, materials used to train FSCPE members noted that one purpose of Full Count Review was to document issues and “fix what can be fixed.” However, this appears to be inconsistent with statements made by bureau officials, who noted that resolving individual issues was beyond the scope of the Full Count Review program. Moreover, according to one bureau official, it was not clear internally what was meant by “fix what can be fixed.” None of the bureau’s documentation or training manuals that we reviewed explicitly stated that the bureau would only check for systemic errors. Because of the inconsistent message on the purpose of the Full Count Review program, the bureau may have set up the expectation that a larger number of issues would be resolved during Full Count Review. For example, one FSCPE member told us that he expected FSCPE members would identify any geographic discrepancies that contrasted with preliminary census data, and the bureau would investigate and make the necessary changes. He noted that both he and his staff were very “dismayed” to find out that certain discrepancies involving group quarters were not resolved prior to the release of the public law data. Another FSCPE member told us that participants were strongly motivated by the expectation that everything would be done to correct the census data. Conclusions: The Full Count Review program was one of a series of quality assurance efforts the bureau implemented throughout the census that helped ensure the bureau released accurate data. Moreover, FSCPE members’ participation, and specifically their expertise and knowledge of local geography, demographics, and housing arrangements, had the potential to identify data issues that the bureau might have otherwise missed. However, the fact that the apportionment and redistricting data were released with around 4,800 unresolved data issues of unknown validity, magnitude, and impact, is cause for concern, and indicates that the bureau missed an opportunity to verify and possibly improve the quality of the public law data. Given the importance of accurate census data and the resources that bureau staff and FSCPE members invest in the Full Count Review program, it will be important for the bureau to explore how to make better use of the program for correcting potential errors in census data in the future. It will also be important for the bureau to clarify the purpose of the Full Count Review program and convey that purpose clearly and consistently to FSCPE members. Doing so could help ensure that the bureau meets FSCPE members’ expectations. Recommendations for Executive Action: To help ensure the accuracy and completeness of census data and take full advantage of the Full Count Review program and FSCPE members’ participation, we recommend that the Secretary of Commerce direct the bureau to develop ways to resolve a larger number of data issues prior to the release of the public law data. Specifically, consideration should be given to: (1) planning the Full Count Review program early in the census cycle and testing procedures under conditions as close to the actual census as possible; (2) integrating the Full Count Review program with other census organizational units and operations to ensure the bureau has sufficient time and field support to investigate issues; (3) developing clear guidelines on the minimum documentation needed for the bureau to investigate individual data issues; (4) categorizing issues on the basis of the quality and precision of the documentation, and investigating first those issues that are best documented and thus more easily resolved, and; (5) exploring the feasibility of using staff from the bureau’s regional offices to help investigate data issues in the field prior to the release of public law data. Moreover, to ensure no expectation gaps develop between the bureau and FSCPE members, the Secretary of Commerce should also ensure that the bureau clarifies and consistently communicates to participants the objectives of the Full Count Review program and how the bureau plans to use the information derived from it. Agency Comments and Our Evaluation: The Secretary of Commerce forwarded written comments from the Bureau of the Census on a draft of this report (see app. II). The bureau concurred with all of our recommendations and had no comments on them. The bureau also provided minor technical corrections that we incorporated in our report as appropriate. As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we will send copies to other interested congressional committees, the Secretary of Commerce, and the Director of the Bureau of the Census. Copies will be made available to others upon request. In addition, the report will be available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. Corinna Wengryn, Ty Mitchell, and Robert Goldenkoff made major contributions to this report. If you have any questions concerning this report, please contact me on (202) 512-6806. Signed by: Patricia A. Dalton: Director: Strategic Issues: [End of section] Appendix I: Disposition of Data Issues Identified During Full Count Review by State: State: Alabama; Issues identified by FSCPE members: 18; Issues identified by bureau personnel: 54; Total accepted for research by Count Question Resolution program: 3. State: Alaska; Issues identified by FSCPE members: 21; Issues identified by bureau personnel: 77; Total accepted for research by Count Question Resolution program: 4. State: Arizona; Issues identified by FSCPE members: 17; Issues identified by bureau personnel: 57; Total accepted for research by Count Question Resolution program: 5. State: Arkansas; Issues identified by FSCPE members: 65; Issues identified by bureau personnel: 72; Total accepted for research by Count Question Resolution program: 16. State: California; Issues identified by FSCPE members: 20; Issues identified by bureau personnel: 71; Total accepted for research by Count Question Resolution program: 12. State: Colorado; Issues identified by FSCPE members: 36; Issues identified by bureau personnel: 29; Total accepted for research by Count Question Resolution program: 5. State: Connecticut; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 70; Total accepted for research by Count Question Resolution program: 16. State: Delaware; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 67; Total accepted for research by Count Question Resolution program: 0. State: District of Columbia; Issues identified by FSCPE members: 18; Issues identified by bureau personnel: 61; Total accepted for research by Count Question Resolution program: 1. State: Florida; Issues identified by FSCPE members: 39; Issues identified by bureau personnel: 61; Total accepted for research by Count Question Resolution program: 14. State: Georgia; Issues identified by FSCPE members: 23; Issues identified by bureau personnel: 42; Total accepted for research by Count Question Resolution program: 6. State: Hawaii; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 127; Total accepted for research by Count Question Resolution program: 5. State: Idaho; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 62; Total accepted for research by Count Question Resolution program: 2. State: Illinois; Issues identified by FSCPE members: 112; Issues identified by bureau personnel: 52; Total accepted for research by Count Question Resolution program: 15. State: Indiana; Issues identified by FSCPE members: 21; Issues identified by bureau personnel: 55; Total accepted for research by Count Question Resolution program: 9. State: Iowa; Issues identified by FSCPE members: 71; Issues identified by bureau personnel: 38; Total accepted for research by Count Question Resolution program: 21. State: Kansas; Issues identified by FSCPE members: 19; Issues identified by bureau personnel: 46; Total accepted for research by Count Question Resolution program: 6. State: Kentucky; Issues identified by FSCPE members: 16; Issues identified by bureau personnel: 60; Total accepted for research by Count Question Resolution program: 11. State: Louisiana; Issues identified by FSCPE members: 25; Issues identified by bureau personnel: 66; Total accepted for research by Count Question Resolution program: 3. State: Maine; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 67; Total accepted for research by Count Question Resolution program: 1. State: Maryland; Issues identified by FSCPE members: 8; Issues identified by bureau personnel: 39; Total accepted for research by Count Question Resolution program: 0. State: Massachusetts; Issues identified by FSCPE members: 29; Issues identified by bureau personnel: 38; Total accepted for research by Count Question Resolution program: 14. State: Michigan; Issues identified by FSCPE members: 23; Issues identified by bureau personnel: 51; Total accepted for research by Count Question Resolution program: 32. State: Minnesota; Issues identified by FSCPE members: 69; Issues identified by bureau personnel: 44; Total accepted for research by Count Question Resolution program: 30. State: Mississippi; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 123; Total accepted for research by Count Question Resolution program: 1. State: Missouri; Issues identified by FSCPE members: 21; Issues identified by bureau personnel: 37; Total accepted for research by Count Question Resolution program: 7. State: Montana; Issues identified by FSCPE members: 27; Issues identified by bureau personnel: 69; Total accepted for research by Count Question Resolution program: 12. State: Nebraska; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 93; Total accepted for research by Count Question Resolution program: 20. State: Nevada; Issues identified by FSCPE members: 29; Issues identified by bureau personnel: 40; Total accepted for research by Count Question Resolution program: 5. State: New Hampshire; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 39; Total accepted for research by Count Question Resolution program: 0. State: New Jersey; Issues identified by FSCPE members: 58; Issues identified by bureau personnel: 52; Total accepted for research by Count Question Resolution program: 13. State: New York; Issues identified by FSCPE members: 33; Issues identified by bureau personnel: 100; Total accepted for research by Count Question Resolution program: 33. State: North Carolina; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 149; Total accepted for research by Count Question Resolution program: 31. State: North Dakota; Issues identified by FSCPE members: 19; Issues identified by bureau personnel: 47; Total accepted for research by Count Question Resolution program: 11. State: Ohio; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 174; Total accepted for research by Count Question Resolution program: 25. State: Oklahoma; Issues identified by FSCPE members: 14; Issues identified by bureau personnel: 41; Total accepted for research by Count Question Resolution program: 13. State: Oregon; Issues identified by FSCPE members: 7; Issues identified by bureau personnel: 57; Total accepted for research by Count Question Resolution program: 3. State: Pennsylvania; Issues identified by FSCPE members: 51; Issues identified by bureau personnel: 56; Total accepted for research by Count Question Resolution program: 12. State: Puerto Rico; Issues identified by FSCPE members: 20; Issues identified by bureau personnel: 57; Total accepted for research by Count Question Resolution program: 18. State: Rhode Island; Issues identified by FSCPE members: 25; Issues identified by bureau personnel: 26; Total accepted for research by Count Question Resolution program: 7. State: South Carolina; Issues identified by FSCPE members: 43; Issues identified by bureau personnel: 51; Total accepted for research by Count Question Resolution program: 11. State: South Dakota; Issues identified by FSCPE members: 33; Issues identified by bureau personnel: 50; Total accepted for research by Count Question Resolution program: 11. State: Tennessee; Issues identified by FSCPE members: 31; Issues identified by bureau personnel: 55; Total accepted for research by Count Question Resolution program: 7. State: Texas; Issues identified by FSCPE members: 86; Issues identified by bureau personnel: 93; Total accepted for research by Count Question Resolution program: 1. State: Utah; Issues identified by FSCPE members: 19; Issues identified by bureau personnel: 59; Total accepted for research by Count Question Resolution program: 8. State: Vermont; Issues identified by FSCPE members: 112; Issues identified by bureau personnel: 35; Total accepted for research by Count Question Resolution program: 12. State: Virginia; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 135; Total accepted for research by Count Question Resolution program: 8. State: Washington; Issues identified by FSCPE members: 59; Issues identified by bureau personnel: 52; Total accepted for research by Count Question Resolution program: 8. State: West Virginia; Issues identified by FSCPE members: 7; Issues identified by bureau personnel: 59; Total accepted for research by Count Question Resolution program: 4. State: Wisconsin; Issues identified by FSCPE members: 0; Issues identified by bureau personnel: 124; Total accepted for research by Count Question Resolution program: 11. State: Wyoming; Issues identified by FSCPE members: 26; Issues identified by bureau personnel: 39; Total accepted for research by Count Question Resolution program: 10. State: Total; Issues identified by FSCPE members: 1,402; Issues identified by bureau personnel: 3,407; Total accepted for research by Count Question Resolution program: 537. Note: The table includes 201 issues that were placed in more than one issue category (the bureau had five categories of issues depending on the nature of the discrepancy). Source: GAO analysis of U.S. Census Bureau data. [End of table] [End of section] Appendix II: Comments From the Department of Commerce: The Secretary Of Commerce: Washington, D.C. 20230: June 11, 2002: Ms. Patricia A. Dalton: Director, Strategic Issues: U.S. General Accounting Office: Washington, DC 20548: Dear Ms. Dalton: The Department of Commerce appreciates the opportunity to comment on the General Accounting Office draft report entitled 2000 Census: Refinements to Full Count Review Program Could Improve Future Data Quality. The Department's comments on this report are enclosed. Warm regards, Signed by: Donald L. Evans: Enclosure: Comments from the U.S. Department of Commerce: U.S. Census Bureau: U.S. General Accounting Office draft report entitled 2000 Census: Refinements to Full Count Review Program Could Improve Future Data Quality: Comments on the Text of the Report: 1. Section: Page 5, last paragraph, continued on page 6 — "The Full Count Review program ran from June through August, 2000." Comment: The period June through August 2000 was the training phase of the Full Count Review program. The program ran from June 2000 through March 2001. 2. Section: Page 6, second full paragraph, first sentence — "The bureau contracted with 53 SCPE members who reviewed data for 38 states, the District of Columbia, and Puerto Rico." Comment: The 53 FSCPE members represented 39 states and Puerto Rico. The District of Columbia did not participate in the Full Count Review program. 3. Section: Page 6, second full paragraph, second sentence — "Bureau employees reviewed data for the 12 remaining states without FSCPE representation in Full Count Review." Comment: The sentence should read "... remaining 11 states ...." 4. Section: Page 7, first paragraph -- "Bureau data show that after reviewing census data for 38 states, the District of Columbia, and Puerto Rico, FSCPE members identified ...." Comment: See comment in response to Item #2 above. 5. Section: Page 7, Table 1. Comment: In the GAO analysis in Table 1, 201 of the issues were placed in more than one category. For this reason, the Census Bureau cannot independently verify the accuracy of these data. 6. Section: Page 9, last paragraph, last sentence — "In all, of the 4,809 issued [sic] identified during Full Count Review, 4,267 (89 percent), were not subjected to any further review by the bureau ...." Comment: The 1,457 issues that were referred to the Count Question Resolution (CQR) program, but rejected, were "subjected to further review," in the context of the CQR program criteria. That they failed to meet the criteria for processing under the CQR, and no further action was taken with respect to these issues, does not mean that no additional review was undertaken after they were identified during the Full Count Review program. 7. Section : Page 10, third paragraph, last sentence – "According to bureau officials, if funding permits, the bureau will include a Full Count Review program as part of a 2004 test of census-taking activities." Comment: the Census Bureau is still developing its plan for the 2004 Census Test and has not determined at this time whether it will include a Full Count Review program. However, we certainly plan to include such a program in census tests later in the decade. Responses to GAO Recommendations: Census Bureau Response: The Census Bureau concurs with the recommendations and has no specific comments on them at this time. [End of enclosure] [End of section] Appendix III: GAO Products on the Results of the 2000 Census and Lessons Learned for a More Cost-Effective Census in 2010: U.S. General Accounting Office. 2000 Census: Coverage Evaluation Matching Implemented As Planned, but Census Bureau Should Evaluate Lessons Learned. GAO-02-297. Washington, D.C.: March 14, 2002. U.S. General Accounting Office. 2000 Census: Best Practices and Lessons Learned for a More Cost-Effective Nonresponse Follow-Up. GAO-02-196. Washington, D.C.: February 11, 2002. U.S. General Accounting Office. 2000 Census: Coverage Evaluation Interviewing Overcame Challenges, but Further Research Needed. GAO- 02-26. Washington, D.C.: December 31, 2001. U.S. General Accounting Office. 2000 Census: Analysis of Fiscal Year 2000 Budget and Internal Control Weaknesses at the U.S. Census Bureau. GAO-02-30. Washington, D.C.: December 28, 2001. U.S. General Accounting Office. 2000 Census: Significant Increase in Cost Per Housing Unit Compared to 1990 Census. GAO-02-31. Washington, D.C.: December 11, 2001. U.S. General Accounting Office. 2000 Census: Better Productivity Data Needed for Future Planning and Budgeting. GAO-02-4. Washington, D.C.: October 4, 2001. U.S. General Accounting Office. 2000 Census: Review of Partnership Program Highlights Best Practices for Future Operations. GAO-01-579. Washington, D.C.: August 20, 2001. U.S. General Accounting Office. Decennial Censuses: Historical Data on Enumerator Productivity Are Limited. GAO-01-208R. Washington, D.C.: January 5, 2001. U.S. General Accounting Office. 2000 Census: Information on Short- and Long-Form Response Rates. GAO/GGD-00-127R. Washington, D.C.: June 7, 2000. [End of section] Footnotes: [1] The Census Act (13 USC sec. 141(a,b)) requires the Secretary of Commerce to deliver state population counts to the President within 9 months after the census date (for the 2000 Census, this meant no later than December 31, 2000). State population counts are used to reapportion seats in the U.S. House of Representatives. The Census Act also requires the Secretary of Commerce to send census population tabulations to the states no later than 1 year after the April 1 decennial census date (13 USC sec. 141 (c)). These numbers are used for redistricting. [2] CQR began in late June 2001, and is scheduled to end in September 2003. [3] Department of Commerce v. United States House of Representatives, 525 U.S. 316 (1999). [End of section] GAO’s Mission: The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site [hyperlink, http://www.gao.gov] contains abstracts and fulltext files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to [hyperlink, http://www.gao.gov] and select “Subscribe to daily E-mail alert for newly released products” under the GAO Reports heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office: 441 G Street NW, Room LM: Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov: (202) 512-4800: U.S. General Accounting Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: