This is the accessible text file for GAO report number GAO-05-463 
entitled 'Data Quality: Improvements to Count Correction Efforts Could 
Produce More Accurate Census Data' which was released on July 20, 2005. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

June 2005: 

Data Quality: 

Improvements to Count Correction Efforts Could Produce More Accurate 
Census Data: 

GAO-05-463: 

GAO Highlights: 

Highlights of GAO-05-463, a report to congressional requesters: 

Why GAO Did This Study: 

The U.S. Census Bureau (Bureau) conducted the Count Question Resolution 
(CQR) program to correct errors in the count of housing units as well 
as dormitories and other group living facilities known as group 
quarters. GAO was asked to assess whether CQR was consistently 
implemented across the country, paying particular attention to whether 
the Bureau identified census errors that could have been caused by more 
systemic problems. GAO also evaluated how well the Bureau transitioned 
to CQR from an earlier quality assurance program called Full Count 
Review. 

What GAO Found: 

The CQR program, which ran from June 30, 2001, to September 30, 2003, 
played an important role in improving the quality of data from the 2000 
Census in that it corrected numbers affecting 47 states and over 1,180 
governmental units. Although this is a small percentage of the nation’s 
more than 39,000 government entities, the count revisions impacted 
private homes, prisons, and other dwellings and, in some cases, were 
significant. For example, when the Bureau deleted duplicate data on 
students at the University of North Carolina at Chapel Hill and made 
other corrections, that state’s head count dropped by 2,828 people. 
Similarly, CQR found that more than 1,600 people in Morehead, Kentucky, 
were counted in the wrong location. 

[See PDF for image]

[End of figure]

GAO identified several shortcomings with the CQR program, including 
inconsistent implementation by the Bureau’s regional offices and the 
posting of inaccurate data to the Bureau’s Web-based errata report. 
Moreover, while CQR found the counting of group quarters to be 
particularly problematic, the Bureau did not perform an active, 
nationwide review of these known trouble spots, and thus missed an 
opportunity to potentially improve the accuracy of the data for these 
dwellings. Further, because CQR had more stringent documentation 
requirements compared to a preceding program called Full Count Review, 
CQR rejected hundreds of unresolved full count issues, missing another 
opportunity to improve the data. As its plans proceed for the 2010 
Census, it will be important for the Bureau to address the operational 
issues GAO identified. Moreover, because the data for apportionment and 
redistricting were later found to be flawed for some jurisdictions, it 
will be important for the Bureau to develop a count correction program 
that is designed to systematically review and correct these essential 
figures. 

What GAO Recommends: 

GAO recommends that the Secretary of Commerce direct the Bureau to take 
such actions as consolidating CQR and Full Count Review into a single 
effort that systematically reviews and corrects any errors prior to the 
release of data for apportionment and redistricting; prioritizing the 
review of errors based on the magnitude of the problem; and ensuring 
the accuracy and accessibility of the revised data on its Web site. The 
Department of Commerce noted our report made several useful 
recommendations, but stated our approach was infeasible because of 
timing and other constraints. We believe our recommendations still 
apply because they could help the Bureau overcome these constraints and 
deliver better quality data. 

www.gao.gov/cgi-bin/getrpt?GAO-05-463. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Orice Williams at (202) 
512-6806 or williamso@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Scope and Methodology: 

CQR Program Corrected Numerous Data Errors, but More Consistent 
Implementation and Other Improvements Are Needed: 

Better Strategic Planning and Other Actions Could Improve Future Count 
Correction Efforts: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendixes: 

Appendix I: Change in State Populations As a Result of Count Question 
Resolution Program: 

Appendix II: Human Error and Other Factors Contributed to University of 
North Carolina Counting Errors: 

Appendix III: Comments from the Department of Commerce: 

Appendix IV: GAO Contact and Staff Acknowledgments: 

Figures: 

Figure 1: Time Line Showing Relationship of CQR Program to Key Census 
2000 Milestones: 

Figure 2: Map of Census Bureau's 12 Regions: 

Figure 3: CQR Revisions Affected Numerous Governmental Units in Most 
States: 

Figure 4: Students in 26 UNC Dormitories Were Counted Twice in the 
Census: 

Figure 5: Prisoners in Cameron, Missouri, Were Mistakenly Omitted From 
the Town's Population Count: 

Figure 6: CQR Table in the Bureau's 2000 Notes and Errata Report 
Showing Faulty Links to Data: 

Figure 7: Initial Census Data on Bureau Web Site Do Not Inform Users 
That Some Numbers Have Been Revised: 

Letter June 20, 2005: 

The Honorable Wm. Lacy Clay: 
Ranking Minority Member: 
Subcommittee on Federalism and the Census: 
Committee on Government Reform: 
House of Representatives: 

The Honorable Carolyn B. Maloney: 
House of Representatives: 

Complete and accurate data from the decennial census are central to our 
democratic system of government. As required by the Constitution, 
census results are used to apportion seats in the House of 
Representatives. Census data are also used to redraw congressional 
districts, allocate billions of dollars in federal assistance to state 
and local governments, and for many other public and private sector 
purposes. Failure to deliver quality data could skew the equitable 
distribution of political power in our society, impair public and 
private decision making, and erode public confidence in the U.S. Census 
Bureau (Bureau). 

To ensure it delivers accurate data, the Bureau employs a number of 
quality assurance programs throughout the course of the census. One 
such effort during the 2000 Census was the Count Question Resolution 
(CQR) program, which enabled state, local, and tribal governments to 
formally challenge the counts of housing units and "group quarters" 
(dormitories, prisons, and other group living facilities), and their 
associated populations. Bureau personnel could initiate a review of the 
counts as well. 

Although the Bureau did not design CQR with the intention of 
incorporating any of the corrections that resulted from it into Census 
2000 data products--including the numbers used for congressional 
apportionment and redistricting (figures commonly referred to as 
"public law data")--governmental entities could use the updated 
information when applying for federal aid that uses census data as part 
of an allocation formula, as well as for other purposes. Because the 
count corrections could have political and financial implications for 
states and localities, it was important for the Bureau to carry out CQR 
consistent with its protocols. CQR began on June 30, 2001, and no new 
submissions were accepted after September 30, 2003. 

This letter responds to your request to review the conduct of the CQR 
program. As agreed with your offices, we reviewed the results of the 
CQR program and assessed whether the program was consistently 
implemented across the country. In doing this, we paid particular 
attention to the extent to which the Bureau reviewed the census data 
for errors that could have been caused by broader, more systemic 
problems. We also evaluated how well the Bureau transitioned to CQR 
from an earlier quality assurance program called Full Count Review. 

To meet these objectives, we reviewed relevant program documents and 
examined case files and conducted on-site inspections at four of the 
Bureau's regional offices where some of the largest CQR corrections 
took place. We also interviewed officials and staff responsible for 
administering the CQR program at the Bureau's headquarters and 12 
regional offices. We did our audit work between February 2004 and March 
2005 in accordance with generally accepted government auditing 
standards. 

Results in Brief: 

The CQR program corrected data affecting over 1,180 of the nation's 
more than 39,000 governmental units including states, counties, and 
cities. Although the national and state-level revisions were relatively 
small, in some cases the corrections at the local level were 
substantial. For example, CQR increased Morehead, Kentucky's, 
population total by more than 1,600 people because the Bureau 
mistakenly attributed local university students, who lived in 
dormitories located within the city, to the population count of an 
unincorporated section of the county in which Morehead is located. 
Likewise, the Bureau added almost 1,500 persons to the population count 
of Cameron, Missouri, when CQR found that a prison's population was 
erroneously omitted. 

That said, we also found critical aspects of the CQR program in need of 
improvement. For example, CQR was not consistently implemented by the 
Bureau's regional offices. Only the Bureau's Los Angeles Regional 
Office appeared to do any comprehensive, systematic research to 
identify possible count errors beyond those that were submitted by 
governmental units. Had it not been for Los Angeles' self-initiated 
review, several data errors--including instances where college 
dormitories were counted in the wrong geographic location--would have 
remained uncorrected because they were not identified by the affected 
jurisdiction. 

In contrast, the Bureau's Charlotte Regional Office found that almost 
2,700 students were counted twice at the University of North Carolina, 
the discovery of which came about largely because two key census 
employees in Charlotte were alumni of the school and curious to see 
whether dormitories there were enumerated correctly. One factor behind 
the disparate execution of the CQR program seems to have been vague and 
sometimes inconsistent guidance and training that left staff in the 
regional offices with different understandings of whether they could 
conduct self-initiated research. 

In addition, although the Bureau maintained an errata report on its Web 
site that listed the CQR revisions to the census data, our partial 
review of that information found several discrepancies between the 
updated figures, and what the numbers should have been. For example, 
the revised number of housing units for Sioux Falls, South Dakota, was 
almost 47,000 units too low. Likewise, the errata data on the total 
housing unit count for Burlington County, New Jersey, mistakenly 
excluded about 145,000 units. Moreover, embedded links on the Web site 
that were supposed to take users to revisions at lower levels of 
geography did not always work and produced error messages instead. 

The CQR program was also poorly integrated with its predecessor 
program, Full Count Review. Although the Bureau planned to fold 
unresolved full count issues into CQR, the latter program had more 
rigorous documentation requirements. Consequently, hundreds of 
unresolved Full Count Review cases lacked CQR's necessary 
documentation, were rejected from CQR, and received no further review. 

Overall, the CQR program was an important quality assurance tool, but 
the Bureau needs to address the operational issues we identified. 
Further, given the growing challenges to counting the nation's 
population, census errors are inevitable, and as the Bureau makes plans 
for the 2010 Census, it will be important for it to have a mechanism 
specifically designed to methodically review and correct errors in the 
public law data and subsequent data releases to the greatest extent 
possible. The lessons the Bureau has learned from CQR should provide 
valuable experience in developing such a program. 

With that in mind, we recommend that the Secretary of Commerce direct 
the Bureau to improve its count correction efforts by taking such 
actions as: (1) consolidating Full Count Review and CQR into a single 
program that systematically reviews and corrects any errors prior to 
the release of the public law data; (2) expediting count correction 
efforts, in part, by using enumerators to help investigate data 
discrepancies while conducting their field work; (3) prioritizing the 
investigation of data challenges based on the magnitude of the 
suspected error; (4) ensuring the accuracy and accessibility of the 
revised data on its Web site; and (5) improving training and guidance 
provided to regional offices to help ensure count correction activities 
are consistently implemented. 

The Acting Deputy Secretary of Commerce provided written comments on a 
draft of this report (see app. III). Commerce acknowledged that "the 
report provides a good overview of program results and makes several 
useful observations and recommendations," and agreed with our finding 
that the process for conducting internal reviews was not consistently 
implemented. Nevertheless, Commerce took exception to our 
recommendations calling for the Bureau to design a count correction 
effort capable of identifying and correcting errors in the 
apportionment and redistricting data before that critical information 
is released. Commerce noted that such an approach was infeasible 
largely because of time and logistical constraints. Our report 
recognizes these challenges; further, the steps we recommend could help 
the Bureau overcome these very challenges and deliver more accurate 
public law data. 

Background: 

The Bureau launched the CQR program on June 30, 2001, as the last in a 
series of quality assurance initiatives aimed at improving the accuracy 
of 2000 Census data (see fig. 1). Specifically, the CQR program 
provided a mechanism for state, local, and tribal governments to have 
the Bureau correct errors in certain types of census data. The Bureau 
referred to these challenges as "external cases." Bureau personnel 
could also initiate reviews of suspected count errors, independent of 
these challenges, for further review. These were known as "internal 
cases." Many of the internal cases were unresolved issues inherited 
from Full Count Review. Indeed, when the Full Count Review program 
began, the Bureau planned to fold unresolved issues from that program 
into CQR. The Bureau accepted no new submissions after the program 
officially ended on September 30, 2003, although it continued to review 
challenges submitted before the deadline and completed the final 
revisions in the summer of 2004. 

Figure 1: Time Line Showing Relationship of CQR Program to Key Census 
2000 Milestones: 

[See PDF for image] 

[End of figure] 

Three types of corrections were permissible under the CQR program: (1) 
boundary corrections, where a jurisdictional boundary of a functioning 
governmental unit was in the wrong location; (2) geocoding corrections, 
where the Bureau placed the living quarters and their associated 
population in the wrong location; and (3) coverage corrections, where 
the Bureau properly enumerated specific living quarters and their 
corresponding population during the census but incorrectly added or 
deleted the information during data processing. Bureau officials were 
to research cases using existing Bureau data gathered during the 2000 
Census; they could not conduct any new fieldwork to resolve count 
questions. The Bureau required governmental entities to accompany their 
challenges with specific documentation before it would investigate 
their claims. 

Importantly, under the design of CQR, if a governmental unit had 
evidence that the Bureau missed housing units or group quarters that 
existed on Census Day 2000 (April 1), but the Bureau's records 
indicated that all of the Bureau's boundary information, geocoding, and 
processing were properly implemented, the Bureau would not change the 
data. Rather, the Bureau was to address this as part of the planning 
process for the 2010 Census. 

If the CQR program corrected the population or housing unit counts of a 
particular entity, the Bureau was to issue revised, official figures 
for that jurisdiction. The governmental unit could then use the updated 
numbers for future programs requiring 2000 Census data. CQR corrections 
were also used to modify annual post-censal estimates beginning 
December 2002 and were publicized on the Bureau's Census 2000 and 
American FactFinder Web sites (www.census.gov and [Hyperlink, 
http://www.factfinder.census.gov], respectively), as part of the 2000 
Census notes and errata. However, CQR was not designed or publicized as 
a mechanism to correct the census results for purposes of apportionment 
and redistricting. 

In compliance with legal requirements, the Bureau produced 
apportionment data by December 31, 2000, and redistricting data by 
April 1, 2001[Footnote 1] (this information is known collectively as 
public law data). Although the law does not require that states use 
census data to redraw the boundaries of congressional districts, most 
states have always done so. Nothing would preclude the states from 
using the corrected data for redistricting. The general perception of 
the impartiality of the Bureau and the great cost and administrative 
effort required to take a census have been strong arguments in favor of 
using the Bureau's data. 

Scope and Methodology: 

As agreed with your offices, we assessed whether the program was 
consistently implemented across the country, paying particular 
attention to the extent to which the Bureau reviewed the census data 
for errors that could have been caused by broader, more systemic 
problems, such as shortcomings with a particular census-taking 
procedure. We also evaluated how well the Bureau transitioned from an 
earlier quality assurance program used in the 2000 Census, Full Count 
Review. 

To assess the implementation of the CQR program, we obtained a 
headquarters perspective by reviewing program documents and case files 
at the Bureau's offices in Suitland, Maryland, as well as program 
results reported on the Bureau's Web site.[Footnote 2] As part of this 
assessment, we reviewed the program's internal controls, especially 
those controls related to ensuring data quality. We also interviewed 
Bureau officials responsible for administering the program. 

To determine how CQR was implemented in the field, we visited 4 of the 
Bureau's 12 regional offices--Charlotte, North Carolina; Denver, 
Colorado; Kansas City, Missouri; and Los Angeles, California (see fig. 
2). 

Figure 2: Map of Census Bureau's 12 Regions: 

[See PDF for image] 

[End of figure] 

We selected these regions because they included the six states and 10 
governmental units within those states where the largest CQR count 
revisions occurred. We supplemented these cases by selecting an 
additional seven states and 61 places within the four regions for 
further examination. The 61 localities were selected because they 
represented the full spectrum of CQR cases and were geographically 
diverse. 

At each of the four regions, we reviewed regional case file information 
and interviewed Bureau personnel responsible for implementing CQR, such 
as program managers and geographers. We also made a site visit to at 
least one type of facility--including prisons, apartment buildings, and 
dormitories--in each region to understand firsthand the nature of the 
errors and the corrections made. To augment these four regional visits 
and obtain a more complete picture of how CQR was implemented, we used 
a structured telephone interview to elicit information from program 
officials at the Bureau's eight remaining regional offices that we did 
not visit in person. 

To determine the extent to which the Bureau reviewed census data for 
systemic errors, and its procedures for folding unresolved cases from 
the Full Count Review program into CQR, we examined program manuals, 
memoranda, and other documents, and interviewed officials in the 
Bureau's headquarters and all of its regional offices. As part of this 
effort, we also analyzed the CQR case-tracking data in an attempt to 
determine the number of unresolved Full Count Review cases that were 
rolled into the CQR program. However, we were unable to do this because 
the tracking system did not contain information on which CQR cases 
originated as full count issues. 

We requested comments on a draft of this report from the Secretary of 
Commerce. On May 20, 2005, we received the Acting Deputy Secretary's 
written comments and have reprinted them in appendix III; we address 
them in the Agency Comments and Our Evaluation section of this report. 

CQR Program Corrected Numerous Data Errors, but More Consistent 
Implementation and Other Improvements Are Needed: 

Overall, the CQR program corrected data affecting over 1,180 of the 
nation's more than 39,000 governmental units. The revisions impacted a 
range of housing types including private homes with only a handful of 
residents, to college dormitories and prison cell blocks with 
populations in the thousands. At the same time, however, we identified 
several shortcomings with the CQR program, including inconsistent 
handling of internal cases by the Bureau's regional offices and 
inaccurate data being posted to the Bureau's public Web site. Moreover, 
while CQR found the counting of group quarters in their correct 
location--a problem known as geocoding error--to be particularly 
challenging, the Bureau did not perform a nationwide review of these 
known trouble spots, and thus missed an opportunity to improve the 
accuracy of the data for these dwellings. 

CQR Program Corrected Errors in Hundreds of Governmental Units: 

Nationwide, the CQR program corrected count errors involving 
governmental units in 47 states, Puerto Rico, and the District of 
Columbia (see fig. 3).[Footnote 3] Three states--Maine, New Hampshire, 
and Rhode Island--had no CQR corrections. 

Figure 3: CQR Revisions Affected Numerous Governmental Units in Most 
States: 

[See PDF for image] 

[A] The Bureau made count changes in Hawaii and the District of 
Columbia, but these revisions were made at the census block level and 
did not change the state's governmental unit counts. 

[End of figure] 

The corrections affected over 1,180 governmental units in the United 
States. Although this is a small percentage of the nation's more than 
39,000 governmental units, the impact of those changes on local 
governments was, in some cases, substantial, and could have 
implications for federal assistance and state funding programs that use 
census numbers in their allocation formulas, as well as other 
applications of census data. 

For example, officials in one Kentucky county challenged the geocoding 
of a housing unit located near new precinct and congressional district 
boundaries. They told the Bureau that the new boundaries split the 
county, and they were concerned that the geocoding error would affect 
where the housing unit's few occupants registered to vote. Because the 
housing unit was improperly geocoded, the Bureau corrected the data. 

With respect to fiscal effects, the Controller of the State of 
California uses population figures as the basis for refunding a portion 
of state taxpayer fees--including automobile licensing fees--to cities 
and counties. Because of an error in the 2000 Census, Soledad, 
California, officials estimated it lost more than $140,000 in state 
refunds when over 11,000 residents were incorrectly counted in two 
nearby cities' populations, according to city and state officials. 
Although CQR eventually corrected the error, Soledad did not recover 
the funds that went to the other cities. 

Other examples of large CQR corrections include the following (See app. 
I for a complete list of state-level population changes):[Footnote 4]

* North Carolina's population count was reduced by 2,828 people, 
largely because the Bureau had to delete duplicate data on almost 2,700 
students in 26 dormitories (see fig. 4) at the University of North 
Carolina (UNC) at Chapel Hill. The erroneous enumerations occurred, in 
large part, because of mistakes that occurred in various preparatory 
activities leading up to the 2000 Census (See app. II for a more 
detailed discussion of this incident.). 

Figure 4: Students in 26 UNC Dormitories Were Counted Twice in the 
Census: 

[See PDF for image] 

[End of figure] 

* The population count of Morehead, Kentucky, increased by more than 
1,600 when CQR found that a large number of students from Morehead 
State University's dormitories were erroneously excluded from the 
city's population. During the 2000 Census, the Bureau had incorrectly 
identified the dormitories as being outside city limits and in an 
unincorporated area of Rowan County. 

* The population count of Cameron, Missouri, was off by nearly 1,500 
people when the Bureau found that the prison population of the state's 
Crossroads Correctional Center was inadvertently omitted from the 
town's headcount (see fig. 5). The correction to the town's population 
accounted for the entire 1,472 person increase in Missouri's total 
population under the CQR program. 

Figure 5: Prisoners in Cameron, Missouri, Were Mistakenly Omitted From 
the Town's Population Count: 

[See PDF for image] 

[End of figure] 

* The population of the city of Waseca, Minnesota, increased by more 
than 1,100. The 2000 Census had mistakenly included the prison 
population for the Waseca Federal Correctional Institute in two 
surrounding townships in Waseca County. The CQR program resulted in the 
population being shifted to the city. 

* The population of Colorado increased by more than 750 in large part 
because a processing error in counting housing units in Grand Junction 
initially excluded almost 700 people from the city's population total. 

* The population of Denver and Arapahoe Counties in Colorado shifted by 
more than 900 because the Bureau had incorrectly assigned the location 
of two apartment complexes. As a result of CQR, the apartment complexes 
were incorrectly identified and counted as being in Denver but under 
CQR were later found to be in adjoining Arapahoe County. 

CQR Program Was Unevenly Executed: 

The Bureau's 12 regional offices did not always adhere to the same set 
of procedures when developing internal cases, and this, in turn, 
produced uneven results. Importantly, the procedures used to execute 
public programs need to be well documented, transparent, and 
consistently applied in order to ensure fairness, accountability, 
defensible decisions, and reliable outcomes. To do otherwise could 
raise equity questions. 

One variation in the way internal cases were handled was evident at the 
Bureau's Los Angeles Regional Office, which appeared to be the only 
region to do comprehensive methodical research to actively identify 
possible count errors beyond those that were submitted by governmental 
entities. According to the office's senior geographer, the geography 
staff developed a structured approach to systematically examine census 
data from all the prisons and colleges within the office's 
jurisdiction, because the data on both types of group quarters were 
known to be problematic. He added that the more problems they found, 
the more they were motivated to keep digging. The geographer noted that 
the in-depth review was possible because the Los Angeles region covers 
only the southern half of California and the state of Hawaii, and thus 
has fewer governmental units compared to the Bureau's other regional 
offices. 

Had it not been for the Los Angeles region's self-initiated and 
systematic review, certain data errors would have gone uncorrected 
because they were not identified by the affected jurisdiction. For 
example, regional staff found instances where college dormitories were 
counted in the wrong geographic location, which, in turn, affected the 
population counts of their surrounding locales. Such was the case with 
California State University Monterey Bay (CSUMB) and the University of 
California at Santa Barbara (UCSB). As a result, the Bureau transferred 
a population of more than 1,400 between the towns in which they were 
initially counted and in which CSUMB is located and shifted a 
population of more than 2,700 between the city and the unincorporated 
area of the county in which UCSB is located. 

The Bureau's Charlotte Region, while also more active than the Bureau's 
other offices that generated internal CQR cases, seemed to be less 
methodical and comprehensive than Los Angeles in its approach. For 
example, although Charlotte geographers detected the duplicate count of 
almost 2,700 students at the University of North Carolina mentioned in 
the previous section, their research was not the result of any 
systematic review. Rather, it came about largely because of the 
curiosity of key employees in the Charlotte office, who were also 
alumni of the school. (See app. II for more details on the 
circumstances surrounding the duplicate count.)

Better Guidance and Training Could Improve Implementation: 

Vague guidance was one reason for the disparate handling of internal 
cases. For example, the CQR procedural manual indicates that the 
Bureau's 12 regional offices were to research CQR cases "as 
appropriate." However, the manual did not define whether this meant 
that the regional offices should initiate their own data reviews or 
merely verify CQR cases submitted by governmental units. More 
generally, numerous geographers we interviewed--the primary users of 
the manual--did not find it user-friendly, noting it was confusing, 
complex, or impractical. For example, a geographer pointed out that the 
manual did not have an index covering the eight chapters and 26 
appendixes, which would have helped them more quickly find information 
and procedures. In addition, we found that the manual and other 
documents did not discuss how program staff were to address Full Count 
Review issues. 

The Bureau's training was also problematic and likely added to the 
implementation disparities. For example, geographers in five regions 
told us that during training they were instructed or given the 
impression they were not to generate additional internal cases beyond 
the small number of count errors that had already been identified at 
the beginning of the program. Also, geographers in two of these regions 
told us they were specifically told not to investigate any count errors 
they found that were outside the scope of the cases that governmental 
units submitted. Conversely, geographers in the other seven regions 
said they were not restricted in any way. 

There were other training issues as well. According to the Bureau's 
draft CQR program assessment, the final version of which is pending, 
some training materials were developed at the last minute and were 
never finalized, and training began before needed software was in place 
at all the research divisions. Proper training was particularly 
important because, as the draft evaluation notes, staff assigned to the 
CQR program had census experience but limited geographic and field 
operations knowledge. Others had limited or no Census 2000 software 
program experience. 

Internal Control and Quality Assurance Problems Led the Bureau to 
Report Erroneous Data: 

Federal internal control standards call on agencies to employ edit 
checks and other procedures to ensure their information processing 
systems produce accurate and complete data.[Footnote 5] However, the 
Bureau's internal controls in reporting CQR results were insufficient 
in that we found, after a partial review, a number of instances where 
the Bureau disseminated inaccurate data on its Web site where it 
maintains an errata report that lists the CQR revisions to the 2000 
Census data. 

Specifically, after comparing data from the errata report to the 
certified numbers in the CQR case files, we found errors with the 
reporting of CQR housing, group quarters, and population counts. 
Importantly, our review found that the revised, certified figures the 
Bureau provided to affected jurisdictions were correct. This is 
significant because the affected jurisdictions could use these updated 
numbers for revenue sharing and other programs that require census 
data. However, users who obtain information from the Bureau's errata 
report--these can be people in academia, government, and the private 
sector--would not have the most up-to-date information. For example: 

* The original state-level total housing unit count for Delaware 
mistakenly excluded 30,000 housing units. 

* The revised housing unit count for the Minnehaha County portion of 
Sioux Falls, South Dakota, was underreported by almost 47,000 housing 
units. 

* The Burlington County, New Jersey, revised total housing count 
mistakenly excluded about 145,000 units. 

* The errata report excluded 8 of the 12 American Indian and Alaska 
Native Areas that had revisions to their housing, group quarter, or 
total population counts. 

The Bureau later corrected these errors after we brought them to its 
attention. 

Although the Bureau had controls in place to ensure accurate research 
and reporting, the problems we found point to the need for the Bureau 
to tighten its procedures to prevent mistakes from slipping into its 
data products. For example, the CQR manual included some quality 
control steps, such as having headquarters divisions review field 
research and results. Further, field geographers told us they consulted 
one another about questions or procedures and checked each other's 
work, and Bureau program managers had procedures in place to review 
final revisions and certify them as correct. 

Documents in the CQR case files we reviewed substantiate these 
practices. Also, managers told us they randomly checked data entered 
into the files that are the basis for the revisions posted to the 
errata report. Still, the number of errors we found after only a 
partial review of the errata files raises questions about how 
effectively the Bureau implemented the quality assurance procedures, as 
well as the quality of the data we did not review. It also underscores 
the importance of adequate control activities to prevent these problems 
from recurring. 

Web-based Errata Report Should Be More User-Friendly: 

Data users may have encountered problems trying to access certain 
information from the Web-based errata report. Additionally, because 
there was no link or cross-walk between some of the initial population 
data the Bureau released and the CQR revisions, users may have been 
unaware that some of the original numbers had been revised. 

As shown in figure 6, the Bureau's Web-based errata report presented 
revised data for states, American Indian and Alaska Native Areas, and 
other jurisdictions at the state or similar geographic level. Although 
the table had embedded links that were supposed to take users to 
revisions at lower levels of geography, these links did not always work 
and produced error messages instead. We found that unless the users' 
software and Internet access paralleled the Bureau's, users could not 
access the more detailed data using the embedded links. Bureau staff 
involved with posting the data to the Web site stated in the summer of 
2004 that they were aware of the problem, but as late as March 2005, 
the problem had yet to be fixed. 

Figure 6: CQR Table in the Bureau's 2000 Notes and Errata Report 
Showing Faulty Links to Data: 

[See PDF for image] 

[End of figure] 

At the same time, the CQR revisions may not be evident to users who 
access certain data from the 2000 Census data posted elsewhere on the 
Bureau's Web site. This is because these sites lack notes or flags 
informing users that updated figures are available in the census errata 
report.[Footnote 6] For example, the Bureau's American FactFinder Web 
site--the Bureau's primary electronic source of census data--does not 
inform users that revised data on group quarter counts, including the 
number of correctional institutions, as well as data on their 
associated populations, exist as part of the Bureau's notes and errata 
report. 

American FactFinder presents data known as Summary File 1 (SF-1), which 
is the first data set the Bureau produces from the census, and is used 
for purposes of apportionment and redistricting. While the SF-1 data 
remain unchanged, other data users may find the revised numbers better 
suited to their needs. Figure 7 illustrates the existence of two sets 
of numbers without any explanation. Summary File data from American 
FactFinder show the population for Soledad, California, as 11,263. 
However, the Bureau's errata report, which reflects the CQR revisions, 
shows the Soledad population at 23,015. Because American FactFinder 
lacks notes or links that tell users about the revised data, users 
might inadvertently obtain erroneous information. 

Figure 7: Initial Census Data on Bureau Web Site Do Not Inform Users 
That Some Numbers Have Been Revised: 

[See PDF for image] 

[End of figure] 

According to Bureau officials, while they thought about adding notes 
directing users to the CQR revisions, they decided against it because 
they thought it would confuse more people than it would help. They 
reasoned that knowledgeable users, such as county planners and state 
data center staff, are likely aware of the CQR information and would 
therefore not need to be informed about the existence of the notes and 
errata Web site. 

CQR Errors Highlight Problems with 2000 Census Address List Development 
Procedures: 

The errors uncovered by the CQR program highlight some of the 
limitations in the way in which the Bureau builds its address list for 
the decennial census, particularly in the procedures used to identify 
and locate group quarters. For the 2000 Census, the Bureau had three 
operations that were primarily designed to locate these types of 
dwellings. However, given the number of prisons and other group 
quarters geocoded to the wrong location, refinements are needed. 
Moreover, the Bureau's draft CQR program assessment found that the 
Bureau's Master Address File had numerous data entry errors including 
incorrect spellings, geocoding, and zip codes. 

To its credit, the Bureau is planning several improvements for 2010, 
including integrating its housing unit and group quarter address lists. 
This could help prevent the type of duplicate counting that occurred at 
UNC when the same dormitories appeared on both lists. Likewise, the 
Bureau's planned use of a satellite-based navigational system could 
help census workers more precisely locate street addresses. 

Better Strategic Planning and Other Actions Could Improve Future Count 
Correction Efforts: 

The CQR program was preceded by a quality assurance program called Full 
Count Review, which ran from June 2000 through March 2001 and, like 
CQR, was designed to find problems with the census data. However, 
although the Bureau planned to fold unresolved full count issues into 
CQR, many full count issues were rejected from CQR because the latter 
program had more stringent documentation requirements. As a result, the 
Bureau was unable to resolve hundreds of additional data issues. 

Numerous Unresolved Full Count Issues Could Not Be Folded into CQR as 
Planned: 

Under the Full Count Review program, analysts were to identify data 
discrepancies to clear census data files and products for subsequent 
processing or public release. Analysts did so by checking the data for 
their overall reasonableness, as well as for their consistency with 
historical and demographic census data and other census data products. 
The types of issues flagged during Full Count Review included potential 
discrepancies involving the counts and/or locations of group quarters, 
housing units, and individual households, among others. 

As we noted in our July 2002 report, Full Count Review identified 4,809 
potential data anomalies.[Footnote 7] However, of these, just five were 
corrected prior to the December 31, 2000, release of apportionment data 
and the April 1, 2001, release of redistricting data. The corrections 
included a military base, a federal medical center, and multiple 
facilities at two prisons and a college that were counted in the wrong 
locations. That the public law data were released with numerous data 
issues of unknown validity, magnitude, and impact, gave us cause for 
concern, and we noted that the Bureau missed an opportunity to verify 
and possibly improve the quality of the information. 

When the Full Count Review program began, the Bureau planned to fold 
unresolved issues from that program into CQR. Indeed, according to a 
June 2000 memo on CQR policy agreements, "a by-product of [Full Count 
Review] is documentation of unresolved issues for potential use in 
CQR." However, because the CQR program had more rigorous documentation 
requirements before it would accept a case compared to Full Count 
Review, a number of issues that were deemed suitable for Full Count 
Review but were unresolved, were rejected from CQR. 

Of the 4,804 issues remaining after Full Count Review, 2,810 issues (58 
percent), were not referred to CQR. Of the 1,994 issues (42 percent) 
that were referred to CQR, 537 were actually accepted by the program. 
The remaining 1,457 issues referred to CQR did not meet the Bureau's 
CQR documentation requirements and, consequently, the Bureau took no 
further action on them. 

The Full Count training materials we examined as part of our 2002 
review did not provide any specific guidance on the type of evidence 
analysts needed to support data issues. Rather, the materials 
instructed analysts to supply as much supporting information as 
necessary. In contrast, the CQR program had more rigorous documentation 
requirements. Guidance available on the Bureau's Web site required 
governmental units to supply maps and other evidence specific to the 
type of correction they were requesting, or the Bureau would not 
investigate their submissions. 

Simply put, Full Count Review identified hundreds of data issues but 
lacked the time to investigate the vast majority of them. Then, when 
the remaining cases were referred to CQR, most were rejected because 
they could not meet CQR's higher evidentiary bar. 

A Mechanism for Correcting Public Law Data Will Be Critical for Future 
Enumerations: 

The Bureau lacked a program specifically designed to correct individual 
count errors contained in the apportionment and redistricting data. 
Because these numbers were later found to be flawed for some 
jurisdictions, as the Bureau proceeds with its plans for the 2010 
Census, it will be important for it to explore options for reviewing 
and correcting this essential information before it is released. 

Precision is critical because, in some cases, small differences in 
population totals could potentially impact apportionment and/or 
redistricting decisions. For example, according to an analysis by the 
Congressional Research Service, under the formula used to apportion 
seats in the U.S. House of Representatives, had Utah's population count 
been 855 persons higher, it would have gained an additional 
congressional seat and North Carolina would have lost a seat. However, 
had the duplicate UNC count and other errors detected by the CQR 
program as of September 30, 2003, been uncovered prior to the release 
of the public law data, the already narrow margin determining whether 
Utah gained a House seat would have dropped to 87 persons.[Footnote 8] 
Although in this particular instance there would not have been a change 
in congressional apportionment, it illustrates how the allocation of 
House seats can be determined by small differences in population 
counts. 

Other Aspects of CQR Could Have Been Better Planned: 

Better planning could have improved the CQR program in other ways. For 
example, the Bureau's draft evaluation of CQR found, among other 
issues, that the three teams working on the planning and development 
phases of CQR should have tested implementation plans earlier in the 
process, and training materials were not based on the Bureau's 
experience in conducting the 1990 CQR program. Also, there was no 
mechanism to prioritize cases based on the magnitude of the error. As a 
result, regional offices wound up expending considerable resources on 
CQR cases that only affected a handful of dwellings. 

The draft evaluation also found that the two software applications the 
Bureau chose to administer and track CQR cases did not appear to be up 
to the task. Lost cases and documentation, poor integration with other 
applications, and the inability to produce reports were among the 
issues the evaluation cited. 

More generally, the integration and coordination issues that affected 
CQR are not unique to that program; to the contrary, our past reports 
have found that other components of the 2000 Census were not well 
planned, which unnecessarily increased the cost and risk of the entire 
enumeration.[Footnote 9] The need for better strategic planning has 
been a consistent theme in many of our past recommendations to improve 
the Bureau's approach to counting the nation's population and 
represents a significant management challenge that the Bureau will need 
to address as it looks toward 2010. 

The Bureau Is Making Count Correction Plans for 2010: 

The Bureau is beginning to develop plans for Full Count Review and CQR 
for the 2010 Census. As it does so, it will be important for it to 
develop an initiative or consolidated program that corrects both 
systemic and individual issues, and does so prior to the release of 
apportionment and redistricting data. Granted, this effort will be no 
simple task given the relatively short time between the closure of the 
local census offices and the need to release the public law data within 
the legally required time frames. 

Still, there are steps the Bureau can explore to methodically check the 
data for nationwide systemic errors, obtain local input, and 
investigate any discrepancies, and do so in an expeditious manner. One 
approach might be to consolidate and leverage CQR, Full Count Review, 
and certain other Bureau programs. 

Indeed, under the Full Count Review program, the Bureau obtained local 
input by contracting out some of the work to members of the Federal- 
State Cooperative Program for Population Estimates (FSCPE), an 
organization composed of state demographers that has worked with the 
Bureau since 1973 to ensure accurate population counts. The Bureau 
worked with FSCPE, in part, because it lacked sufficient staff to 
complete the review on its own, but also because the Bureau believed 
that the members' knowledge of the demographic characteristics of their 
states could help the Bureau examine data files and products, including 
public law data. FSCPE members reviewed data for 39 states and Puerto 
Rico; Bureau employees reviewed data for the remaining states and the 
District of Columbia without FSCPE representation in Full Count Review. 
Both sets of analysts checked the data for their overall 
reasonableness, as well as for their consistency with historical and 
demographic data, and other census data products. 

Bureau staff from its regional offices reviewed the data as well. They 
focused on identifying inconsistent demographic characteristics and did 
not necessarily concentrate on any one particular state or locality. 
Thus, the Bureau obtained local input that focused on individual states 
and smaller jurisdictions, and also performed its own, broader review. 

Verifying any data discrepancies could be accomplished by beginning the 
count correction effort as local census offices complete nonresponse 
follow-up, when enumerators are still available to investigate issues. 
In fact, the Bureau is already planning to do this to some degree in 
2010 under another operation called Coverage Improvement Follow-up 
(CIFU), where the Bureau is to call or visit housing units that have 
been designated as vacant or nonexistent but not confirmed as such by a 
second source. In the 2000 Census, CIFU began June 26, 2000, and ended 
on August 23, 2000. During that time, enumerators contacted 8.9 million 
housing units and counted 5.3 million people, according to the Bureau. 

The Bureau could explore adding the count correction workload to 
enumerators' CIFU assignments, which would enable the agency to 
reconcile possible data errors, as well as add any housing units and 
group quarters the Bureau missed during the initial enumeration (As 
noted in the background section, CQR could not add any residences that 
existed on Census Day but the Bureau had failed to count.) 

Further, the Bureau could help automate the count correction process by 
using computers to flag any data that exceed any predetermined 
tolerances. The Bureau could also develop a system to prioritize count 
correction issues to help manage its verification workload. 

Importantly, to the extent the Bureau reviews and corrects census 
counts prior to the release of the public law data, the Bureau might 
not need separate Full Count Review and CQR programs; a consolidated 
effort might be more cost effective. At a minimum, to the extent a 
separate CQR program is needed, it may not have to be as large or last 
as long because presumably the earlier program would have caught the 
bulk of the problems. 

Regardless, given the possibility that similar data errors might again 
occur during the 2010 Census, exploring options for resolving them 
prior to the release of public law data would be a sound investment. 
Reapportionment and redistricting data would be more accurate; the 
Bureau's credibility would be enhanced; and the need for a large-scale 
count correction program along the lines of CQR could be reduced or 
eliminated. 

Conclusions: 

The CQR program played an important role in improving the quality of 
data from the 2000 Census. Although the net changes in housing and 
population counts from the program were small on a national scale, in a 
number of instances, they were substantial at the local level, and 
could affect various revenue sharing formulas and other programs that 
use decennial census data. 

Because the program functioned as a safety net--a final opportunity to 
catch and correct mistakes that occurred along the chain of events that 
led to, and extended beyond Census Day 2000--the results shed light on 
the performance and limitations of certain upstream census operations, 
and areas where the Bureau should focus its efforts as its plans unfold 
for 2010. In this regard, the following is clear: although the Bureau 
puts forth tremendous effort to ensure a complete and accurate census, 
its numerous procedures and quality assurance operations will be 
challenged to stay ahead of the increasing difficulties associated with 
enumerating a population that is growing larger, more diverse, and 
increasingly hard to locate and count. 

The timing of any count correction effort will also be critical. 
Indeed, we are concerned that key decisions using data from the 2000 
Census employed figures that, for a number of jurisdictions, were later 
found to be flawed. As a result, it will be important for the Bureau to 
consider developing a count correction initiative that can complete its 
work in time to correct the public law data before that information is 
released. 

Moreover, beyond the inherent demographic obstacles to a successful 
census, the results of our CQR review echo several of our past reports 
on other aspects of the census, which note that some of the Bureau's 
difficulties stem from a lack of adequate strategic planning and other 
management challenges. Ultimately, the success of the 2010 Census will 
hinge on the extent to which senior Bureau leadership resolves these 
challenges. 

With this in mind, resolute action is needed across three fronts. 
First, it will be important for the Bureau to ensure, via thorough 
field testing, that its planned changes to its address list development 
procedures help resolve the geocoding and other operational problems 
revealed by CQR. Second, it will be important for the Bureau to improve 
its count correction efforts by designing a program that can 
systematically and consistently review the public law data and make any 
corrections prior to the release of those figures. Third, it will be 
important for the Bureau to address persistent strategic planning 
challenges. 

Recommendations for Executive Action: 

To help ensure the nation has the best possible data for purposes of 
apportionment, redistricting, and other uses of census data, we 
recommend that the Secretary of Commerce direct the Bureau to improve 
its count correction efforts for the 2010 Census by taking such actions 
as: 

1. Thoroughly testing improvements to the Bureau's group quarters and 
other address list development activities to help ensure the Bureau has 
resolved geocoding and other problems with its master address file. 

2. Consolidating Full Count Review and CQR into a single program that 
systematically reviews and corrects any errors in the public law data 
prior to their release. 

3. Expediting count correction efforts by initiating data reviews 
toward the end of nonresponse follow-up, when the Bureau starts getting 
complete data for geographic entities, and enumerators are available to 
help investigate any discrepancies. As part of this effort, the Bureau 
should consider using computers to systematically search for possible 
errors nationwide by checking data at the appropriate level of 
geography to ensure population, housing unit, and group quarter counts, 
as well as demographic characteristics, appear reasonable and are 
consistent with population estimates. Those areas that are outside of 
predetermined tolerances should be flagged for further review. The 
Bureau should also pay special attention to ensure group quarters are 
properly geocoded and counted. 

4. Prioritizing the investigation of errors based on the magnitude of 
the suspected error or similar triaging formula. 

5. Ensuring that instructions on the Bureau's Web site make it clear 
that updated information exists and that users can readily access this 
information. 

6. Improving the Bureau's quality assurance procedures to help ensure 
there are no mistakes in the data the Bureau posts on its Web site. 

7. Enhancing the training and guidance provided to regional offices to 
help ensure they share the same understanding of their roles and 
responsibilities and will implement the program consistently. 

8. Addressing persistent strategic management challenges, in part, 
through early testing to help ensure information systems, training, and 
other activities are fully integrated. 

Agency Comments and Our Evaluation: 

The Acting Deputy Secretary of Commerce provided written comments on a 
draft of this report on May 20, 2005, which are reprinted in appendix 
III. Commerce stated that "the report provides a good overview of 
program results and makes several useful observations and 
recommendations," and specifically agreed with our finding that the 
process for conducting internal reviews was not consistently 
implemented. More generally, however, Commerce believes the 
shortcomings we describe reflect "a fundamental misunderstanding of the 
goals of the CQR program," and noted that our observations and 
recommendations indicate we believe that CQR should have been designed 
to correct the public law data before they were released during the 
2000 Census. 

Our concern over the CQR program centers on the way it was implemented 
in 2001, rather than the fact that the Bureau did not design the 
program to correct the apportionment and redistricting numbers. We 
agree with Commerce that this was not the intent of CQR and, as 
Commerce notes, we acknowledge this in our report. At the same time, 
based on the lessons learned from the 2000 Census, enumeration errors 
are almost inevitable. Thus, our recommendations focus on the future, 
and specifically, the importance of developing mechanisms for the 2010 
Census to review and correct errors in the public law data to the 
greatest extent possible before they are released. We have clarified 
the report to better reflect GAO's position. 

Commerce specifically addressed two of our eight recommendations, 
disagreeing with both of them. With respect to our recommendation to 
consolidate Full Count Review and CQR into a single program for the 
2010 Census, Commerce noted that preliminary counts at the census tract 
or block level are needed to conduct an effective CQR program, and that 
information is not available until close to the deadline for releasing 
the apportionment data. Commerce maintains there would be little 
opportunity for local entities to review the counts and document 
potential problems and even less time for the Bureau to conduct the 
necessary research and field work. 

Our report recognizes that it would be a challenge for the Bureau to 
review and correct census figures and still release the public law data 
by the legally required time frames. Still, as we note in our report, 
we believe the Bureau could expedite the process by taking such steps 
as (1) using computers to check census data for their overall 
reasonableness and flagging areas that exceed predetermined tolerances; 
(2) focusing on known trouble spots such as group quarters; and (3) 
beginning the review process earlier, such as, when local census 
offices complete their nonresponse follow-up efforts. 

Moreover, as we state in the report, during the 2000 Census, the Bureau 
already had programs in place that obtained local input on the census 
numbers before the release of the public law data (Full Count Review), 
and conducted extensive field operations to investigate certain 
discrepancies (Coverage Improvement Follow-up). We believe that it will 
be important for the Bureau to not simply replicate these programs for 
the 2010 Census or make incremental improvements, but to see whether 
these programs could be better leveraged and be more strategically 
employed to improve the accuracy of the apportionment and redistricting 
data. 

The other recommendation that Commerce specifically addressed was our 
call for the Bureau in 2010 to prioritize the investigation of errors 
based on the magnitude of the suspected problem. Commerce maintains 
that the Bureau's policy in 2000 was to handle cases in the order they 
were received from local jurisdictions, and asserts this was a fair and 
reasonable practice. While this practice is not unreasonable, we 
continue to believe that it would be more cost-effective for the Bureau 
to give priority to those cases where it could achieve a greater return 
on its investment in resources (especially given our findings involving 
group quarters such as prisons and college dormitories that affected 
relatively large population clusters). Our recommendation echoes the 
Bureau's draft evaluation of the CQR program, which noted that regional 
offices expended considerable resources on CQR cases that affected only 
a handful of dwellings. Moreover, as we state in our report, 
prioritizing the Bureau's workload could help expedite the count 
correction process. 

Commerce's comments also included some technical corrections and 
suggestions where greater clarity was needed. We revised the report as 
appropriate. 

We will send copies of this report to the Chairman of the House 
Committee on Government Reform, the Secretary of Commerce, and the 
Director of the U.S. Census Bureau. Copies will be made available to 
others on request. This report will also be available at no charge on 
GAO's home page at [Hyperlink, http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-6806 or [Hyperlink, williamso@gao.gov]. Contact 
points for our Office of Congressional Relations and Public Affairs may 
be found on the last page of this report. GAO staff who made major 
contributions to this report are listed in appendix IV. 

Signed by: 

Orice M. Williams: 
Director: 
Strategic Issues: 

[End of section]

Appendixes: 

Appendix I: Change in State Populations As a Result of Count Question 
Resolution Program: 

State: U.S. Total; 
2000 Census total population: 281,421,906; 
CQR total population: 281,424,603; 
Total population change: 2,697. 

State: Alabama; 
2000 Census total population: 4,447,100; 
CQR total population: 4,447,351; 
Total population change: 251. 

State: Alaska; 
2000 Census total population: 626,932; 
CQR total population: 626,931; 
Total population change: -1. 

State: Arizona; 
2000 Census total population: 5,130,632; 
CQR total population: 5,130,632; 
Total population change: 0. 

State: Arkansas; 
2000 Census total population: 2,673,400; 
CQR total population: 2,673,400; 
Total population change: 0. 

State: California; 
2000 Census total population: 33,871,648; 
CQR total population: 33,871,653; 
Total population change: 5. 

State: Colorado; 
2000 Census total population: 4,301,261; 
CQR total population: 4,302,015; 
Total population change: 754. 

State: Connecticut; 
2000 Census total population: 3,405,565; 
CQR total population: 3,405,602; 
Total population change: 37. 

State: Delaware; 
2000 Census total population: 783,600; 
CQR total population: 783,600; 
Total population change: 0. 

State: District of Columbia; 
2000 Census total population: 572,059; 
CQR total population: 572,059; 
Total population change: 0. 

State: Florida; 
2000 Census total population: 15,982,378; 
CQR total population: 15,982,824; 
Total population change: 446. 

State: Georgia; 
2000 Census total population: 8,186,453; 
CQR total population: 8,186,816; 
Total population change: 363. 

State: Hawaii; 
2000 Census total population: 1,211,537; 
CQR total population: 1,211,537; 
Total population change: 0. 

State: Idaho; 
2000 Census total population: 1,293,953; 
CQR total population: 1,293,956; 
Total population change: 3. 

State: Illinois; 
2000 Census total population: 12,419,293; 
CQR total population: 12,419,647; 
Total population change: 354. 

State: Indiana; 
2000 Census total population: 6,080,485; 
CQR total population: 6,080,517; 
Total population change: 32. 

State: Iowa; 
2000 Census total population: 2,926,324; 
CQR total population: 2,926,382; 
Total population change: 58. 

State: Kansas; 
2000 Census total population: 2,688,418; 
CQR total population: 2,688,824; 
Total population change: 406. 

State: Kentucky; 
2000 Census total population: 4,041,769; 
CQR total population: 4,042,285; 
Total population change: 516. 

State: Louisiana; 
2000 Census total population: 4,468,976; 
CQR total population: 4,468,958; 
Total population change: -18. 

State: Maine; 
2000 Census total population: 1,274,923; 
CQR total population: 1,274,923; 
Total population change: 0. 

State: Maryland; 
2000 Census total population: 5,296,486; 
CQR total population: 5,296,507; 
Total population change: 21. 

State: Massachusetts; 
2000 Census total population: 6,349,097; 
CQR total population: 6,349,105; 
Total population change: 8. 

State: Michigan; 
2000 Census total population: 9,938,444; 
CQR total population: 9,938,480; 
Total population change: 36. 

State: Minnesota; 
2000 Census total population: 4,919,479; 
CQR total population: 4,919,492; 
Total population change: 13. 

State: Mississippi; 
2000 Census total population: 2,844,658; 
CQR total population: 2,844,656; 
Total population change: -2. 

State: Missouri; 
2000 Census total population: 5,595,211; 
CQR total population: 5,596,683; 
Total population change: 1,472. 

State: Montana; 
2000 Census total population: 902,195; 
CQR total population: 902,195; 
Total population change: 0. 

State: Nebraska; 
2000 Census total population: 1,711,263; 
CQR total population: 1,711,265; 
Total population change: 2. 

State: Nevada; 
2000 Census total population: 1,998,257; 
CQR total population: 1,998,257; 
Total population change: 0. 

State: New Hampshire; 
2000 Census total population: 1,235,786; 
CQR total population: 1,235,786; 
Total population change: 0. 

State: New Jersey; 
2000 Census total population: 8,414,350; 
CQR total population: 8,414,347; 
Total population change: -3. 

State: New Mexico; 
2000 Census total population: 1,819,046; 
CQR total population: 1,819,046; 
Total population change: 0. 

State: New York; 
2000 Census total population: 18,976,457; 
CQR total population: 18,976,821; 
Total population change: 364. 

State: North Carolina; 
2000 Census total population: 8,049,313; 
CQR total population: 8,046,485; 
Total population change: -2,828. 

State: North Dakota; 
2000 Census total population: 642,200; 
CQR total population: 642,200; 
Total population change: 0. 

State: Ohio; 
2000 Census total population: 11,353,140; 
CQR total population: 11,353,145; 
Total population change: 5. 

State: Oklahoma; 
2000 Census total population: 3,450,654; 
CQR total population: 3,450,652; 
Total population change: -2. 

State: Oregon; 
2000 Census total population: 3,421,399; 
CQR total population: 3,421,436; 
Total population change: 37. 

State: Pennsylvania; 
2000 Census total population: 12,281,054; 
CQR total population: 12,281,054; 
Total population change: 0. 

State: Rhode Island; 
2000 Census total population: 1,048,319; 
CQR total population: 1,048,319; 
Total population change: 0. 

State: South Carolina; 
2000 Census total population: 4,012,012; 
CQR total population: 4,011,816; 
Total population change: -196. 

State: South Dakota; 
2000 Census total population: 754,844; 
CQR total population: 754,844; 
Total population change: 0. 

State: Tennessee; 
2000 Census total population: 5,689,283; 
CQR total population: 5,689,267; 
Total population change: -16. 

State: Texas; 
2000 Census total population: 20,851,820; 
CQR total population: 20,851,790; 
Total population change: -30. 

State: Utah; 
2000 Census total population: 2,233,169; 
CQR total population: 2,233,198; 
Total population change: 29. 

State: Vermont; 
2000 Census total population: 608,827; 
CQR total population: 608,827; 
Total population change: 0. 

State: Virginia; 
2000 Census total population: 7,078,515; 
CQR total population: 7,079,030; 
Total population change: 515. 

State: Washington; 
2000 Census total population: 5,894,121; 
CQR total population: 5,894,141; 
Total population change: 20. 

State: West Virginia; 
2000 Census total population: 1,808,344; 
CQR total population: 1,808,350; 
Total population change: 6. 

State: Wisconsin; 
2000 Census total population: 5,363,675; 
CQR total population: 5,363,715; 
Total population change: 40. 

State: Wyoming; 
2000 Census total population: 493,782; 
CQR total population: 493,782; 
Total population change: 0. 

State: Puerto Rico; 
2000 Census total population: 3,808,610; 
CQR total population: 3,808,603; 
Total population change: -7. 

Source: GAO analysis of U.S. Census Bureau data. 

[End of table]

[End of section]

Appendix II: Human Error and Other Factors Contributed to University of 
North Carolina Counting Errors: 

The duplicate counting of nearly 2,700 students at the University of 
North Carolina (UNC) at Chapel Hill during the 2000 Census resulted 
from a combination of factors. The incident is interesting because it 
shows how the various safety nets the Bureau has built to ensure an 
accurate count can be undermined by human error, the limitations of 
census-taking operations, and other events that in some cases occur 
years before Census Day (April 1, 2000). 

The duplicate count was discovered after CQR began when the director of 
the Charlotte regional office (a UNC graduate), asked one of her 
geographers (also a UNC graduate), to see whether the UNC dormitories 
were counted in their correct locations. According to the geographer, 
the director's curiosity was aroused after the CQR program found 
problems with the geocoding of dormitories at other schools in the 
Charlotte region. The geographer told us he initiated an internal CQR 
case in the summer of 2001 after discovering that two UNC dormitories 
were geocoded to the wrong census block. Upon further research, where 
he reviewed information from the census address file and the UNC Web 
site, the geographer concluded that, in addition to the geocoding 
error, a large number of dormitories and their occupants were counted 
in error. 

Ultimately, by matching census records, the Bureau determined that 
1,583 dormitory rooms in 26 buildings--and the 2,696 students who had 
resided in them--were included twice in the 2000 Census. On the basis 
of our interviews with Bureau staff and review of pertinent documents, 
the following sequence of events led to these erroneous enumerations: 

The Bureau divides the places where people live into two broad 
categories: group quarters, which include prisons, dormitories, and 
group homes; and housing units, which consist of single family homes, 
apartments, and mobile homes. During the 2000 Census, the Bureau had 
distinct procedures for building its group quarters and housing unit 
address lists and enumerating their residents. For example, the Bureau 
typically enumerates college dormitories by working with schools to 
distribute census questionnaires to students. Conversely, the Bureau 
enumerates residents of housing units by delivering questionnaires 
directly to them through the mail. In the UNC situation, the 26 UNC 
dormitories were listed correctly in the Bureau's group quarters 
database and incorrectly in the Bureau's housing unit database. 

Concerned there could be systemic issues with the Bureau's address 
list, staff at the Bureau's headquarters investigated the source of the 
problem following the initial discovery by Charlotte employees. The 
headquarters review found that the dormitories were improperly included 
in the U.S. Postal Service's address file, which it initially shared 
with the Bureau in November 1997 and continued to update through early 
2000. The Bureau uses this database to help build its housing unit 
address list. Specifically, the Bureau discovered that the data field 
that normally contains a street address erroneously contained a unit 
number and the name of a UNC dormitory. The Bureau had no explanation 
for how the dormitory names got into the U.S. Postal Service's address 
file. 

Other procedures designed to verify census addresses produced 
conflicting results, compounding the problem. One procedure in 1998 
mistakenly confirmed the dormitories as housing units, while another 
procedure--called block canvassing--correctly flagged the addresses for 
deletion from the Bureau's housing unit address list. However, under 
the Bureau's protocols, to ensure an address was not improperly removed 
from the census, an address had to be flagged twice to be deleted. 
During nonresponse follow-up in 2000, where enumerators visited housing 
units that failed to send back the questionnaires that were mailed to 
them, the Bureau had a third opportunity to uncover the error. Because 
the enumerators involved in this operation provided inconsistent 
information, the Bureau ultimately did not delete any housing units 
included in the initial census. 

As part of the CQR case analysis, staff in the Bureau's Decennial 
Statistical Studies Division checked the Bureau's address file for any 
records that contained the word "dorm" in the address field to 
determine whether a similar duplication occurred at other schools. This 
would have picked up the word "dormitory" and its variants. On the 
basis of this search, the Bureau concluded that a similar issue was not 
problematic elsewhere in the country. 

[End of section]

Appendix III: Comments from the Department of Commerce: 

THE DEPUTY SECRETARY OF COMMERCE: 
Washington, D.C. 20230: 

May 18, 2005: 

Ms. Orice M. Williams: 
Director:
Strategic Issues:
U.S. Government Accountability Office:
Washington, DC 20548-0001: 

Dear Ms. Williams: 

The U.S. Department of Commerce appreciates the opportunity to comment 
on the U.S. Government Accountability Office draft report entitled Data 
Quality: Improvements to Count Correction Efforts Could Produce More 
Accurate Census Data (GAO-OS-463). l enclose the Department's comments 
on this report. 

Sincerely, 

Signed by: 

David A. Sampson: 
(Acting): 

Enclosure: 

U.S. Department of Commerce: 

Comments on U.S. Government Accountability Office Draft Report Entitled 
"Data Quality: Improvements to Count Correction Efforts Could Produce 
More Accurate Census Data" GAO-05-463: 

The Department of Commerce has the following general and specific 
comments on this draft audit report. 

1. According to this report, the Government Accountability Office (GAO) 
was asked to review the results of the Count Question Resolution (CQR) 
program and to assess whether the program was consistently implemented 
across the country. While the report provides a good overview of 
program results and makes several useful observations and 
recommendations, we believe almost all of the shortcomings described by 
the GAO illustrate a fundamental misunderstanding of the goals of the 
CQR program rather than criticisms of its results or implementation. 

The Census 2000 CQR program was not designed or publicized as a 
mechanism to correct the census results for apportionment or 
redistricting purposes. This was stated explicitly in the January 22, 
2001, Federal Register notice that announced this program. While the 
report acknowledges this fact at the outset (see page 2), it then goes 
on to make various observations and recommendations that indicate the 
GAO believes the program should have been designed to achieve this goal 
for Census 2000 and that the U.S. Census Bureau should establish this 
as a goal for the 2010 Census. 

We strongly disagree with such conclusions and recommendations 
concerning the Census 2000 CQR program. The Census Bureau does the best 
job it canto correctly enumerate the population within very strict and 
stringent time constraints imposed by federal law (13 U.S.C. § 141 and 
P.L. 94-171). At the end of that effort, errors inevitably will remain 
in the results. Nonetheless, at specified points in time (December 31 
of the census year for apportionment, April 1 of the following year for 
redistricting), the census results must be reported so that the 
Congress, the states, and local and tribal governments can use them for 
the purposes prescribed in those same laws. The Census Bureau does not 
have the authority to delay the reporting of those results, or to 
continue correcting those results over some period of time after the 
fact. 

The Census Bureau does, however, have the authority and responsibility 
to evaluate the results of the decennial census and to provide whatever 
information it can to assist all data users in understanding the 
limitations of census data. Further, it has the authority to conduct 
Special Censuses and other efforts aimed at providing more current data 
about the population of any particular jurisdiction. As you note, 
apportionment and redistricting are not the only uses of decennial 
census results. Over the course of the decade, decennial census results 
are used to distribute hundreds of billions of dollars in federal, 
state, local, and tribal funding. While many of the laws that govern 
these programs specify the use of the most recent decennial census 
results, some of them permit the use of other official Census Bureau 
data. This is why the Census Bureau offers the Special Census program 
and why it produces intercensal estimates and projections of the 
population. And, it was to this end that the Census Bureau created the 
CQR program to provide local governments a mechanism to identify 
potential problems in Census 2000 data that might have resulted from 
census processing errors and that might have affected the 
jurisdiction's funding streams over the decade. 

If any updates were made to a jurisdiction's population and housing 
unit counts as a result of the CQR program, a statement documenting and 
certifying the updated figures was sent from the Director of the Census 
Bureau to the highest elected official of the jurisdiction. A copy of 
the statement also was sent to each affected jurisdiction's Secretary 
of State and to other state officials. The statement included the 
following language: 

"Census counts used for Congressional apportionment and legislative 
redistricting, and Census 2000 data products, will remain unchanged. 
The Census Bureau will include the corrections in the errata 
information to be made available via the Internet on the American 
FactFinder system and used specifically to modify the decennial census 
file for use in yearly postcensal estimates beginning in December 
2002." 

While the GAO may not be suggesting that the Census Bureau should have 
delayed (or issued updated) counts for apportionment and redistricting 
based on the Census 2000 CQR program, it seems clear the GAO believes 
the Census Bureau should have designed the 2000 program so that it 
could have been completed before those counts were issued. In addition, 
the report recommends that the Census Bureau pursue such a design for 
the 2010 Census. Although the Census Bureau is exploring ways to 
improve the CQR program for the 2010 Census, it does not believe such a 
design is feasible: 

* By its very nature, the CQR program requires census counts for very 
small areas. The Census Bureau believes it would be infeasible and 
unproductive to provide preliminary counts only at the city or county 
level and then expect jurisdictions to detect and pinpoint problems 
that could be reviewed in any systematic or timely fashion. 

* The preliminary counts at the census tract or block level that are 
needed to conduct an effective CQR program are not available until the 
apportionment deadline is fast approaching. This leaves virtually no 
time for local jurisdictions to review the counts and assemble 
documentation supporting any potential problems, and even less time for 
the Census Bureau to conduct necessary research and field work to 
determine what corrections are needed. 

2. The report does not acknowledge until page 13 that the number of 
potential count problems identified by local jurisdictions was 
extremely small compared to the total number of jurisdictions (1,180 
out of about 39,000-only about 3%) and that the total population change 
resulting from all CQR cases was even smaller relative to the total 
U.S. population (less than 3,000 persons out of 281 million-barely one 
one-thousandth of one percent). The Census Bureau agrees, as also 
stated on page 13, that some of these changes may have had relatively 
large effects for a particular location. However, the summary of 
findings on the cover page of the report does not provide any of this 
context. Further, the first sentence of the report (the CQR program) 
says "corrected numbers affecting 47 states and over 1,180 governmental 
units" and then goes on to cite a few cases that involved relatively 
large count changes. Most count changes resulting from CQR were very 
small. For all of these reasons, we believe the summary of findings 
presented on the cover page of the report is very misleading as to the 
magnitude of problems revealed by this effort. 

3. Based on its review, the GAO concluded that different Census Bureau 
regional offices implemented the program to different degrees. While we 
agree this is a valid criticism with respect to the internal review 
process the Census Bureau conducted in parallel with the CQR, we do not 
believe it is valid with respect to the way the Census Bureau handled 
external CQR challenges from local jurisdictions. For those external 
challenges, the Census Bureau believes that all regions followed the 
same procedures for their investigations. 

The internal review process involved Census Bureau staff, the Federal- 
State Cooperative Program for Population Estimates, and others who were 
checking Census 2000 data for reasonableness, internal and intra- 
product consistency, and consistency with historical and external data 
sources. Reviewers identified, addressed, and/or explained issues or 
problems related to coverage, content, processing, and geocoding. 
Unresolved potential problems were forwarded to the CQR staff for 
additional analysis. Changes made as a result of this internal review 
and/or research were incorporated into the CQR process and documented 
the same way as changes based on external CQR challenges from 
jurisdictions. 

The Census Bureau agrees the process for conducting internal reviews 
was not planned or implemented as systematically as the review of 
external challenges submitted by jurisdictions. 

4. The report also criticizes the Census Bureau for not prioritizing 
CQR cases based on the size of the problem or some other measure of 
criticality. The Census Bureau's policy was to handle the cases in the 
order they were received from local jurisdictions, and we believe that 
was fair and reasonable for a program where cases were submitted over a 
two-year period. The Census Bureau also gave higher priority to 
external challenges from jurisdictions than to internal review cases. 

5. Page 6 of the report states that CQR was one of several Census 
Bureau quality assurance activities intended to improve the accuracy of 
Census 2000 data. We believe this statement is misleading because it 
implies the program was part of those coverage improvement operations 
built into the decennial census operations that produced the final 
apportionment and redistricting data. As stated earlier, this is not 
the case. 

The last paragraph of page 12 includes a statement that there were no 
CQR count corrections for the District of Columbia. However, while the 
CQR program did not change the population count for the District of 
Columbia, the program did identify some geocoding errors within the 
District of Columbia. Figure 3 also should be revised to include the 
District of Columbia. 

6. The paragraph in the middle of page 30 should be revised to state 
that the Full Count Review identified hundreds of potential data 
issues. 

[End of section]

Appendix IV: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Orice Williams, (202) 512-6806: 

Acknowledgments: 

In addition to the contact named above, Robert Goldenkoff, Keith Steck, 
Timothy Wexler, Robert Parker, Michael Volpe, Andrea Levine, and Elena 
Lipson made key contributions to this report. 

(450308): 

FOOTNOTES

[1] 13 U.S.C. §§ 141(b) - (c). 

[2] Based on our assessment of the data, we found the case file 
information and program results sufficiently reliable for our review. 

[3] The Bureau also made corrections to governmental units classified 
as American Indian/Alaska Native Areas. 

[4] Because a population increase in one government entity was 
typically offset by a loss in population in a neighboring entity (or 
vice-versa), there was generally little net change in population counts 
at the national and state levels as a result of the CQR program and no 
effect on apportionment. 

[5] GAO, Internal Control Management and Evaluation Tool, GAO-01-1008G 
(Washington, D.C.: Aug. 1, 2001). 

[6] The Bureau and American FactFinder home pages do not list or 
provide direct links to the 2000 Census notes and errata report or the 
CQR program Web site. However, the Bureau's Census 2000 Gateway Web 
site provides links to both of them and its American FactFinder Web 
site provides an indirect link to the notes and errata through that 
site's data sets. 

[7] GAO, 2000 Census: Refinements to Full Count Review Program Could 
Improve Future Data Quality, GAO-02-562 (Washington, D.C.: July 3, 
2002). 

[8] Congressional Research Service, House Apportionment: Could Census 
Corrections Shift a House Seat?, RS21638 (Washington, D.C.: Oct. 8, 
2003). 

[9] See for example, GAO, 2000 Census: Lessons Learned for Planning a 
More Cost-Effective 2010 Census, GAO-03-40 (Washington, D.C.: Oct. 31, 
2002), 14 - 17, and GAO, 2010 Census: Cost and Design Issues Need to Be 
Addressed Soon, GAO-04-37 (Washington, D.C.: Jan. 15, 2004), 25 - 31. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director,

NelliganJ@gao.gov

(202) 512-4800

U.S. Government Accountability Office,

441 G Street NW, Room 7149

Washington, D.C. 20548: