This is the accessible text file for GAO report number GAO-05-644 
entitled 'Information Quality Act: National Agricultural Statistics 
Service Implements First Steps, but Documentation of Census of 
Agriculture Could Be Improved' which was released on September 23, 
2005. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Committees: 

September 2005: 

Information Quality Act: 

National Agricultural Statistics Service Implements First Steps, but 
Documentation of Census of Agriculture Could Be Improved: 

GAO-05-644: 

GAO Highlights: 

Highlights of GAO-05-644, a report to congressional committees: 

Why GAO Did This Study: 

The Information Quality Act (IQA) required the Office of Management and 
Budget to issue guidelines for ensuring the quality, objectivity, 
utility, and integrity of information disseminated by federal agencies. 
As part of our long-term examination of the quality of federal 
information, under the Comptroller General’s authority, we reviewed how 
the act was implemented by the National Agricultural Statistics Service 
(NASS), and assessed the transparency of the documentation supporting 
its Census of Agriculture. NASS is part of the U.S. Department of 
Agriculture (USDA). 

What GAO Found: 

NASS fulfilled its various procedural responsibilities and reporting 
requirements under the Office of Management and Budget’s (OMB) 
guidelines for implementing the act. For example, NASS drafted its own 
implementation guidance, and developed a mechanism allowing affected 
parties to request the correction of information they believe is of 
poor quality. As a result of our review, NASS has also taken steps to 
better document the criteria it uses to evaluate data users’ input on 
the content of the Census of Agriculture. 

The Census of Agriculture Provides a Detailed Picture of U.S. Farms and 
Ranches: 

[See PDF for image] 

[End of figure] 

Building on these efforts, better documentation could improve the 
transparency of census data products. For example, the nine key 
products from the 2002 Census we examined lacked, among other things, 
discussions of any data limitations. This is contrary to NASS’s own 
guidelines for ensuring transparency, which stress the importance of 
describing the methods, data sources, and other items to help users 
understand how the information was designed and produced. 

Although NASS complied with OMB’s requirement to establish a mechanism 
under IQA to address requests to correct information, NASS has not 
documented its approach for handling correction requests not filed 
under IQA (NASS handles these correction requests using an existing, 
informal method). Agency officials told us that data users have been 
satisfied with the way NASS had responded to these requests. However, 
because NASS does not document its informal procedures for handling 
correction requests and lacks a recordkeeping system to log and track 
them, NASS could not provide us with specific data on the number of 
such requests it has handled, the nature of those requests, and whether 
and how they were addressed. 

What GAO Recommends: 

To help enhance the transparency of the Census of Agriculture, we 
recommend that the Secretary of Agriculture direct NASS to 
(1) ensure that census products fully address NASS’s guidelines for 
data documentation or at least contain links to such information, and 
(2) document and post on NASS’s Web site its procedures for handling 
data correction requests not filed under IQA. NASS agreed with our 
findings and described the steps it is taking in response to our 
recommendations. Additional actions, consistent with our 
recommendations, would enhance NASS’s efforts. 

www.gao.gov/cgi-bin/getrpt?GAO-05-644. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Orice Williams at (202) 
512-6806 or williamso@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Objectives, Scope, and Methodology: 

NASS Met the Procedural and Reporting Requirements of OMB's IQA 
Guidelines: 

Better Documentation Could Improve the Transparency of Data Products 
and Correction Procedures: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendixes: 

Appendix I: Comments from the Department of Agriculture: 

Appendix II: How Census of Agriculture Reports Address Various 
Documentation Elements: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: NASS Addressed OMB's Agencywide Guidelines for Implementing 
IQA: 

Table 2: Census Reports Need More Robust Documentation: 

Table 3: NASS Is Using More Extensive Outreach to Develop the 2007 
Census Compared to 2002: 

Letter September 23, 2005: 

The Honorable Saxby Chambliss: 
Chairman: 
The Honorable Tom Harkin: 
Ranking Democratic Member: 
Committee on Agriculture, Nutrition and Forestry: 
United States Senate: 

The Honorable Collin C. Peterson: 
Ranking Democratic Member: 
Committee on Agriculture: 
House of Representatives: 

The information disseminated by federal agencies is a critical 
strategic asset. For example, data collected for statistical purposes 
provide indicators of the economic and social well-being of the nation, 
while health, safety, environmental, and other scientific data help 
inform agencies' rule-making activities. Given the widespread use and 
impact of federal information, it is important for it to meet basic 
quality standards. 

Section 515 of the Treasury and General Government Appropriations Act 
for Fiscal Year 2001--legislation that has come to be known as the 
Information Quality Act (IQA)[Footnote 1]--required the Office of 
Management and Budget (OMB) to issue governmentwide guidelines that 
provide policy and procedural guidance to federal agencies for ensuring 
and maximizing the quality, objectivity, utility, and integrity of 
information disseminated by federal agencies. 

OMB's guidelines, issued in final form in February 2002, directed 
agencies covered by the act to issue their own quality guidelines, and 
noted that, where appropriate, agencies should support their data with 
transparent documentation. OMB's guidelines also required agencies to, 
among other actions, report annually to the Director of OMB on the 
number and nature of complaints received regarding compliance with the 
guidelines, and establish an administrative mechanism whereby affected 
parties can request the correction of information they deem to be of 
poor quality. 

As part of our long-term examination of the collection, dissemination, 
and quality of federal information, we are reviewing the governmentwide 
implementation of the IQA. As an initial step in our research on the 
quality of statistical data, under the Comptroller General's authority, 
we conducted a case study of the National Agricultural Statistics 
Service (NASS), and its Census of Agriculture. NASS is a statistical 
agency within the U.S. Department of Agriculture (USDA). We selected 
the census because it is one of the largest government surveys with a 
universe of 2.3 million respondents and an estimated paperwork burden 
in excess of 1.3 million burden hours.[Footnote 2] The last census took 
place in 2002 and the next census is scheduled for 2007. 

Specifically, our objectives were to (1) review how NASS met OMB's 
guidelines covering the IQA, and (2) examine the transparency of the 
documentation behind the Census of Agriculture's processes and 
products, both for the recently completed work on the 2002 Census and 
the efforts underway for the 2007 Census. To achieve both objectives, 
we reviewed OMB's and NASS's information quality guidelines and other 
relevant documents. We also interviewed senior agency officials and 
other personnel responsible for implementing the census including the 
NASS Administrator, Associate Administrator, and Deputy Administrator 
for Programs and Products. 

To evaluate the transparency of census products, we reviewed nine 
census products--eight reports and the Frequently Asked Questions (FAQ) 
section on NASS's 2002 Census Web site--to determine the extent to 
which NASS followed its own documentation guidelines. To obtain an 
external perspective of how NASS processes and products address the IQA 
guidelines, we interviewed six data users from different types of 
agricultural and research organizations. We selected these six because 
they use census data on a regular basis and have attended NASS's 
outreach meetings. Additional information on our approach is provided 
in the Objectives, Scope, and Methodology section below. 

We performed our work in Washington, D.C., from August 2004 through 
August 2005 in accordance with generally accepted government auditing 
standards. 

Results in Brief: 

NASS fulfilled its various procedural responsibilities and reporting 
requirements under OMB's guidelines. For example, NASS (1) drafted its 
own implementation guidelines and posted them on its Web site, (2) 
developed an administrative mechanism allowing affected persons to seek 
the correction of information, (3) designated an official responsible 
for ensuring NASS's compliance with OMB's requirements, and (4) 
reported to OMB the number of complaints received regarding data 
quality they received under the IQA.[Footnote 3]

With respect to the transparency of the documentation underlying the 
Census of Agriculture's data products and processes, as a result of our 
review, NASS has taken steps to better document the criteria it uses to 
evaluate data users' suggestions on the questionnaire content. The 
development of the 2002 and 2007 Censuses was led by the Census Content 
Team, which consisted of experienced NASS statisticians. The 2002 Team 
assessed users' input using a documented set of criteria, which 
considered such factors as whether the questionnaire items were 
mandated by Congress or whether they would provide data on current 
agricultural issues. However, because of staff turnover and 
reassignments, the 2007 Team was unaware of the 2002 criteria, and 
initially relied on professional judgment rather than documented 
factors to evaluate input on the 2007 Census. According to NASS, our 
review raised the 2007 Team's awareness of the earlier criteria, and it 
has since developed similar documentation that it will use in the 
future. This approach is more consistent with NASS's own IQA guidelines 
concerning transparency, and could help create a closer link between 
the questions included in the census and evolving agricultural policy 
requirements, and thus a more cost-effective data collection program. 
Documenting the content selection criteria will also guard against the 
loss of institutional memory to the extent there is further turnover in 
Content Team membership. 

Building on these efforts, better documentation could improve the 
transparency of census data products. Although NASS's guidelines for 
ensuring transparency stress the importance of describing the methods, 
data sources, assumptions, and other items in order to help users 
understand how information was designed and produced, the eight census 
reports as well as the FAQ section we examined lacked a discussion of 
such important documentation practices as the limitations of the data; 
the impact of imputations, by item; and whether any of the collected 
data have been suppressed for data quality reasons. 

The transparency of NASS's procedures for handling data correction 
requests filed outside of the IQA could also be improved. NASS complied 
with OMB's requirement to establish a mechanism to process requests for 
correction of disseminated information under the IQA. As part of this 
process, an individual must state that their request is being submitted 
under the IQA. To date, no individual has done so. NASS handles all 
other correction requests using its existing informal, undocumented 
procedures. Agency officials told us that it has resolved the informal 
requests it has handled so far to the data users' satisfaction. 
Nevertheless, because NASS does not document its informal procedures 
for handling correction requests and lacks a recordkeeping system to 
log and track them, NASS could not provide us with specific data on the 
number of requests it has handled, the nature of those requests, and 
whether and how they were addressed. 

To help enhance the transparency of the Census of Agriculture's 
processes and products, we recommend that the Secretary of Agriculture 
direct NASS to (1) ensure its products fully address its own 
requirements for transparent data documentation or at least contain 
links to such information and (2) document and post on its Web site its 
procedures for handling data correction requests not filed under the 
IQA and track the disposition of those requests. 

The NASS Administrator provided written comments on a draft of this 
report (see app. I). NASS said the information was insightful, and 
noted it will be used to strengthen the transparency of its methods and 
procedures. In particular, NASS agreed with our findings and, 
consistent with one of our two recommendations, said it will take steps 
to better document its specialized reports. NASS also said it plans to 
make a list of "common issues" raised by data users available on its 
Web site, which is in line with our second recommendation to improve 
the transparency of its procedures for handling data correction 
requests not filed under the IQA. NASS's commitment to continually 
improve its products is commendable, and its efforts to improve the 
transparency of its processes and products would be further enhanced 
if, consistent with our recommendations, it (1) ensures that all of its 
census products fully address NASS's own guidelines for data 
documentation and (2) posts on its Web site its procedures for handling 
correction requests not filed under the IQA. 

Background: 

The IQA directed OMB to issue guidelines to federal agencies covered by 
the Paperwork Reduction Act designed to ensure the "quality, 
objectivity, utility, and integrity" of information disseminated to the 
public.[Footnote 4] The IQA also directed OMB to include in its 
guidelines requirements for agencies to (1) develop their own 
information quality guidelines, (2) establish administrative mechanisms 
for affected persons to seek correction of information that does not 
comply with OMB's guidelines, and (3) annually report to OMB the number 
and nature of complaints they receive regarding the accuracy of the 
information they disseminate.[Footnote 5]

Prior to the IQA, there were several governmentwide actions aimed at 
improving agency data. For example, Statistical Policy Directive No. 2, 
first issued in 1952, required statistical agencies to inform users of 
conceptual or other limitations of the data, including how the data 
compare with similar statistics. In 1996, the Federal Committee on 
Statistical Methodology--an OMB-sponsored interagency committee 
dedicated to improving the quality of federal statistics--established a 
subcommittee to review the measurement and reporting of data quality in 
federal data collection programs. The results of the subcommittee's 
work were published in a 2001 report that addressed such issues as what 
information on sources of error federal data collection programs should 
provide, and how they should provide it.[Footnote 6] For all federal 
government information collections, the 1995 amendments to the 
Paperwork Reduction Act called on federal agencies to manage 
information resources with the goal of improving "the integrity, 
quality, and utility of information to all users within and outside the 
agency."[Footnote 7]

OMB's IQA guidelines were issued in final form in February 
2002.[Footnote 8] They required agencies subject to the IQA to take 
such steps as: 

* issue information quality guidelines designed to ensure the quality, 
objectivity, utility, and integrity of information disseminated to the 
public;

* establish administrative mechanisms for affected persons to seek 
correction of information they believe is not in compliance with the 
guidelines;

* report annually to the Director of OMB on the number and nature of 
complaints received regarding compliance with the guidelines and how 
the agencies handled those complaints; and: 

* designate an official responsible for ensuring compliance with OMB's 
guidelines. 

The OMB guidelines defined quality as an encompassing term comprising: 

* utility, which is the usefulness of the information to its intended 
users;

* integrity, which refers to the security of information and its 
protection from unauthorized access or revision; and: 

* objectivity, which addresses both presentation (i.e., whether the 
information is being presented in an accurate, clear, complete, and 
unbiased manner) and substance (i.e., whether the information is 
accurate, reliable, and unbiased). 

In addition, OMB addresses transparency within the definition of 
objectivity and utility. As recognized in OMB's guidelines, agencies 
that disseminate influential scientific, financial, or statistical 
information must demonstrate a high degree of transparency about data 
and methods. These measures are in place to facilitate the 
information's reproducibility by an outside party or reanalysis of an 
agency's results. 

The National Research Council of the National Academies considers 
transparency a key principle for federal statistical agencies, and 
stated in a recent report that transparency, which it defines as "an 
openness about the sources and limitations of the data," is 
particularly important for instilling credibility and trust among data 
users and providers.[Footnote 9]

As an agency within USDA, NASS is required to comply with the IQA. One 
statistical program administered by NASS is the quinquennial Census of 
Agriculture. According to NASS, the census provides a detailed picture 
of U.S. farms and ranches every 5 years and is the only source of 
uniform, comprehensive agricultural data at the county level. The 
results are published in 18 reports divided among three categories: 
Geographic Area Series, Census Quick Stats, and Specialty Products and 
Special Studies. Users of this information include federal agencies 
(for program and statistical purposes), farm organizations, businesses, 
universities, state departments of agriculture, elected 
representatives, legislative bodies at all levels of government, and 
academia. The next Census of Agriculture is scheduled for 2007. 

Objectives, Scope, and Methodology: 

Our objectives were to (1) review how NASS met OMB's guidelines 
covering the IQA and (2) examine the transparency of the documentation 
behind the Census of Agriculture's processes and products, including 
the recently completed work on the 2002 Census, and the efforts 
currently underway for the 2007 Census. 

To achieve both of these objectives, we reviewed OMB's and NASS's 
information quality guidelines, Census of Agriculture reports,[Footnote 
10] submissions to OMB, and other relevant documents. We also 
interviewed NASS officials about how NASS conducted the 2002 Census and 
how it is planning for the 2007 Census. The officials included the NASS 
Administrator, Associate Administrator, and Deputy Administrator for 
Programs and Products. 

In addition, to evaluate the transparency of Census of Agriculture 
products, we reviewed eight census reports and the Frequently Asked 
Questions area of the 2002 Census Web site, to determine the extent to 
which NASS followed its own procedures for ensuring the transparency of 
its information products. NASS's IQA guidelines define transparency as, 
"a clear description of the methods, data sources, assumptions, 
outcomes, and related information that allows a data user to understand 
how an information product was designed and produced."

NASS's guidelines state that its survey activities include such 
activities as sample design, questionnaire design, pre-testing, 
analysis of sampling, and imputation of missing data. However, the 
guidelines were not clear as to the specific activities to be 
documented. Consequently, we reviewed the practices employed by such 
statistical agencies as the National Academies of Sciences, 
International Monetary Fund, and U.S. Census Bureau, and developed a 
set of 20 practices associated with transparent documentation that 
encompassed the items NASS laid out in its own guidelines. The 
practices include such actions as defining data items, discussing 
sample design, and describing how the content of the survey differs 
from past iterations (see app. II). 

We looked for the presence or absence of these practices in 9 out of 
the 18 census reports and related forms of data that NASS disseminates, 
and verified the results with a second, independent analysis. In 
instances where a report did not include a particular documentation 
practice, we reviewed whether the report instead informed data users 
where to obtain this information. We chose these 9 reports because they 
all stem from the original census data collection, represent different 
product categories, and were available on the census Web site as of 
February 1, 2005. 

To obtain an external perspective of how NASS processes and products 
address the IQA guidelines, we interviewed six data users from 
different types of agricultural and research organizations. We selected 
these data users from lists of registrants for USDA and NASS outreach 
meetings within the past 5 years. We selected these six data users 
because they use information from the census on a regular basis. 
Moreover, these data users attended the most recent NASS outreach 
meeting, which specifically addressed the 2002 and 2007 Censuses. Some 
data users had also provided NASS with feedback on the content of the 
agricultural census. Their views cannot be projected to the larger 
population of census data users. 

We requested comments on a draft of this report from the Secretary of 
Agriculture. On September 8, 2005, we received the NASS Administrator's 
written comments and have reprinted them in appendix I. They are 
addressed in the Agency Comments and Our Evaluation section of this 
report. 

NASS Met the Procedural and Reporting Requirements of OMB's IQA 
Guidelines: 

NASS fulfilled the various procedural responsibilities and reporting 
requirements under OMB's guidelines. For example, NASS released its own 
IQA guidelines for public comment on March 27, 2002. NASS officials 
stated they received no substantive comments on them and OMB approved 
the guidelines with only minimal changes. The officials also noted that 
no revisions have been made since then. Table 1 shows in greater detail 
how NASS addressed OMB's guidelines. 

Table 1: NASS Addressed OMB's Agencywide Guidelines for Implementing 
IQA: 

OMB directed agencies to: 
* Prepare a draft report explaining how their guidelines will ensure 
and maximize the quality of information; 
* Publish a notice in the Federal Register announcing the availability 
of this report on the agency's Web site for public comment; 
NASS's response: 
* NASS posted its draft report and guidelines to its Web site from 
March 27, 2002, until September 30, 2002; 
* Notice of availability was published in the June 4, 2002, Federal 
Register[A] and the report itself was available on NASS's Web site. 

OMB directed agencies to: 
* Post their final report and guidelines to their Web sites; 
NASS's response: 
* Final guidelines are on the NASS Web site. 

OMB directed agencies to: 
* Develop administrative mechanisms allowing affected persons to 
correct disseminated information that does not comply with the OMB 
guidelines; 
NASS's response: 
* NASS outlines its correction procedures in detail on its Web site. 

OMB directed agencies to: 
* Submit a report by January 1 of each year on the number and nature of 
complaints received in the prior fiscal year; 
NASS's response: 
* USDA submitted a report to OMB both years since the guidelines have 
been in effect; NASS reported no complaints. 

OMB directed agencies to: 
* Designate an official to be responsible for the agency's compliance 
with OMB's guidelines; 
NASS's response: 
* USDA has designated its Chief Information Officer (CIO) as this 
official. The CIO in turn delegates compliance questions to lower-level 
offices, including the Standards Officer within the office of the 
Associate Administrator at NASS. 

Source: GAO analysis of OMB and NASS documents. 

[A] 67 Fed. Reg. 38,467. 

[End of table]

Better Documentation Could Improve the Transparency of Data Products 
and Correction Procedures: 

NASS's IQA guidelines define transparency as, "a clear description of 
the methods, data sources, assumptions, outcomes, and related 
information that allows a data user to understand how an information 
product was designed and produced." NASS's guidelines also note that 
"NASS will make the methods used to produce information as transparent 
as possible" and that its "internal guidelines call for clear 
documentation of data and methods used in producing estimates and 
forecasts. . . ."

To assess the extent to which NASS processes help ensure the 
transparency of the information it publishes, we examined key 
publications from the 2002 Census of Agriculture. Census reports vary 
in terms of scope and intended audience (see table 2). On the one hand, 
the United States Summary and State Data report contains over 100 data 
tables, an introduction, and four appendices. On the other hand, County 
Profile reports summarize each county's agricultural situation on two 
pages. 

Overall, we assessed eight census reports within three product 
categories, as well as the Frequently Asked Questions (FAQ) section of 
the 2002 Census Web site, to determine the extent to which NASS 
followed its own guidelines for ensuring the transparency of its 
products. As shown in table 2, the transparency of the data 
documentation in the reports we reviewed varied between the Geographic 
Area Series reports--which are the most comprehensive of NASS's 
products and addressed 15 of the 20 data documentation practices--and 
the Specialty Products and Special Studies which, depending on the 
specific product, addressed no more than 1 of the practices. 

Table 2: Census Reports Need More Robust Documentation: 

Product category: Geographic Area Series: 

Product title: United States Summary and State Data; 
General description: Contains over 100 national and state data tables; 
Portion of 20 documentation practices addressed: 15. 

Product title: State and County Data; 
General description: Contains over 100 state and county data tables; 
Portion of 20 documentation practices addressed: 15. 

Product category: Census Quick Stats: Ag Statistics Data Base: 

Product title: 2002 Census of Agriculture Downloadable Application; 
General description: A database for users to download and generate data 
tables at the national, state, and county levels; 
Portion of 20 documentation practices addressed: 14. 

Product category: Specialty Products and Special Studies: 

Product title: State and County Profiles; 
General description: Two-page reports containing summary data about a 
state or county; 
Portion of 20 documentation practices addressed: 0. 

Product title: Quick Facts from the 2002 Census of Agriculture; 
General description: This report presents national data in 16 charts or 
graphs; 
Portion of 20 documentation practices addressed: 1. 

Product title: Ranking of Market Value of Agricultural Products Sold; 
General description: This report contains state tables that rank the 
agricultural products sold by market value. The report also includes 
table definitions; 
Portion of 20 documentation practices addressed: 1. 

Product title: Congressional District Profiles; 
General description: Each profile is a two-page report that contains 
summary data about one congressional district; 
Portion of 20 documentation practices addressed: 0. 

Product title: Ranking of Congressional Districts; 
General description: Contains tables for 46 data items, such as number 
of farms, and ranks the congressional districts for each of these data 
items; 
Portion of 20 documentation practices addressed: 1. 

Product category: Additional Information: 

Product title: Frequently Asked Questions; 
General description: This section of the census Web site contains 
questions and answers grouped into four categories; 
Portion of 20 documentation practices addressed: 6. 

Source: GAO analysis of NASS reports. 

[End of table]

All eight reports and the FAQ Web site lacked a discussion of four 
documentation practices, including the following: 

1. Questionnaire testing. NASS produced a separate, internal report 
that discusses questionnaire testing in detail; however, publicly 
available census publications do not address this topic. 

2. Limitations of the data. NASS does not discuss data limitations in 
the census reports we reviewed. 

3. Impact of imputations, by item. When a statistical agency receives a 
report form with missing values, it normally estimates or "imputes" 
those values based on comparable data sources such as a similar farm 
operation. Although NASS uses a complex editing and imputation process 
to estimate missing values, and describes this process in the United 
States Summary and State Data report appendix, it does not quantify the 
impact of imputations by item in reports. 

4. Whether any of the collected data have been suppressed for data 
quality reasons. Without information on whether any of the data had 
been suppressed because the quality was lacking, data users must assume 
that reports include all data items collected in the census had met 
agency publication standards. 

Although NASS appropriately recognizes the variation in data user needs 
by publishing several types of specialized reports, none of the reports 
we reviewed direct data users where to find either a complete set of 
documentation or additional documentation. For example, given the short 
length and summary format of the County Profile reports, it is not 
surprising that they lack documentation. However, in order for users to 
assess the quality of the data contained in the reports, it is 
important for NASS to at least provide links on its Web site or to 
other publications where users can access definitions, response rates, 
and other relevant information. 

NASS Should Document Its Procedures for Handling Correction Requests 
Not Filed under the IQA: 

NASS has two methods for handling data correction requests, depending 
on how they are submitted: a formal approach prescribed by OMB for 
correction requests filed under IQA, and an informal approach that NASS 
uses to address correction requests that are not filed under IQA. 
NASS's informal correction procedures lack transparency because they 
are not documented and individual cases are not tracked. As a result, 
we could not determine the nature of these correction requests or 
whether or how they were addressed. 

Consistent with OMB's guidelines, NASS detailed its procedures to 
request corrections under IQA on its Web site, and posted appropriate 
Federal Register notices. For example, NASS's Web site explains that to 
seek a correction under IQA, petitioners must, among other steps: (1) 
state that their request for correction is being submitted under IQA, 
(2) clearly identify the information they believe to be in error, and 
(3) describe which aspects of NASS's IQA guidelines were not followed 
or were insufficient.[Footnote 11]

According to the instructions posted on its Web site, NASS's IQA 
procedures are triggered only when petitioners explicitly state they 
are submitting a correction request under IQA. To date, none have done 
so. NASS addresses all other correction requests using informal, 
undocumented procedures that were in place before IQA was enacted. NASS 
officials explained that such requests are forwarded to the agency 
official responsible for preparing the report containing the 
information in question. That official, in turn, determines if the 
request can be resolved by clarifying the data, or whether a correction 
is needed. If a data item needs to be corrected, NASS has a set of 
procedures for documenting errors and issuing errata reports that are 
detailed in its Policy and Standards Memorandum No. 38. The memorandum 
describes the circumstances under which errata reports will be printed, 
and provides a mechanism for NASS staff to describe the nature of the 
error, its cause, and the action taken to resolve it. 

According to the Administrator, Associate Administrator, and other 
senior NASS officials we interviewed, the requests it has handled from 
the 2002 Census have so far been resolved to the petitioners' 
satisfaction, and none resulted in any corrections to the data from the 
2002 Census. However, because NASS does not document its informal 
procedures for handling inquiries and data correction requests, and 
lacks a recordkeeping system to log and track them, NASS could not 
provide us with firm information on the number of inquiries it has 
handled, the nature of those inquiries, and whether and how they were 
addressed. 

This is not to say that all complaints should follow the same 
procedures required by the IQA mechanism. For efficiency's sake, it is 
important for agencies to respond to complaints in accordance with the 
magnitude of the problem. However, to provide a more complete picture 
of the questions NASS receives about its data and how those questions 
were handled, it will be important for NASS to better document its 
approach for handling correction requests not filed under IQA, and 
track their disposition. 

NASS Has Taken Steps to Better Document Its Criteria for Assessing 
Input on Census Content: 

The 2002 Census of Agriculture was the first in which NASS developed 
the questionnaire (the 1997 Census of Agriculture was moved from the 
Census Bureau to NASS after the content had been determined). In doing 
so, NASS went to great lengths to obtain input from data users on what 
questions to ask, and evaluated their suggestions using a documented 
set of criteria. In preparing for the 2007 Census, NASS sought feedback 
on the questionnaire content from a broader spectrum of data users, in 
part because NASS solicited suggestions via the Internet. However, 
unlike the 2002 cycle, the criteria NASS used to assess the feedback 
were not initially documented, which is contrary to NASS's IQA 
guidelines. However, as a result of our review, NASS has developed 
documented criteria similar to that used during the previous census. 

Under the Paperwork Reduction Act, agencies must obtain OMB's approval 
prior to collecting information from the public. As part of this 
process, agencies must certify to OMB that, among other things, the 
effort is necessary for the proper performance of agency functions, 
avoids unnecessary duplication, and reduces burden on small entities. 
Agencies must also provide an estimate of the burden the information 
collection would place on respondents.[Footnote 12]

For the 2002 Census, NASS submitted its request for approval--a form 
called "OMB 83-I"--in August 2001, and OMB approved it in October 2001. 
NASS estimated that the census would require a cumulative total of more 
than 1.3 million hours for respondents to complete and would cost them, 
in terms of their time, in excess of $21 million. 

OMB's approval process also requires agencies to solicit input from 
external sources. NASS obtained input on the 2002 Agricultural Census 
content through a Federal Register notice, meetings with data users, 
and by contacting federal and state agencies that use census statistics 
to discuss data needs. 

Likewise, NASS is obtaining input on the content of the 2007 Census 
through a variety of channels. According to an agency official, the 
process began around June 2004, when NASS began releasing publications 
from the 2002 Census. NASS sent an evaluation form to its state offices 
requesting feedback on the census, including their suggestions for 
changing the content. NASS also asked the state offices to identify 
users from whom it could obtain additional feedback. 

NASS solicited further input by reaching out to data users within USDA 
and other federal agencies, querying organizations included in a list 
of "typical" data users maintained by NASS's Marketing and Information 
Services Office, and holding periodic regional meetings with data 
users. NASS also has a "hot button" on its Web site where visitors are 
asked what items, if any, should be added or deleted from the 
census.[Footnote 13]

In all, NASS obtained input on the 2007 Census through 10 distinct 
conduits. Moreover, compared to the process used to develop the content 
of the 2002 Census, its 2007 efforts were open to a wider spectrum of 
customers, and involved more direct contact with data users during the 
planning phase. Indeed, as shown in table 3, NASS's outreach via the 
Internet, regional meetings, and queries to data users was over and 
above the steps it took when developing the 2002 Census. This openness 
was reflected in the comments of the six data users we interviewed. 
Five of the six users said NASS's approach to eliciting input was 
adequate, while three of the six had requested new content items for 
the 2007 Census to better meet the needs of their organizations. 

The content evaluation process began in December 2004, and NASS is 
currently testing the questionnaire content. Following any refinements, 
mail-out of the actual census is scheduled for December 2007. 

Table 3: NASS Is Using More Extensive Outreach to Develop the 2007 
Census Compared to 2002: 

Method of outreach: Posted Federal Register notices; 
2002 Census: Yes; 
2007 Census: Yes. 

Method of outreach: Solicited input from state agricultural statistical 
offices; 
2002 Census: Yes; 
2007 Census: Yes. 

Method of outreach: Solicited input from state governors; 
2002 Census: Yes; 
2007 Census: No. 

Method of outreach: Solicited input from Advisory Committee on 
Agricultural Statistics; 
2002 Census: Yes; 
2007 Census: Yes. 

Method of outreach: Solicited input from land grant universities; 
2002 Census: Yes; 
2007 Census: Yes. 

Method of outreach: Solicited input from federal data users; 
2002 Census: Yes; 
2007 Census: Yes. 

Method of outreach: Held federal data user working group meetings; 
2002 Census: Yes; 
2007 Census: Yes. 

Method of outreach: Held USDA-wide national data user outreach meeting; 
2002 Census: Yes; 
2007 Census: Yes. 

Method of outreach: Solicited input from a list of "typical" census 
users maintained by NASS's Marketing and Information Services Office; 
2002 Census: No; 
2007 Census: Yes. 

Method of outreach: Solicited input via Web site feedback form; 
2002 Census: No; 
2007 Census: Yes. 

Method of outreach: Held NASS-specific, regional data user meeting; 
2002 Census: No; 
2007 Census: Yes. 

Source: GAO analysis of NASS data. 

[End of table]

For both the 2002 and 2007 Census cycles, the solicitation, review, and 
ultimate determination of the questionnaire content was led by the 
Census Content Team, a group consisting of experienced NASS 
statisticians representing different segments of the agency such as 
livestock, crops, and marketing. The 2002 Content Team used specific, 
documented criteria to inform its decisions. Specifically, suggestions 
were assessed according to the following factors, which were also made 
available to data users: 

* items directly mandated by Congress or items that had strong 
congressional support;

* items proposed by other federal agencies where legislation called for 
that agency to provide data for Congress;

* items needed for evaluation of existing federal programs;

* items which, if omitted, would result in additional respondent burden 
and cost for a new survey for other agencies or users;

* items required for classification of farms by historical groupings;

* items needed for improving coverage in the census; and: 

* items that would provide data on current agricultural issues. 

However, the criteria the 2007 Team used to assess input on the 
questionnaire content were not initially documented. According to 
agency officials we interviewed, NASS largely relied on professional 
judgment to evaluate the feedback it received, considering such factors 
as the need to keep the data comparable to past censuses and not 
increase the length of the questionnaire. 

Although a certain amount of professional judgment will invariably be 
used in making determinations on questionnaire content, the absence of 
documented assessment criteria is inconsistent with NASS's guidelines. 
Indeed, these guidelines note that transparent documentation "allows a 
data user to understand how an information product was designed and 
produced." Moreover, without documented criteria, it is not clear 
whether members of the Content Team are considering the same set of 
factors, or even if they are weighing those factors in the same manner. 

According to NASS, the shift in approach stemmed from staff turnover 
and reassignments of members of the 2002 Team and, as a result, the 
2007 Team was not aware of the criteria used in 2002. Our review made 
the 2007 Team aware of the earlier set of criteria, and the Team has 
since developed similar documentation. NASS noted that all future 
content teams will use and update these criteria when developing the 
content of subsequent censuses. 

It will be important for NASS to continue with this approach because it 
is more consistent with its own IQA guidelines, and will also help NASS 
to do the following: 

Ensure the utility and relevance of information. A key principle for 
federal statistical agencies is to provide information relevant to 
issues of public policy.[Footnote 14] However, the nation's information 
needs are constantly evolving, and it is important for statistical 
agencies to adapt accordingly. This is particularly true with 
agriculture, where a variety of factors such as changing technology and 
agricultural trends can affect what information should be collected. 
Rigorous content selection criteria could help NASS methodically 
evaluate the needs of different users, establish priorities, and keep 
the census synchronized with changing public policy requirements. 

Maximize cost-effectiveness and reduce public burden. As with all 
federal surveys, there are financial and nonfinancial costs to 
conducting the Census of Agriculture. These costs include the direct 
expenditures related to planning, implementing, and analyzing the 
census, as well as disseminating the information. There is also a cost 
to respondents in terms of the time they take to complete the 
questionnaire. Additionally, there are opportunity costs in that for 
every question that is included in the census, another question might 
need to be excluded so as not to increase the length of the census. 
Rigorous, consistently applied criteria can help promote cost- 
effectiveness because they can ensure that only those questions that 
meet a particular, previously identified need are included in the 
census. Applying such criteria also help inform decisions on the 
appropriate role of the federal government in collecting the data, and 
whether a particular question might be more appropriately addressed by 
a different survey, government organization, or the by the private 
sector. 

Maintain credibility. Content selection criteria provide a basis for 
consistent decision making on what to include in the census and what 
gets left off. This is especially important for maintaining NASS's 
credibility given the input it receives from various sources. Without 
documented criteria, NASS's actions could be perceived as arbitrary or 
disproportionately swayed by one particular interest or another; thus, 
NASS's decisions would be more defensible. 

Further, documented criteria will guard against the loss of 
institutional memory to the extent there is further turnover in Content 
Team membership. 

Conclusions: 

NASS satisfied the procedural responsibilities and reporting 
requirements under OMB's IQA guidelines. Moreover, to the extent that 
NASS continues to use the documented criteria it developed to inform 
future decisions on the content of the Census of Agriculture, it could 
help establish a closer alignment between the questions included in the 
census and evolving agricultural policy requirements, resulting in a 
more cost-effective data collection program. 

Building on these efforts, the transparency of census data products 
could be improved with more robust documentation. NASS's procedures for 
addressing correction requests not filed under IQA could be more 
transparent as well. More than just a paperwork issue, greater 
transparency will help enhance NASS's accountability to public data 
users and increase the credibility of census information. 

Recommendations for Executive Action: 

To help enhance the transparency of the Census of Agriculture's 
processes and products, we recommend that the Secretary of Agriculture 
direct NASS to take the following two steps: 

1. Ensure that census products fully address NASS's own guidelines for 
data documentation or at least contain links to such information. The 
list of 20 documentation practices that we developed, while not 
necessarily exhaustive, represents sound actions used by other 
statistical agencies and could form a starting point for NASS. 

2. Document and post on NASS's Web site its procedures for handling 
data correction requests not filed under IQA, and track the disposition 
of those requests. 

Agency Comments and Our Evaluation: 

The NASS Administrator provided written comments on a draft of this 
report on September 8, 2005, which are reprinted in appendix I. NASS 
noted that our "report and recommendations are insightful and will be 
used to further strengthen the transparency of NASS methods and 
procedures."

In particular, NASS concurred with our finding that the methods and 
procedures in its specialized reports should be better documented and, 
consistent with our recommendation, stated that these products "will 
now provide links to this information." NASS's efforts, if fully 
implemented, should make it easier for data users to understand how 
these products were designed and produced, and NASS should be commended 
for its actions to continually improve its products and better meet the 
needs of its customers. 

While NASS's more comprehensive products were better documented, our 
analysis found that they could also benefit from more robust 
documentation. Thus, in keeping with our recommendation, it will be 
important for NASS to ensure that all of its census products--its 
larger reports and more focused studies--fully address NASS's own 
guidelines for data documentation. 

In commenting on our recommendation for NASS to document and post on 
its Web site its procedures for handling data correction requests not 
filed under IQA, NASS concurred with our view that this information 
would provide it with a better sense of the questions it receives about 
its data, but added that "a detailed recordkeeping system to log and 
track every inquiry" would not be the best use of its resources. 
Instead, NASS plans to "compile a listing of the more common issues" 
and make them available on its Web site in the form of frequently asked 
questions. NASS believes this approach would be useful for future 
planning, as well as provide answers to questions most likely to arise 
among other data users. 

As noted in our report, our recommendation stemmed from our finding 
that NASS could not provide us with information on the number of 
inquiries not filed under IQA, the characteristics of those inquiries, 
and how they were addressed. Although the details remain to be seen, 
NASS's proposed approach could provide this information and, consistent 
with the intended outcome our recommendation, address the need for 
greater transparency. NASS's efforts will be further strengthened if, 
consistent with our recommendation, it posts on its Web site its 
procedures for handling correction requests not filed under IQA. 

We will send copies of this report to other interested congressional 
parties, the Secretary of Agriculture, and the NASS Administrator. 
Copies will be made available to others on request. This report will 
also be available at no charge on GAO's Web site at [Hyperlink, 
http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact me at (202) 512-6806 or [Hyperlink, williamso@gao.gov]. Contact 
points for our Offices of Congressional Relations and Public Affairs 
may be found on the last page of this report. GAO staff who made major 
contributions to this report are listed in appendix III. 

Signed by: 

Orice M. Williams: 
Director: 
Financial Markets and Community Investments: 

[End of section]

Appendixes: 

Appendix I: Comments from the Department of Agriculture: 

United States Department of Agriculture:
National Agricultural Statistics Service: 

1400 Independence Avenue, SW 
Washington, DC 20250-2000: 

September 8, 2005: 

Ms. Orice M. Williams: 
Director, Financial Markets and Community Investments:
U.S. General Accountability Office: 
Washington, D.C. 20548: 

Dear Ms. Williams: 

This is in response to the draft General Accountability Office (GAO) 
report on Information Quality Act: National Agricultural Statistics 
Service Implements First Steps but Documentation of Census of 
Agriculture Could be Improved. NASS appreciates the detail and 
professionalism exhibited by the General Accountability Office 
auditors. The draft report and recommendations are insightful and will 
be used to further strengthen the transparency of NASS methods and 
procedures. 

For clarity, the NASS comments have been organized under each of the 
two recommendations to the Secretary of Agriculture. 

1. Ensure that census products fully address NASS's own guidelines for 
data documentation, or at least contain links to such information. The 
list of 20 documentation practices that we developed, while not 
necessarily exhaustive, represents sound actions used by other 
statistical agencies and could form a starting point for NASS. 

The draft GAO report correctly depicts the NASS guidelines on data 
documentation. The NASS goal is to provide a clear description of the 
methods and procedures used so a data user will understand how an 
information product was designed and produced. 

NASS concurs that it has done a much more thorough job of documenting 
methods and procedures in its comprehensive products, such as the 
"United States Summary: Geographic Area Series." NASS also agrees that 
specialized reports, though generally short in length, should inform 
data users where to obtain similar procedural information. 

NASS will continue to provide thorough documentation of its methods and 
procedures in its comprehensive products and will now provide links to 
this information in the shorter, specialized reports. 

2. Document and post on NASS's Web site its procedures for handling 
data correction requests not filed under IQA, and track the disposition 
of those requests. 

NASS strives to handle all data correction requests in an unbiased and 
thorough manner. These requests usually result from an individual's 
professional opinion differing from the official estimate. The official 
estimate is the product of predefined, consistent statistical 
procedures. As stated in the draft report, none of the data correction 
requests received to date for the 2002 Census of Agriculture resulted 
in corrections to the data. 

NASS concurs with the draft report observation that it might be useful 
to obtain "a more complete picture of the questions NASS receives about 
its data." However, NASS does not believe it would be the best use of 
resources to attempt to maintain a detailed recordkeeping system to log 
and track every inquiry. An alternative approach, which NASS will 
implement, is to compile a listing of the more common issues raised 
after census publication and disseminate, via the NASS Web site, a post-
census summary of frequently asked questions (FAQ) that seek data 
clarification. This will allow NASS to develop a better picture of the 
questions received for future planning as well as provides answers to 
those questions most likely to arise among other data users. 

We appreciate the opportunity to review and provide comments on the 
draft report. 

Sincerely,

Signed by: 

R. Ronald Bosecker: 
Administrator: 

[End of section]

Appendix II: How Census of Agriculture Reports Address Various 
Documentation Elements: 

Documentation practices: Discussion of how NASS developed the content 
of the census; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database; 
Frequently Asked Questions. 

Documentation practices: Discussion of 2002 Census content consistency 
with 1997; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database; 
Frequently Asked Questions. 

Documentation practices: Description of how the content differs; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database; 
Frequently Asked Questions. 

Documentation practices: Discussion of why NASS made content changes; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database. 

Documentation practices: Definition of the population; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database; 
Quick Facts; 
Frequently Asked Questions. 

Documentation practices: Definition of data items; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database; 
Quick Facts; 
Ranking of Congress. Districts. 

Documentation practices: Discussion of sample design; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database.  

Documentation practices: Discussion of questionnaire design; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database. 

Documentation practices: Discussion of questionnaire testing. 

Documentation practices: Copy of the questionnaire; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama). 

Documentation practices: Discussion of data collection procedures; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database. 

Documentation practices: Discussion of nonresponse followup; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database. 

Documentation practices: Discussion of data entry procedures; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database. 

Documentation practices: Discussion of data editing procedures; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database. 

Documentation practices: Quantify the impact of imputations, by item. 

Documentation practices: Discussion of response rates; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database; 
Frequently Asked Questions. 

Documentation practices: Discussion of nonsampling error; 
Product titles: 
United States Summary and State Report; 
State and County Reports (Alabama); 
Quick Stats: Ag Statistics Database. 

Documentation practices: Discussion of the limitations of the data. 

Documentation practices: Discussion of whether any of the collected 
data have been suppressed for data quality reasons. 

Documentation practices: Comparison of census results to other survey 
results for consistency; 
Product titles: 
Frequently Asked Questions. 

Source: GAO analysis of NASS data. 

Note: See the Objectives, Scope, and Methodology section of this report 
for a complete explanation of our analysis. 

[End of table]

[End of section]

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Orice M. Williams, (202) 512-6806: 

Acknowledgments: 

In addition to the contact named above, Robert Goldenkoff, Assistant 
Director; David Bobruff; Jennifer Cook; Richard Donaldson; Andrea 
Levine; Robert Parker; John Smale; and Michael Volpe made key 
contributions to this report. 

(450344): 

FOOTNOTES

[1] Consolidated Appropriations - Fiscal Year 2001, Pub. L. No. 106- 
544, § 515, 114 Stat. 2763A-153 to 2763A-154 (2000) (44 U.S.C. § 3516 
note). 

[2] Under the Paperwork Reduction Act, agencies must estimate the 
burdens their data collections impose on the public. 

[3] NASS officials reported that they have not received any complaints 
under the IQA. 

[4] Agencies covered by the IQA include all agencies subject to the 
Paperwork Reduction Act (PRA) - cabinet departments, independent 
regulatory agencies (e.g., Federal Communications Commission), and 
other independent agencies (e.g., the Environmental Protection Agency). 
44 U.S.C. § 3502(1). 

[5] Discussion of the IQA often centers on its impact on agencies' 
regulatory activities. Supporters of IQA, many of whom represent 
businesses and other regulated entities, maintain that IQA could 
enhance the quality of agency science and improve the rule-making 
process. Critics of IQA, including some environmental and public 
interest groups, view the law as a device to curtail health, safety, 
and other regulations. 

[6] OMB, Measuring and Reporting Sources of Error in Surveys, 
Statistical Policy Working Paper 31, July 2001. 

[7] 44 U.S.C. § 3506(b)(1)(C). 

[8] 67 Fed. Reg. 8452 (Feb. 22, 2002). 

[9] Margaret E. Martin, Miron L. Straf, and Constance F. Citro eds., 
Principles and Practices for a Federal Statistical Agency, (3rd ed.) 
(Washington, D.C.: National Academies Press, 2005), pp. 8, 29. 

[10] Reports can be obtained from USDA's Web site; see www.usda.gov/
nass. 

[11] See www.usda.gov/nassinfo/infocorrection.htm. 

[12] 44 U.S.C. § 3506(c). 

[13] See http://www.nass.usda.gov/census/census02/feedback.htm. 

[14] Margaret E. Martin, Miron L. Straf, and Constance F. Citro eds., 
Principles and Practices for a Federal Statistical Agency, (3rd ed.) 
(Washington, D.C.: National Academies Press, 2005), p. 4. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director,

NelliganJ@gao.gov

(202) 512-4800

U.S. Government Accountability Office,

441 G Street NW, Room 7149

Washington, D.C. 20548: