This is the accessible text file for GAO report number GAO-09-413T 
entitled 'Information Technology: Census Bureau Needs to Strengthen 
Testing of 2010 Decennial Systems' which was released on March 6, 2009.

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Testimony: 

Before the Subcommittee on Federal Financial Management, Government 
Information, Federal Services, and International Security, Committee on 
Homeland Security and Governmental Affairs, U.S. Senate: 

United States Government Accountability Office: 
GAO: 

For Release on Delivery:
Expected at 2:30 p.m. EST: 
Thursday, March 5, 2009: 

Information Technology: 

Census Bureau Needs to Strengthen Testing of 2010 Decennial Systems: 

Statement of David A. Powner:
Director, Information Technology Management Issues: 

Robert Goldenkoff:
Director, Strategic Issues: 

GAO-09-413T: 

GAO Highlights: 

Highlights of GAO-09-413T, a testimony before the Subcommittee on 
Federal Financial Management, Government Information, Federal Services, 
and International Security, Committee on Homeland Security and 
Governmental Affairs, U.S. Senate. 

Why GAO Did This Study: 

The Decennial Census is mandated by the U.S. Constitution and provides 
vital data that are used, among other things, to reapportion and 
redistrict congressional seats and allocate federal financial 
assistance. In March 2008, GAO designated the 2010 Decennial Census a 
high-risk area, citing a number of long-standing and emerging 
challenges, including weaknesses in the U.S. Census Bureau’s (Bureau) 
management of its information technology (IT) systems and operations. 
In conducting the 2010 census, the Bureau is relying on both the 
acquisition of new IT systems and the enhancement of existing systems. 
Thoroughly testing these systems before their actual use is critical to 
the success of the census. GAO was asked to testify on its report, 
being released today, on the status and plans of testing of key 2010 
decennial IT systems. 

What GAO Found: 

Although the Bureau has made progress in testing key decennial systems, 
critical testing activities remain to be performed before systems will 
be ready to support the 2010 census. Bureau program offices have 
completed some testing of individual systems, but significant work 
still remains to be done, and many plans have not yet been developed 
(see table below). In its testing of system integration, the Bureau has 
not completed critical activities; it also lacks a master list of 
interfaces between systems and has not developed testing plans and 
schedules. Although the Bureau had originally planned what it refers to 
as a Dress Rehearsal, starting in 2006, to serve as a comprehensive end-
to-end test of key operations and systems, significant problems were 
identified during testing. As a result, several key operations were 
removed from the Dress Rehearsal and did not undergo end-to-end 
testing. The Bureau has neither developed testing plans for these key 
operations, nor has it determined when such plans will be completed. 

Weaknesses in the Bureau’s testing progress and plans can be attributed 
in part to a lack of sufficient executive-level oversight and guidance. 
Bureau management does provide oversight of system testing activities, 
but the oversight activities are not sufficient. For example, Bureau 
reports do not provide comprehensive status information on progress in 
testing key systems and interfaces, and assessments of the overall 
status of testing for key operations are not based on quantitative 
metrics. Further, although the Bureau has issued general testing 
guidance, it is neither mandatory nor specific enough to ensure 
consistency in conducting system testing. Without adequate oversight 
and more comprehensive guidance, the Bureau cannot ensure that it is 
thoroughly testing its systems and properly prioritizing testing 
activities before the 2010 Decennial Census, posing the risk that these 
systems may not perform as planned. 

Table: Status and Plans of 2010 System Testing: 

System: Headquarters processing; 
Testing status: In progress; 
Testing plan completed: Partial; 
Testing schedule completed: Partial. 

System: Master address and geographic information; 
Testing status: In progress; 
Testing plan completed: Partial; 
Testing schedule completed: Partial. 

System: Decennial response integration; 
Testing status: In progress; 
Testing plan completed: Partial; 
Testing schedule completed: Partial. 

System: Field data collection automation; 
Testing status: In progress; 
Testing plan completed: Partial; 
Testing schedule completed: Partial. 

System: Paper-based operations; 
Testing status: In progress; 
Testing plan completed: No; 
Testing schedule completed: Partial. 

System: Data access and dissemination; 
Testing status: In progress; 
Testing plan completed: Partial; 
Testing schedule completed: Partial. 

Source: GAO analysis of Bureau data. 

[End of table] 

What GAO Recommends: 

In its report, GAO recommended that the Secretary of Commerce direct 
the Bureau to complete key system testing activities, develop and 
maintain plans and schedules for integration testing, and improve the 
oversight of and guidance for systems testing. In comments on a draft 
of this report, the department agreed with GAO’s recommendations. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-09-413T]. For more 
information, contact David A. Powner at (202) 512-9286 or 
pownerd@gao.gov or Robert Goldenkoff at (202) 512-2757 or 
goldenkoffr@gao.gov. 

[End of section] 

Mr. Chairman and Members of the Subcommittee: 

Thank you for the opportunity to participate in today's hearing on the 
2010 census. The U.S. Census Bureau (Bureau) is relying on both the 
acquisition of new systems and the enhancement of existing legacy 
systems for conducting operations for the 2010 Decennial Census. As you 
know, the census is mandated by the U.S. Constitution and provides data 
that are vital to the nation. These data are used, for example, to 
reapportion and redistrict the seats of the U.S. House of 
Representatives, realign the boundaries of the legislative districts of 
each state, and allocate federal financial assistance. Carrying out the 
census is the responsibility of the Department of Commerce's Census 
Bureau, which is relying on automation and technology to improve the 
coverage, accuracy, and efficiency of the 2010 census. Because the 
accuracy of the 2010 census depends in part on the proper functioning 
of these systems, both individually and when integrated, thorough 
testing of these systems before their actual use is critical to the 
success of the census. 

As you know, in March 2008, we designated the 2010 Decennial Census as 
a high-risk area, citing a number of long-standing and emerging 
challenges,[Footnote 1] including weaknesses in the Bureau's management 
of its information technology (IT) systems and operations. The 2010 
Decennial Census remained as one of our high-risk areas in our recent 
high-risk update issued in January 2009.[Footnote 2] This statement 
summarizes the findings in our report, being released by the 
subcommittee today, on the status and plans of testing of key 2010 
decennial IT systems.[Footnote 3] 

Our work for this report was performed in accordance with generally 
accepted government auditing standards. Those standards require that we 
plan and perform the audit to obtain sufficient, appropriate evidence 
to provide a reasonable basis for our findings and conclusions based on 
our audit objective. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit 
objective. 

Background: 

The Bureau's mission is to provide comprehensive data about the 
nation's people and economy. The 2010 census enumerates the number and 
location of people on Census Day, which is April 1, 2010. However, 
census operations begin long before Census Day and continue afterward. 
For example, address canvassing for the 2010 census will begin in April 
2009, while the Secretary of Commerce must report tabulated census data 
to the President by December 31, 2010, and to state governors and 
legislatures by March 31, 2011. 

* The decennial census is a major undertaking for the Bureau that 
includes the following major activities: 

* Establishing where to count. This includes identifying and correcting 
addresses for all known living quarters in the United States (address 
canvassing) and validating addresses identified as potential group 
quarters, such as college residence halls and group homes (group 
quarters validation). 

* Collecting and integrating respondent information. This includes 
delivering questionnaires to housing units by mail and other methods, 
[Footnote 4] processing the returned questionnaires, and following up 
with nonrespondents through personal interviews (nonresponse follow-
up). It also includes enumerating residents of group quarters (group 
quarters enumeration) and occupied transitional living quarters 
(enumeration of transitory locations), such as recreational vehicle 
parks, campgrounds, and hotels. It also includes a final check of 
housing unit status (field verification) where Bureau workers verify 
potential duplicate housing units identified during response processing. 

* Providing census results. This includes tabulating and summarizing 
census data and disseminating the results to the public. 

Role of IT in the Decennial Census: 

Automation and IT are to play a critical role in the success of the 
2010 census by supporting data collection, analysis, and dissemination. 
Several systems will play a key role in the 2010 census. For example, 
enumeration "universes," which serve as the basis for enumeration 
operations and response data collection, are organized by the Universe 
Control and Management (UC&M) system, and response data are received 
and edited to help eliminate duplicate responses using the Response 
Processing System (RPS). Both UC&M and RPS are legacy systems that are 
collectively called the Headquarters Processing System. 

Geographic information and support to aid the Bureau in establishing 
where to count U.S. citizens are provided by the Master Address File/ 
Topologically Integrated Geographic Encoding and Referencing (MAF/ 
TIGER) system. The Decennial Response Integration System (DRIS) is to 
provide a system for collecting and integrating census responses from 
all sources, including forms and telephone interviews. The Field Data 
Collection Automation (FDCA) program includes the development of 
handheld computers for the address canvassing operation and the 
systems, equipment, and infrastructure that field staff will use to 
collect data. Paper-Based Operations (PBO) was established in August 
2008 primarily to handle certain operations that were originally part 
of FDCA. PBO includes IT systems and infrastructure needed to support 
the use of paper forms for operations such as group quarters 
enumeration activities, nonresponse follow-up activities, enumeration 
at transitory locations activities, and field verification activities. 
These activities were originally to be conducted using IT systems and 
infrastructure developed by the FDCA program. Finally, the Data Access 
and Dissemination System II (DADS II) is to replace legacy systems for 
tabulating and publicly disseminating data. 

Comprehensive Testing Improves Chances of a Successful Decennial Census: 

As stated in our testing guide and the Institute of Electrical and 
Electronics Engineers (IEEE) standards,[Footnote 5] complete and 
thorough testing is essential for providing reasonable assurance that 
new or modified IT systems will perform as intended. To be effective, 
testing should be planned and conducted in a structured and disciplined 
fashion that includes processes to control each incremental level of 
testing, including testing of individual systems, the integration of 
those systems, and testing to address all interrelated systems and 
functionality in an operational environment. 

Further, this testing should be planned and scheduled in a structured 
and disciplined fashion. Comprehensive testing that is effectively 
planned and scheduled can provide the basis for identifying key tasks 
and requirements and better ensure that a system meets these specified 
requirements and functions as intended in an operational environment. 

Dress Rehearsal Includes Testing of Certain Systems and Operations: 

In preparation for the 2010 census, the Bureau planned what it refers 
to as the Dress Rehearsal. The Dress Rehearsal includes systems and 
integration testing,[Footnote 6] as well as end-to-end testing of key 
operations in a census-like environment. During the Dress Rehearsal 
period, running from February 2006 through June 2009, the Bureau is 
developing and testing systems and operations, and it held a mock 
Census Day on May 1, 2008. The Dress Rehearsal activities, which are 
still under way, are a subset of the activities planned for the actual 
2010 census and include testing of both IT and non-IT related 
functions, such as opening offices and hiring staff. 

The Dress Rehearsal identified significant technical problems during 
the address canvassing and group quarters validation operations. For 
example, during the Dress Rehearsal address canvassing operation, the 
Bureau encountered problems with the handheld computers, including slow 
and inconsistent data transmissions, the devices freezing up, and 
difficulties collecting mapping coordinates. As a result of the 
problems observed during the Dress Rehearsal, cost overruns and 
schedule slippage in the FDCA program, and other issues, the Bureau 
removed the planned testing of several key operations from the Dress 
Rehearsal and switched key operations, such as nonresponse follow-up, 
to paper-based processes instead of using the handheld computers as 
originally planned. 

Bureau Is Making Progress in Key System Testing, but Lacks Plans and 
Schedules: 

Through the Dress Rehearsal and other testing activities, the Bureau 
has completed key system tests, but significant testing has yet to be 
done, and planning for this is not complete. Table 1 summarizes the 
status and plans for system testing. 

Table 1: Status of System Testing and Plans: 

System: Headquarters Processing--UC&M and RPS; 
Dress Rehearsal system testing: In progress; 
2010 system testing: Testing status: In progress; 
2010 system testing: Testing plan completed: Partial; 
2010 system testing: Testing schedule completed: Partial. 

System: MAF/TIGER; 
Dress Rehearsal system testing: Completed; 
2010 system testing: Testing status: In progress; 
2010 system testing: Testing plan completed: Partial; 
2010 system testing: Testing schedule completed: Partial. 

System: DRIS; 
Dress Rehearsal system testing: Completed; 
2010 system testing: Testing status: In progress; 
2010 system testing: Testing plan completed: Partial[A]; 
2010 system testing: Testing schedule completed: Partial[A]. 

System: FDCA; 
Dress Rehearsal system testing: Partially completed[B]; 
2010 system testing: Testing status: In progress; 
2010 system testing: Testing plan completed: Partial; 
2010 system testing: Testing schedule completed: Partial. 

System: PBO; 
Dress Rehearsal system testing: N/A[C]; 
2010 system testing: Testing status: In progress; 
2010 system testing: Testing plan completed: No; 
2010 system testing: Testing schedule completed: Partial. 

System: DADS; 
Dress Rehearsal system testing: DADS[D] in progress; 
2010 system testing: Testing status: DADS II in progress; 
2010 system testing: Testing plan completed: Partial; 
2010 system testing: Testing schedule completed: Partial. 

Source: GAO analysis of Bureau data. 

[A] Program officials stated that DRIS's test plan and schedule were 
completed but will be modified to reflect changes resulting from the 
switch to paper-based operations. 

[B] System testing related to operations removed from the Dress 
Rehearsal was not completed. These operations were later moved to PBO. 

[C] The office to support PBO was created in August 2008. 

[D] DADS system is being used for Dress Rehearsal system testing, but 
the replacement system, DADS II, is being developed and tested for 2010 
operations. 

[End of section] 

Bureau Has Conducted Limited Integration Testing, but Has Not Developed 
2010 Test Plans and Schedules for Integration Testing: 

Effective integration testing ensures that external interfaces work 
correctly and that the integrated systems meet specified requirements. 
This testing should be planned and scheduled in a disciplined fashion 
according to defined priorities. 

For the 2010 census, each program office is responsible for and has 
made progress in defining system interfaces and conducting integration 
testing, which includes testing of these interfaces. However, 
significant activities remain to be completed. For example, for systems 
such as PBO, interfaces have not been fully defined, and other 
interfaces have been defined but have not been tested. In addition, the 
Bureau has not established a master list of interfaces between key 
systems, or plans and schedules for integration testing of these 
interfaces. A master list of system interfaces is an important tool for 
ensuring that all interfaces are tested appropriately and that the 
priorities for testing are set correctly. As of October 2008, the 
Bureau had begun efforts to update a master list it had developed in 
2007, but it has not provided a date when this list will be completed. 

Without a completed master list, the Bureau cannot develop 
comprehensive plans and schedules for conducting systems integration 
testing that indicate how the testing of these interfaces will be 
prioritized. With the limited amount of time remaining before systems 
are needed for 2010 operations, the lack of comprehensive plans and 
schedules increases the risk that the Bureau may not be able to 
adequately test system interfaces, and that interfaced systems may not 
work together as intended. 

Bureau Has Conducted Limited End-to-End Testing as Part of the Dress 
Rehearsal, but Has Not Developed Testing Plans for Critical Operations: 

Although several critical operations underwent end-to-end testing in 
the Dress Rehearsal, others did not. As of December 2008, the Bureau 
had not established testing plans or schedules for end-to-end testing 
of the key operations that were removed from the Dress Rehearsal, nor 
has it determined when these plans will be completed. These operations 
include: 

* update/leave, 

* nonresponse follow-up, 

* enumeration of transitory locations, 

* group quarters enumeration, and: 

* field verification. 

The decreasing time available for completing end-to-end testing 
increases the risk that testing of key operations will not take place 
before the required deadline. Bureau officials have acknowledged this 
risk in briefings to the Office of Management and Budget. However, as 
of January 2009, the Bureau had not completed mitigation plans for this 
risk. According to the Bureau, the plans are still being reviewed by 
senior management. Without plans to mitigate the risks associated with 
limited end-to-end testing, the Bureau may not be able to respond 
effectively if systems do not perform as intended. 

Bureau Lacks Sufficient Executive-Level Oversight and Guidance for 
Testing: 

As stated in our testing guide and IEEE standards, oversight of testing 
activities includes both planning and ongoing monitoring of testing 
activities. Ongoing monitoring entails collecting and assessing status 
and progress reports to determine, for example, whether specific test 
activities are on schedule. In addition, comprehensive guidance should 
describe each level of testing and the types of test products expected. 

In response to prior recommendations, the Bureau took initial steps to 
enhance its programwide oversight; however, these steps have not been 
sufficient. For example, in June 2008, the Bureau established an 
inventory of all testing activities specific to all key decennial 
operations. However, the inventory has not been updated since May 2008, 
and officials have no plans for further updates. 

In another effort to improve executive-level oversight, the Decennial 
Management Division began producing (as of July 2008) a weekly 
executive alert report and has established (as of October 2008) a 
dashboard and monthly reporting indicators. However, these products do 
not provide comprehensive status information on the progress of testing 
key systems and interfaces. Further, the assessment of testing progress 
has not been based on quantitative and specific metrics. The lack of 
quantitative and specific metrics to track progress limits the Bureau's 
ability to accurately assess the status and progress of testing 
activities. In commenting on our draft report, the Bureau provided 
selected examples where they had begun to use more detailed metrics to 
track the progress of end-to-end testing activities. 

The Bureau also has weaknesses in its testing guidance. According to 
the Associate Director for the 2010 census, the Bureau did establish a 
policy strongly encouraging offices responsible for decennial systems 
to use best practices in software development and testing, as specified 
in level 2 of Carnegie Mellon's Capability Maturity Model® Integration. 
[Footnote 7] However, beyond this general guidance, there is no 
mandatory or specific guidance on key testing activities such as 
criteria for each level or the type of test products expected. The lack 
of guidance has led to an ad hoc--and, at times--less than desirable 
approach to testing. 

Implementation of Recommendations Could Help Ensure Key Testing 
Activities are Completed: 

In our report, we are making ten recommendations for improvements to 
the Bureau's testing activities. Our recommendations include finalizing 
system requirements and completing development of test plans and 
schedules, establishing a master list of system interfaces, 
prioritizing and developing plans to test these interfaces, and 
establishing plans to test operations removed from the Dress Rehearsal. 
In addition, we are recommending that the Bureau improve its monitoring 
of testing progress and improve executive-level oversight of testing 
activities. 

In written comments on the report, the department had no significant 
disagreements with our recommendations. The department stated that its 
focus is on testing new software and systems, not legacy systems and 
operations used in previous censuses. However, the systems in place to 
conduct these operations have changed substantially and have not yet 
been fully tested in a census-like environment. Consistent with our 
recommendations, finalizing test plans and schedules and testing all 
systems as thoroughly as possible will help to ensure that decennial 
systems will work as intended. 

In summary, while the Bureau's program offices have made progress in 
testing key decennial systems, much work remains to ensure that systems 
operate as intended for conducting an accurate and timely 2010 census. 
This work includes system, integration, and end-to-end testing 
activities. Given the rapidly approaching deadlines of the 2010 census, 
completing testing and establishing stronger executive-level oversight 
are critical to ensuring that systems perform as intended when they are 
needed. 

Mr. Chairman and members of the subcommittee, this concludes our 
statement. We would be pleased to respond to any questions that you or 
other members of the subcommittee may have at this time. 

Contacts and Staff Acknowledgements: 

If you have any questions about matters discussed in this testimony, 
please contact David A. Powner at (202) 512-9286 or pownerd@gao.gov or 
Robert Goldenkoff at (202) 512-2757 or goldenkoffr@gao.gov. Other key 
contributors to this testimony include Sher`rie Bacon, Barbara Collier, 
Neil Doherty, Vijay D'Souza, Elizabeth Fan, Nancy Glover, Signora May, 
Lee McCracken, Ty Mitchell, Lisa Pearson, Crystal Robinson, Melissa 
Schermerhorn, Cynthia Scott, Karl Seifert, Jonathan Ticehurst, Timothy 
Wexler, and Katherine Wulff. 

[End of section] 

Footnotes: 

[1] GAO, Information Technology: Significant Problems of Critical 
Automation Program Contribute to Risks Facing 2010 Census, [hyperlink, 
http://www.gao.gov/products/GAO-08-550T] (Washington, D.C.: Mar. 5, 2008). 

[2] GAO, High-Risk Series: An Update, [hyperlink, 
http://www.gao.gov/products/GAO-09-271] (Washington, D.C.: Jan. 22, 2009). 

[3] GAO, Information Technology: Census Bureau Testing of 2010 
Decennial Systems Can Be Strengthened, [hyperlink, 
http://www.gao.gov/products/GAO-09-262] (Washington, D.C.: Mar. 5, 2009). 

[4] For example, in the "update/leave" operation, after enumerators 
update addresses, they leave questionnaires at housing units; this 
occurs mainly in rural areas lacking street names, house numbers, or 
both. 

[5] GAO, Year 2000 Computing Crisis: A Testing Guide, [hyperlink, 
http://www.gao.gov/products/GAO/AIMD-10.1.21] (Washington, D.C.: Nov. 
1, 1998) and IEEE Std. 12207-2008, Systems and Software Engineering--
Software Lifecycle Processes (Piscataway, N.J.: 2008). 

[6] Individual program offices manage individual system testing for the 
Dress Rehearsal, and integration testing is managed by the pairs of 
program offices whose interfaces are being tested. 

[7] Capability Maturity Model® Integration is intended to provide 
guidance for improving an organization's processes and the ability to 
manage the development, acquisition, and maintenance of products and 
services. The model uses capability levels to assess process maturity. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: