This is the accessible text file for GAO report number GAO-11-381 
entitled 'Drinking Water: Unreliable State Data Limit EPA's Ability to 
Target Enforcement Priorities and Communicate Water Systems' 
Performance' which was released on July 19, 2011. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Requesters: 

June 2011: 

Drinking Water: 

Unreliable State Data Limit EPA's Ability to Target Enforcement 
Priorities and Communicate Water Systems' Performance: 

GAO-11-381: 

GAO Highlights: 

Highlights of GAO-11-381, a report to congressional requesters. 

Why GAO Did This Study: 

The nation’s drinking water is among the safest in the world, but 
contamination has occurred, causing illnesses and even deaths. Under 
the Safe Drinking Water Act (SDWA), the Environmental Protection 
Agency (EPA) has authorized most states, territories, and tribes to 
take primary responsibility for ensuring that community water systems 
provide safe water. EPA needs complete and accurate data on systems’ 
compliance with SDWA to conduct oversight. GAO was asked to assess the 
(1) quality of the state data EPA uses to measure compliance with 
health and monitoring requirements of the act and the status of 
enforcement efforts, (2) ways in which data quality could affect EPA’s 
management of the drinking water program, and (3) actions EPA and the 
states have been taking to improve data quality. GAO analyzed EPA 
audits of state data done in 2007, 2008, and 2009, and surveyed EPA 
and state officials to obtain their views on factors that have 
affected data quality and steps that could improve it. 

What GAO Found: 

The data states reported to EPA for measuring compliance with health 
and monitoring requirements of SDWA did not reliably reflect the 
number of health-based and monitoring violations that community water 
systems have committed or the status of enforcement actions. Using 
data from the 14 states EPA audited in 2009, GAO estimates that those 
14 states did not report or inaccurately reported 26 percent of the 
health-based violations that should have been reported and 84 percent 
of the monitoring violations that should have been reported. GAO’s 
findings were consistent with the results of prior EPA audits. In 
addition, according to EPA headquarters and regional officials GAO 
interviewed and surveyed, state-reported data underreported the 
percentage of water systems with violations against which the states 
have taken enforcement actions. Survey respondents and other officials 
reported that numerous factors contribute to errors in reported data 
on violations and enforcement, including inadequate training, 
staffing, and guidance, and inadequate funding to conduct those 
activities. 

Unreported health-based and monitoring violations and incomplete 
enforcement data limit EPA’s ability to identify water systems with 
the most serious compliance problems and ensure that it is achieving 
its goal of targeting for enforcement those systems with the most 
serious compliance problems. Specifically, incomplete and inaccurate 
data on both violations and enforcement actions affect a scoring tool 
EPA and the states are using to rank systems for enforcement actions. 
In addition, unreliable data quality impedes EPA’s ability to monitor 
and report progress toward a strategic objective of reducing exposure 
to contaminants in drinking water. For example, EPA’s 2011 national 
program guidance contains a performance measure for the number and 
percentage of systems with certain repeated health-based violations, 
but EPA’s ability to reliably use this type of measure requires 
complete and accurate data on violations. Because of unreported 
violations data, EPA may not be able to report accurate performance 
information on systems with these violations. 

EPA and the states have collaborated over many years to identify and 
address the causes of incomplete and inaccurate violations data, but 
those efforts have not been fully successful, according to EPA and 
state officials GAO surveyed. EPA’s efforts have included (1) 
conducting audits-—discontinued in 2010 because of funding 
constraints—-to determine the completeness and accuracy of the 
violations data states reported to EPA, (2) establishing three work 
groups to address data management and quality, and (3) urging EPA 
regions and states to use data management tools the agency has 
developed. However, EPA has encouraged but not required that its 
regions or the states take specific actions that could improve data 
quality. EPA’s 2010 drinking water strategy calls for, among other 
things, an increase in shared data between the agency and the states. 
EPA also plans to redesign its drinking water data system to provide 
it with greater access to, and oversight of, the states’ 
determinations of SDWA violations. 

What GAO Recommends: 

GAO is making recommendations to improve EPA’s ability to oversee the 
states’ implementation of SDWA and provide Congress and the public 
with more complete and accurate data on compliance and enforcement. 
EPA partially agreed with two of the recommendations, disagreed with 
one, and neither agreed nor disagreed with one. GAO believes that EPA 
needs to implement all of the recommendations to improve its ability 
to oversee SDWA. 

View [hyperlink, http://www.gao.gov/products/GAO-11-381] or key 
components. For more information, contact David Trimble at (202) 512-
3841 or trimbled@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

SDWIS Data from States Did Not Reliably Reflect Community Water 
Systems' Violations of SDWA or the Status of Enforcement Actions: 

Incomplete and Inaccurate SDWIS/Fed Data Hamper EPA's Ability to 
Manage the PWSS Program and Communicate Progress toward Its Strategic 
Objective: 

Actions EPA and States Are Taking to Improve the Quality of Data in 
SDWIS Have Not Been Fully Successful and More Actions Are Planned: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Survey Methodology and Analysis: 

Appendix III: Comments from the Environmental Protection Agency: 

Appendix IV: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Primacy Agencies Covered by EPA's Data Verification Audits, 
by Year: 

Table 2: Survey Respondents' Answers to Closed-Ended Questions: 

Figures: 

Figure 1: Number of Community Drinking Water Systems that Serve 
Various Populations: 

Figure 2: Estimated Percentage of Health-based Violations That States 
Audited in 2007 and 2009 Did Not Report or Inaccurately Reported: 

Figure 3: Estimated Percentage of Health-Based Violations That States 
Audited in 2002 through 2004 Did Not Report or Reported Inaccurately: 

Figure 4: Enforcement Status of Health-based and Monitoring Violations 
at Community Water Systems Reported by States to SDWIS/Fed, 2005-2009: 

Figure 5: Factors Contributing to Incorrect Compliance Determinations 
and Data Flow Errors, According to Survey Respondents: 

Figure 6: Percentage of Audited Water Systems with a Difference in 
Their Enforcement Targeting Tool Scores Derived from SDWIS/Fed Data 
and Audited Data, 2007-2009: 

Abbreviations: 

CDC: Centers for Disease Control and Prevention: 

eDV: electronic data verification: 

EPA: Environmental Protection Agency: 

GPRA: Government Performance and Results Act: 

MCL: maximum contaminant level: 

PWSS: Public Water System Supervision: 

SDWA: Safe Drinking Water Act: 

SDWIS/Fed: Safe Drinking Water Information System/Federal: 

SDWIS/State: Safe Drinking Water Information System/State: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

June 17, 2011: 

The Honorable Henry A. Waxman:
Ranking Member:
Committee on Energy and Commerce:
House of Representatives: 

The Honorable John D. Dingell:
The Honorable Edward J. Markey:
House of Representatives: 

Americans rely on more than 51,000 community water systems for safe 
drinking water.[Footnote 1] Even though this drinking water supply is 
generally considered among the safest in the world, 11 states had 20 
outbreaks of illness associated with drinking water in 2005 and 2006 
that resulted in 612 illnesses and 4 deaths, according to data 
published by the Centers for Disease Control and Prevention (CDC) in 
2008.[Footnote 2] In part to safeguard against such outbreaks, the 
Environmental Protection Agency (EPA), which is generally responsible 
for the regulation of the nation's drinking water, requires public 
water systems to comply with regulations it established under the Safe 
Drinking Water Act (SDWA). Among other things, these regulations 
establish (1) health-based requirements, including limitations, and 
treatment techniques for controlling contaminants that could harm 
human health and (2) monitoring requirements to determine whether 
drinking water meets the health-based requirements. 

EPA authorizes and assists state, territorial, and tribal regulatory 
agencies--referred to as states in this report--to administer SDWA 
through EPA's Public Water System Supervision (PWSS) program. States 
that have accepted "primacy" responsibility for the PWSS program 
collect and review data from community water systems to determine 
their compliance with SDWA; all states except Wyoming and the District 
of Columbia have received primacy.[Footnote 3] With the exception of 
the Navajo Nation, EPA maintains primacy for community water systems 
in Indian Country. Primacy states are responsible for, among other 
things, determining when systems have violated SDWA, taking timely and 
appropriate enforcement action, and reporting those actions to EPA. 
EPA's regions and headquarters oversee the states to ensure they meet 
their primacy responsibilities; the EPA regions also act as the 
primacy agency in nonprimacy states, where a state has not yet 
received primacy for a particular drinking water regulation, and on 
tribal lands where the tribe has not assumed primacy.[Footnote 4] To 
determine water systems' compliance with federal standards for safe 
drinking water, EPA must have access to reliable data on the inventory 
of community water systems, which, along with other public water 
systems are subject to these standards; the quality of drinking water; 
and violations of SDWA's requirements including those to monitor 
drinking water to ensure the water meets health standards. EPA also 
needs reliable data regarding the status of enforcement actions to 
inform its oversight role. These data play a critical role in helping 
EPA manage the PWSS program by identifying, for example, systems' 
return to compliance after committing violations of the safe drinking 
water standards for microbiological and chemical contaminants. 

The states collect and manage relevant data (including violations and 
enforcement information) in either a database provided by EPA--known 
as the Safe Drinking Water Information System/State (SDWIS/State)--or 
in a data system of their own design. The states also periodically 
transfer from their database information on violations and enforcement 
actions to the EPA headquarters version of SDWIS known as SDWIS/Fed. 
EPA generally uses the data in SDWIS/Fed--along with other 
documentation provided on request--to review state determinations of 
when water systems are complying with the act. EPA also uses these 
data to determine whether water systems, in the aggregate, are 
achieving the agency's national targets for compliance. Additionally, 
EPA can use enforcement data to determine whether the states or EPA 
regions have taken actions consistent with EPA's Drinking Water 
Enforcement Response Policy. The policy calls for states or EPA 
regions to take enforcement actions that are timely and appropriate 
for returning the water system to compliance with safe drinking water 
standards. The quality of drinking water data in SDWIS/Fed was called 
into question in the late 1990s and was the subject of a 2004 report 
by EPA's Office of Inspector General.[Footnote 5] 

In this context, you asked us to review the SDWIS/Fed data. Our 
objectives were to examine the (1) quality of the SDWIS/Fed data that 
EPA uses to measure community water systems' compliance with the 
health-based and monitoring requirements in SDWA and the status of the 
states' and EPA regions' enforcement actions, (2) ways in which SDWIS 
data quality could affect EPA's management of the PWSS program, and 
(3) actions EPA and the states have been taking to improve the quality 
of data in SDWIS/Fed. 

To address the first objective, we examined the results of audits EPA 
conducted from 1996 through 2009 to assess the completeness and 
accuracy of the data that states submitted to SDWIS/Fed (data 
verification audits). EPA's most recent published analysis of its 
audits was released in 2008 and covered audits done in 2002 through 
2004. We evaluated the methods that EPA used to conduct those audits 
to test the methods' validity and determined that these methods 
produced audit data that were sufficiently reliable for the purposes 
of our review. EPA also conducted audits in 2005 through 2009, but it 
had not published its analysis of those audits at the time of our 
review. We therefore obtained the results of the 2007, 2008, and 2009 
audits from EPA and conducted our own analysis of data quality using 
the methods that the agency used in its 2008 report.[Footnote 6] To 
identify factors that affected the quality of the data, we surveyed 
all 44 members of three joint EPA-state work groups that were created 
to address various aspects of data management; we received the views 
of all of the members. We examined EPA's national SDWIS/Fed data from 
2005 through 2009 to determine the percentage of violations that the 
states identified as returned to compliance, addressed through an 
enforcement action, or not addressed. Because EPA's recent audits of 
state data did not assess the completeness and accuracy of these data, 
we interviewed EPA and state officials to obtain their views on the 
completeness and accuracy of those data and analyzed relevant comments 
from our survey respondents. 

To address the second objective, we examined the potential impact data 
quality could have on EPA's Drinking Water Enforcement Response 
Policy, which uses a scoring system that identifies community water 
systems that are a high priority for enforcement action because of 
unresolved violations.[Footnote 7] We examined the impact that using 
data from the data verification audits could have on the scoring 
system compared with using data from SDWIS/Fed. We also examined the 
views of the survey respondents on the impact that data quality may 
have on implementation of the Enforcement Response Policy. Further, we 
examined the impact data quality could have on the agency's ability to 
inform the public and Congress about water systems' compliance with 
drinking water standards relative to strategic targets it has set 
under the Government Performance and Results Act. To address the third 
objective, we examined the survey respondents' views on steps that EPA 
and the states could take to address data reliability--including the 
adoption of particular data management tools--and ways in which the 
three EPA-state work groups could be more effective. We also examined 
information EPA provided on recent actions it has taken to improve 
data quality, including its current proposal for modifying the SDWIS 
data management system. We did not evaluate the merits of that 
proposal. A more detailed description of our scope and methodology can 
be found in appendixes I and II. 

We conducted this performance audit from February 2010 through June 
2011 in accordance with generally accepted auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient and appropriate evidence to provide a reasonable basis for 
our findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

Background: 

This section provides information on the risks posed by unsafe 
drinking water, the authority EPA gives to states under SDWA, 
differences among community water systems, EPA and the states' 
processes for entering water systems' data into SDWIS/Fed, EPA's 
Enforcement Response Policy and Enforcement Targeting Tool, and EPA's 
strategic targets for compliance with SDWA. 

Risks of Unsafe Drinking Water: 

While the nation's public drinking water supplies are much less prone 
to outbreaks of waterborne diseases such as cholera and typhoid than 
they were in the 19th and early 20th centuries, waterborne-disease 
outbreaks caused by microorganisms do still occur.[Footnote 8] For 
example, according to a 2006 study, an estimated 4.3 million to 11.7 
million annual cases of acute gastrointestinal illnesses in the United 
States are attributable to drinking water from community drinking 
water systems supplied by surface-water and ground-water sources. 
[Footnote 9] 

Other contaminants found in drinking water may also pose a threat to 
human health from long-term exposure at certain levels. For example: 

* Long-term exposure to disinfectants--such as chlorine--that are 
added to water to control microorganisms and the byproducts of 
disinfectants may cause anemia, stomach discomfort, and eye or nose 
irritations. In small children and infants, inappropriate exposure to 
disinfectants could lead to nervous system problems. In addition, long-
term ingestion of water with disinfection byproducts may increase the 
risk of cancer and may affect the nervous system, liver, and kidneys. 

* Arsenic, which occurs naturally and in industrial waste, may cause 
skin damage and circulatory system problems and increase the risk of 
cancer if it is not treated. 

* Lead and copper introduced into drinking water from the corrosion of 
household plumbing systems or the erosion of natural deposits may 
cause liver or kidney damage. Long-term exposure to lead may delay 
physical or mental development in infants and children. 

* Nitrate, which comes from fertilizer runoff, septic tanks, and 
erosion of natural deposits, is especially harmful to infants below 
the age of 6 months, and exposure may cause shortness of breathe, a 
serious illness known as blue-baby syndrome,[Footnote 10] and, if left 
untreated, death. 

EPA Provides Authority to State Primacy Agencies: 

Under SDWA, EPA may authorize states meeting specified requirements to 
implement the PWSS program--referred to as primacy authority. For 
example, states must have regulations for contaminants that are no 
less stringent than those promulgated by EPA, adequate record keeping 
and reporting requirements, and adequate enforcement authority to 
compel water systems to comply with drinking water requirements. EPA 
has approved primacy authority for 49 states, 5 territories, and the 
Navajo Nation. EPA's regions administer the programs in Wyoming and 
the District of Columbia, and for most tribes. EPA provides annual 
grants through the PWSS program to the states using a formula that 
takes into account population, geographical area, and the number of 
water systems covered. (EPA may also consider other relevant factors 
in its allocation formula.) In recent years, total EPA allocations to 
these grants have averaged about $100 million per year. States must 
provide matching funds; under the act, the PWSS grant can provide no 
more than 75 percent of the costs expended by a state to carry out its 
PWSS program.[Footnote 11] EPA's drinking water program guidance 
instructs EPA regions to work with the states to develop grant 
workplans that include the states' commitments to report key 
activities. For example, the State of Washington's workplan includes a 
commitment to assure complete and accurate identification and 
reporting of public water system compliance. 

Differences among Community Water Systems: 

As of July 2009, more than 51,000 community water systems supplied 
water to the same populations year-round.[Footnote 12] Community water 
systems vary widely in the number of people they serve, from 25 to 
over a million. As figure 1 shows, small systems are the most common. 
However, the 8 percent of community water systems that serve more than 
10,000 people supply approximately 82 percent of all community water 
system users. Figure 1 shows the number of community water systems in 
2009 categorized by the number of people they serve. 

Figure 1: Number of Community Drinking Water Systems that Serve 
Various Populations: 

[Refer to PDF for image: illustrated horizontal bar graph] 

Population served: 25-500; 
Number of Systems (N=51,651): 28,804. 

Population served: 501-3,300; 
Number of Systems (N=51,651): 13,820. 

Population served: 3,301-10,000; 
Number of Systems (N=51,651): 4,871. 

Population served: 10,001-100,000; 
Number of Systems (N=51,651): 3,746. 

Population served: greater than 100,000; 
Number of Systems (N=51,651): 410. 

Source: EPA SDWIS/Fed, as of June 30, 2009. 

[End of figure] 

Community water systems obtain their water from groundwater reserves 
or from surface water sources. They may obtain, treat, and distribute 
their water entirely on their own, or they may purchase treated water 
from another system. Treatment generally consists of filtration, 
sedimentation, and other processes to remove impurities and harmful 
agents, and disinfection processes such as chlorination to eliminate 
biological contaminants. 

Community water systems must meet a variety of health-based 
requirements under SDWA. These include providing drinking water that 
meets numerical limits for some contaminants,[Footnote 13] using 
treatment techniques for other contaminants, and using laboratory 
testing to monitor and report on the quality of the drinking water 
that they provide. Under SDWA, EPA may establish an enforceable 
standard--called a maximum contaminant level, or MCL--that limits the 
amount of a contaminant that may be present in drinking water. If EPA 
determines it is not economically or technically feasible to ascertain 
the level of a contaminant, the agency may instead establish a 
treatment technique to prevent known or anticipated health effects. In 
total, EPA has set MCL or treatment technique standards--known as the 
National Primary Drinking Water Regulations--for 89 regulated 
contaminants. We refer to violations of these standards as health-
based violations. 

EPA has also established monitoring, reporting, and other requirements 
for each of the 89 regulated contaminants. In this report we refer to 
these requirements collectively as monitoring requirements and refer 
to violations of these requirements as monitoring violations. These 
requirements may vary depending on several factors. For example, the 
frequency of monitoring may depend on whether the system obtains its 
water from ground water or surface water sources or upon the size of 
the water system. Additionally, if the water system detects certain 
contaminants above a specified amount, it may need to increase the 
frequency of its monitoring. Community water systems must also notify 
the public within specified times about the occurrence of health-based 
or monitoring violations and provide their customers with an annual 
Consumer Confidence Report containing data on the presence and 
concentrations of the 89 regulated contaminants. 

The States' Processes for Entering Water Systems' Data into SDWIS: 

Most of the states enter data they collect and generate on community 
water systems into a version of SDWIS designed for use by the states 
known as SDWIS/State. As EPA promulgates new or revised regulations 
for particular contaminants, it develops new SDWIS versions to capture 
data associated with those regulations. EPA encourages the states to 
place their water systems' data into SDWIS/State but the states may 
choose not to if they have an alternative database that meets their 
needs while also complying with EPA recordkeeping requirements. The 
data include inventory information about each system, such as its 
name, owner, address, and the size of the population it serves. The 
data also include the results of the water monitoring conducted 
according to contaminant-specific schedules by each system, the 
state's determination of whether the system has committed violations, 
and a record of enforcement actions taken. 

According to EPA, every 3 months, the states must transfer certain 
information from either SDWIS/State or their alternative data system 
to SDWIS/Fed. Specifically, the states transfer to SDWIS/Fed data on 
the health and monitoring violations identified and enforcement 
actions taken, and whether the state has determined that the system 
has returned to compliance.[Footnote 14] SDWIS/Fed is the data system 
EPA uses to gauge community water systems' compliance with SDWA. In 
2006, EPA and the Association of State Drinking Water Administrators 
set a goal that 90 percent of health-based drinking water violations 
be completely and accurately reported to SDWIS/Fed. EPA, however, does 
not have a goal for the completeness and accuracy of data on 
monitoring violations. 

EPA's Drinking Water Enforcement Response Policy and Enforcement 
Targeting Tool: 

Under its 2009 Drinking Water Enforcement Response Policy, EPA is to 
identify water systems with the most serious compliance problems and 
direct enforcement resources to these systems. An important component 
of EPA's enforcement policy is its Enforcement Targeting Tool for 
identifying water systems with the most serious compliance problems. 
The Enforcement Targeting Tool assigns a score to each water system 
based on an accounting of unresolved violations over a 5-year period. 
Because some violations may have more serious health consequences than 
others, the tool assigns each violation a "weight" or number of points 
based on the potential threat to public health: acute health 
violations are worth 10 points, other health violations and some major 
monitoring violations are worth 5 points, and all other monitoring 
violations are worth 1 point. Additional points are added for each 
year a violation remains unresolved. Points for each violation at a 
water system are summed to generate the system's score. Water systems 
whose scores meet or exceed a certain threshold--EPA has set the 
threshold at 11 points--are considered to have serious compliance 
problems and are placed on a priority list of water systems that the 
states and EPA are to target for enforcement. Using this approach, EPA 
and the states target resources to address those water systems that 
EPA determines have the most significant problems complying with 
SDWA's requirements.[Footnote 15] 

EPA's Enforcement Response Policy also provides guidance on the amount 
of time in which states and EPA regions should address violations at 
priority water systems.[Footnote 16] Once systems have been targeted 
as a priority, states and the regions have 6 months to work with them 
in whatever manner they deem appropriate to resolve violations and 
return the system to compliance. Enforcement and compliance assistance 
may include a range of actions such as providing violation 
notification letters to systems, offering them technical assistance, 
conducting site visits to resolve violations, entering into compliance 
agreements such as consent orders, and additional formal actions such 
as issuing administrative orders, assessing fines or penalties, and 
filing or referring judicial cases. In situations where the system is 
unlikely to return to compliance within the 6-month time frame, EPA's 
policy calls for states or the regions to take a formal enforcement 
action within 6 months that will put the system "on the path" to 
compliance by laying out future actions and time frames the system 
needs to follow.[Footnote 17] According to EPA, states and the regions 
are required to enter information on enforcement actions, including 
violation resolution, into SDWIS/State or the equivalent system as 
they occur and send those data to SDWIS/Fed every 3 months. 

EPA's Strategic Targets for Compliance with SDWA: 

EPA's two most recent strategic plans issued in 2006 and 2010 have 
included the strategic objective of protecting human health by 
reducing exposure to contaminants in drinking water.[Footnote 18] 
These strategic plans are required by the Government Performance and 
Results Act (GPRA), which calls for related annual performance plans 
to outline the process for communicating goals and strategies 
throughout the agency, and for assigning accountability to managers 
and staff for goal achievement. As we have previously reported, a 
clear relationship should exist between an agency's long-term 
strategic goals and the performance goals in the annual performance 
plan.[Footnote 19] Successful organizations try to link performance 
measures to the organization's strategic goals and, to the extent 
possible, have performance measures that will show annual progress 
toward achieving their long-term strategic goals. GPRA also requires 
that the agency publish an annual performance report communicating to 
managers, policymakers, and the public what was actually accomplished 
for the resources expended. 

To help gauge its progress relative to its objective of reducing 
exposure to contaminants, EPA uses annual performance measures and 
strategic targets to track national rates of drinking water 
compliance, including the: 

* percentage of community water systems that meet all applicable 
health-based standards--the strategic target for 2009 was 90 percent; 

* percentage of population served by community water systems that will 
receive drinking water that meet all applicable health-based drinking 
water standards--the strategic target for 2009 was 90 percent; 
[Footnote 20] and: 

* percentage of person months during which community water systems 
provide drinking water that meets all applicable health-based 
standards--the strategic target for 2009 was 95 percent.[Footnote 21] 

EPA uses the data on violations that the states report to SDWIS/Fed to 
gauge the performance of community water systems in relation to these 
GPRA strategic targets. According to EPA, the SDWIS/Fed data indicated 
that community water systems either met, or came close to meeting, 
these strategic targets in 2007 through 2009.[Footnote 22] As part of 
its effort to achieve the objective in its 2010 strategic plan to 
reduce exposure to contaminants in drinking water, EPA has also 
adopted performance indicators that it will use to track the number 
and percentage of small water systems with repeat health-based 
violations and the average time for those systems to return to 
compliance. 

SDWIS Data from States Did Not Reliably Reflect Community Water 
Systems' Violations of SDWA or the Status of Enforcement Actions: 

The data that states provided to EPA did not reliably reflect the 
frequency of community water systems' violations of SDWA's health-
based standards, according to our analysis of EPA's audit data for 
2007 and 2009 and past EPA audit reports. In addition, the data did 
not reliably reflect the frequency of monitoring violations, which are 
a predictor of health-based violations. Survey respondents support the 
concept of EPA setting a numerical goal for the percentage of 
monitoring violations accurately reported in order to increase the 
reliability of data in SDWIS/Fed. Furthermore, data provided by the 
states on the status of enforcement actions taken against systems with 
violations were incomplete, according to EPA and state officials we 
interviewed. Officials identified several factors, such as inadequate 
training, staffing, and guidance, as contributing to errors in data on 
violations and enforcement. 

States' Data Did Not Reliably Report the Frequency of Health-based 
Violations: 

Using EPA's 2007 and 2009 audits of the data that the states provided 
to SDWIS/Fed, we found that the states did not completely and 
accurately report health-based violations committed by community water 
systems.[Footnote 23] For example, we estimate that the 19 states EPA 
audited in 2007 did not report or reported inaccurately 20 percent, or 
543, of the health-based violations that EPA determined should have 
been reported.[Footnote 24],[Footnote 25] For 2009, we estimate that 
the 14 states EPA audited in that year did not report or reported 
inaccurately 26 percent, or 778, of the health-based violations that 
EPA determined should have been reported.[Footnote 26] Figure 2 shows 
our estimates of the percentage of health-based violations the states 
did not report or inaccurately reported. 

Figure 2: Estimated Percentage of Health-based Violations That States 
Audited in 2007 and 2009 Did Not Report or Inaccurately Reported: 

[Refer to PDF for image: horizontal bar graph] 

Year: 2007; 
Unreported or inaccurately reported: 20%; 
Confidence interval: the upper and lower bounds of the 95 percent 
confidence interval: 4-35%. 

Year: 2009; 
Unreported or inaccurately reported: 26%; 
Confidence interval: the upper and lower bounds of the 95 percent 
confidence interval: 12-40%. 

Source: GAO’s analysis of data from EPA’s audits of 19 states in 2007 
and 14 states audited in 2009. 

[End of figure] 

EPA's audits from 1996 through 2004 also found that violations had 
been unreported.[Footnote 27] For example, on the basis of its 2002 
through 2004 audits, EPA reported that the 37 states it audited did 
not report or inaccurately reported about 49 percent of health-based 
violations committed by community water systems to SDWIS/Fed, as shown 
in figure 3.[Footnote 28] It is not possible to infer a trend between 
2002 and 2009 because EPA's 2002-2004 results are not directly 
comparable to the results of our analysis of 2007 and 2009 audit data. 
That is because, among other things, EPA's analysis combined 3 years 
of audits and because the audits were of different states than were 
audited in 2007 and 2009. In its analysis of the completeness and 
accuracy of state data for all types of public water systems, EPA 
found that the reliability of state data on violations varied for 
different health-based standards. For example, data on violations of 
the total coliform rule and surface water treatment rules were more 
reliable than data on violations of the lead and copper treatment 
technique standards.[Footnote 29] 

Figure 3: Estimated Percentage of Health-Based Violations That States 
Audited in 2002 through 2004 Did Not Report or Reported Inaccurately: 

[Refer to PDF for image: horizontal bar graph] 

Unreported or inaccurately reported: 49%; 
Confidence interval: the upper and lower bounds of the 95 percent 
confidence interval: 37-61%. 

Source: EPA's 2002 through 2004 audits of state data. 

[End of figure] 

EPA's audits have shown two types of errors in the data the states 
submitted. The first type, known as a compliance determination error, 
occurs when a violation occurs but the state (or EPA region acting as 
a primacy agency) does not issue a violation notice to the water 
system and does not report that violation to SDWIS/Fed. The second 
type, known as a data flow error, occurs when the state or region 
issues a violation notice to the water system and is reported to the 
state data system but information about the violation is not correctly 
transferred to SDWIS/Fed. Compliance determination errors, according 
to our analysis of EPA's data, are much more common than data flow 
errors. For example, using EPA's audit data from 2009, we estimate 
that 91 percent of the errors in health-based violations between the 
audited data and SDWIS/Fed were compliance determination errors and 9 
percent were data flow errors.[Footnote 30] Among these errors were 
some state-reported violations to SDWIS/Fed that EPA determined had 
not occurred (e.g., false positives). 

States' Data Did Not Reliably Reflect the Frequency of Monitoring 
Violations, Which Are a Predictor of Health-based Violations: 

According to our analysis of EPA's audit data from 2007 through 2009, 
the states did not report or inaccurately reported the number of 
monitoring violations.[Footnote 31] For example, we estimate that the 
14 states audited in 2009 did not report or inaccurately reported 
about 54,600--or 84 percent--of the monitoring violations committed by 
community water systems to SDWIS/Fed.[Footnote 32] On the basis of 
these audit results, we conclude that the total nationwide number of 
actual monitoring violations had to have been considerably higher than 
the 82,000 reported in SDWIS/Fed.[Footnote 33] Monitoring violations, 
as we have defined them in this report, include a variety of 
situations, ranging from instances in which a water system did not do 
required monitoring, did not report the results to the state on time, 
or did not issue public notice of a health-based violation in a timely 
fashion. It is important to note that the underreporting of monitoring 
violations may affect what is known about health-based violations. 
Some unknown percentage of both reported and unreported monitoring 
violations may have hidden the presence of a health violation, 
particularly when the violation was that required monitoring was not 
done at all. 

Our analysis found that having a monitoring violation was a strong and 
statistically significant predictor of whether a system had a health- 
based violation, among systems sampled in EPA's audit data for 2007 to 
2009.[Footnote 34] Furthermore, the number of monitoring violations 
was positively and statistically significantly related to the rate of 
health-based violations. In its 2010 report on 2007 and 2008 national 
drinking water compliance rates, EPA noted that monitoring violations 
were a concern because "if a water system does not monitor and report 
on the quality of its water it is impossible to know if there are 
health-based violations."[Footnote 35] Therefore, the presence of 
monitoring violations may "mask" the presence of health-based 
violations. The total number of "masked" health-based violations is 
unknown, but may be affected by the total number of monitoring 
violations. As we have shown, the total number of monitoring 
violations is much higher than indicated by the SDWIS/Fed data, 
suggesting that the total number of health-based violations is also 
larger than indicated. 

Majority of Survey Respondents Support a Goal for the Quality of Data 
on Monitoring Violations: 

Regarding the low quality of SDWIS/Fed data on monitoring violations, 
a majority of survey respondents who expressed an opinion (20 of 34) 
indicated they thought EPA should--as it has for health-based 
violations--establish a numerical data quality goal for the percentage 
of monitoring violations that are completely and accurately reported. 
These respondents had a range of views on what a numerical goal should 
be, from a low of 41 percent to a high of 100 percent; the average was 
about 83 percent. When asked to describe the actions they thought EPA 
and the states need to take to achieve their preferred goal, the most 
common responses focused on increasing management prioritization, 
training, and information system technology. When respondents who 
indicated they thought EPA should not establish a data quality goal 
were asked to explain their answers, the most common response was that 
the quality of data on monitoring violations was not a high priority. 
For example, one respondent said that unless the states and EPA can 
address inadequate staffing, monitoring violations will continue to be 
the lowest priority. 

State Data on Enforcement Actions Were Incomplete, According to State 
and EPA Officials: 

EPA's data verification audits in 2005 through 2009 did not include 
any analysis of the accuracy of the data the states reported to 
SDWIS/Fed on their enforcement actions. According to EPA's audits for 
2002 through 2004, the audited states did not accurately report to EPA 
27 percent of the enforcement actions they took against community 
water systems.[Footnote 36] However, EPA arrived at this estimate by 
comparing data in SDWIS/Fed with the data in SDWIS/State to determine 
whether they matched. EPA did not examine original source documents 
that could have shown whether the data in SDWIS/State accurately 
represented the status of the states' enforcement actions. The 
approach EPA used in its audits to estimate the accuracy of the 
reporting of enforcement actions by states differed from the approach 
it took in its audits of violations data, in which EPA examined the 
sampling data that community water systems provided to the states. 
Consequently, EPA's estimates of the completeness and accuracy of 
enforcement data were less likely to be as reliable as its audits of 
violations data. 

EPA has not conducted recent audits of enforcement data, but officials 
we spoke with from EPA's drinking water and enforcement offices and 
three regions--as well as survey respondents--stated that current 
SDWIS/Fed data underreport the percentage of water systems where 
enforcement actions have been taken. They also indicated that the 
SDWIS/Fed data do not accurately report the percentage of water 
systems that have returned to compliance. For example, state officials 
told us that when they have quarterly discussions with the regions 
about the status of enforcement actions as shown in SDWIS/Fed they 
discover that the database is not accurate because the states have not 
consistently entered the data on enforcement actions into SDWIS/State. 

We examined violations that occurred from 2005 through 2009 to 
determine what the states have reported to EPA. According to our 
analysis of SDWIS/Fed data on enforcement, the states reported that 
less than half of these health-based and monitoring violations were 
resolved as of March 31, 2010 (see figure 4). Specifically, we found 
that about 59 percent of health-based violations and about 49 percent 
of monitoring violations committed by community water systems had not 
been resolved. However, given that the enforcement data have not been 
audited for several years, as well as the concerns of officials we 
spoke with, we cannot be certain that the results of our analysis 
accurately reflect the status of enforcement actions. 

Figure 4: Enforcement Status of Health-based and Monitoring Violations 
at Community Water Systems Reported by States to SDWIS/Fed, 2005-2009: 

[Refer to PDF for image: 2 pie-charts] 

Health based violations, 2005-2009: 
Violations that have been resolved: 41%; 
Violations with no enforcement action reported: 15%; 
Violations that received an enforcement action, but that were not yet 
resolved: 44%; 
Total, violations not resolved: 59%. 

Monitoring violations, 2005-2009: 
Violations that have been resolved: 51%; 
Violations with no enforcement action reported: 18%; 
Violations that received an enforcement action, but that were not yet 
resolved: 31%; 
Total, violations not resolved: 49%. 

Source: GAO analysis of EPA's SDWIS/Fed enforcement data through March 
31, 2010. 

[End of figure] 

EPA and State Officials Indicated That Violation and Enforcement Data 
Are Unreliable for Several Reasons: 

EPA and state officials responding to our survey or in interviews 
cited several factors as contributing to inaccuracies in SDWIS/Fed 
data on health-based and monitoring violations and the status of 
enforcement. For the violations data, some factors were cited as 
contributing to both compliance determination and data flow errors--
such as inadequate training and guidance--but the importance of these 
factors varied by the type of error. For enforcement data problems, 
other factors were often cited, such as higher priorities, inadequate 
guidance, and information system flaws. 

Incorrect Compliance Determination Data and Data Flow Errors Occur for 
Several Reasons, According to EPA and State Officials: 

We asked survey respondents whether they thought any of five factors 
(information system structure; training by state or federal agencies; 
funding from state or federal agencies; state staffing levels; or 
guidance from EPA) contributed to incorrect compliance determinations 
and to the less common data flow errors; we also asked them to 
indicate if other factors were important. As figure 5 shows, at least 
half of the 41 respondents identified each of the factors as 
contributing to compliance determination errors, with training and 
staffing cited most often. And, as the figure shows, more than two-
thirds of the 41 respondents cited the information system structure as 
contributing to data flow errors and more than half of respondents 
cited state staffing and training as contributing factors. 

Figure 5: Factors Contributing to Incorrect Compliance Determinations 
and Data Flow Errors, According to Survey Respondents: 

[Refer to PDF for image: vertical bar graph] 

Number of respondents: 

Contributing factor: Information System; 
Compliance determination factors: 22; 
Data flow errors: 28. 

Contributing factor: Training; 
Compliance determination factors: 35; 
Data flow errors: 24. 

Contributing factor: Funding; 
Compliance determination factors: 26; 
Data flow errors: 18. 

Contributing factor: Staffing; 
Compliance determination factors: 34; 
Data flow errors: 25. 

Contributing factor: Guidance; 
Compliance determination factors: 25; 
Data flow errors: 20. 

Contributing factor: Other; 
Compliance determination factors: 21; 
Data flow errors: 10. 

Source: GAO survey of EPA and state drinking water officials. 

Note: The survey population included 41 individuals, including EPA 
drinking water officials, state drinking water officials, and 
representatives of the Association of State Drinking Water 
Administrators. 

[End of figure] 

Respondents also provided more detailed information on the factors 
they identified as contributing to incorrect compliance determinations 
and data flow errors. For example, with regard to incorrect compliance 
determinations, one respondent said that training for new drinking 
water rules was limited and training for old drinking water rules was 
virtually nonexistent. Another respondent said that staffing levels 
were at an all-time low while another said that states had always 
experienced a revolving door for compliance staff. An EPA official 
responded that a state might not issue a violation because of 
"sympathy" for a water system if the state viewed the violation as not 
being a major health problem. With regard to data flow errors, several 
respondents said that SDWIS does not have adequate quality control 
features to clearly identify errors that might occur when the states 
transfer violations data from SDWIS/State to SDWIS/Fed. 

When we asked survey respondents to identify the most important steps 
they believe that EPA could take to address compliance determination 
errors, the most frequent suggestions--from 18 of the 41 respondents-- 
concerned training and guidance. For example, 9 respondents said that 
EPA needs to improve the timeliness of guidance on how to make 
compliance determinations to ensure that the guidance does not come 
after the date that a drinking water rule takes effect. With respect 
to actions states should take, the most frequent comments related to 
management and training, from 25 and 22 respondents, respectively. 
Many of the comments regarding management called for states to conduct 
more thorough oversight or to hold their staff accountable. 

Survey respondents also identified the most important steps they 
believe that EPA could take to address data flow errors. The most 
frequent suggestions for lowering the error rate concerned information 
system structures and management, from 16 and 15 respondents, 
respectively. With respect to information systems, respondents said 
that EPA should take action to address the quality, complexity, or 
ease of use of SDWIS. Most who commented on management called for more 
oversight and accountability. 

Unreliable Data on Enforcement Actions Are Attributed to Higher 
Priorities, Inadequate Guidance, and Information System Flaws: 

According to EPA and state officials we interviewed, as well as survey 
respondents, the factors that contributed to concerns about incomplete 
data on enforcement actions and water systems' return to compliance 
are similar to those that contributed to unreliable data on 
violations.[Footnote 37] For example, EPA and state officials told us 
that some state agencies have not routinely and thoroughly entered 
data on enforcement actions or returns to compliance into SDWIS/Fed 
because it is a low priority for their limited staff. Officials from 
EPA regions said this is particularly the case for monitoring 
violations that states may have considered less serious than 
violations of health standards. 

State and EPA officials also cited a lack of guidance from EPA on what 
conditions must exist for a system with a violation to be recorded as 
returned to compliance as having been a factor contributing to 
incomplete data on enforcement. Recognizing the importance of these 
definitions, EPA collaborated with states and the Association of State 
Drinking Water Administrators on guidance it issued in 2010. EPA 
regional officials told us the new definitions would likely lead to 
improvements in the states' reporting on returns to compliance. In 
addition, officials we spoke with stated that SDWIS/Fed used to have 
an automated function that would categorize some common violations as 
returned to compliance if certain subsequent conditions existed. 
However, that function is no longer available, meaning that state 
officials need to enter the information manually. In its comments on a 
draft of this report, EPA said that the function was removed because 
it did not work correctly. 

See appendix II for more details on the results of our survey. 

Incomplete and Inaccurate SDWIS/Fed Data Hamper EPA's Ability to 
Manage the PWSS Program and Communicate Progress toward Its Strategic 
Objective: 

Incomplete and inaccurate data on violations and enforcement actions 
limit EPA's ability to identify water systems with the most serious 
compliance problems and ensure its enforcement goals are met. 
Unreported violations and unreliable enforcement data also impede 
EPA's ability to monitor and fully communicate to Congress and the 
public the agency's progress toward its strategic objective of 
reducing the public's exposure to contaminants in drinking water. 

Incomplete and Inaccurate Data on Violations and Enforcement Actions 
Reduce EPA's Ability to Ensure Its Enforcement Goals Are Met: 

Incomplete and inaccurate data on violations and enforcement actions 
reduce EPA's ability to ensure that it is achieving its goal of 
targeting for enforcement those systems with the most serious 
compliance problems. Specifically, the lack of reliable data in SDWIS/ 
Fed reduces the usefulness of EPA's Enforcement Targeting Tool for 
identifying water systems with the most serious compliance problems. 
That is, water systems without a complete violations record in SDWIS/ 
Fed could receive a lower enforcement targeting score indicating a 
higher level of compliance than other systems whose violation record 
is complete. Conversely, systems whose return to compliance has not 
been recorded in SDWIS/Fed could receive a score that is higher, or 
worse, than warranted. According to EPA's current enforcement policy, 
water systems whose scores equal or exceed 11 points are considered to 
have serious compliance problems and are targeted for enforcement 
actions. 

To demonstrate the effect that unreported health and monitoring 
violations have on the implementation of the Enforcement Targeting 
Tool, we calculated two scores for each community water system audited 
by EPA in 2007, 2008, and 2009--a total of 1,225 systems over the 
period. One score was based on more complete data incorporating the 
violations found in the data verification audits, and the other score 
was based on violations in SDWIS/Fed.[Footnote 38] Because the audited 
data are a more complete dataset, we expected to see, and indeed 
found, differences between the two scores for each system. For 16 
percent of the systems, the point difference between the two scores 
alone equaled or exceeded the 11-point threshold. Another 14 percent 
had scores that were 6 to 10 points higher, which would increase the 
likelihood that these systems would have been prioritized for 
enforcement under EPA's targeting tool. The results of our analysis 
are shown in figure 6. 

Figure 6: Percentage of Audited Water Systems with a Difference in 
Their Enforcement Targeting Tool Scores Derived from SDWIS/Fed Data 
and Audited Data, 2007-2009: 

[Refer to PDF for image: pie-chart and associated stacked bar] 

No difference in scores: 27%; 
Difference of 10 points or less: 57%; 
Difference exceeds 10 points: 16%. 

Differences subset: 
+11 to +170 points: 16%; 
+6 to +10 points: 14%; 
+1 to +5 points: 41%; 
-1 to -18 points: 2%. 

Sources: GAO analysis of EPA’s SDWIS/Fed data and data verification 
audit data from 2007 through 2009. 

Notes: The range of differences shown in the stacked bar result from 
subtracting the score we derived using SDWIS/Fed data from the score 
we derived using the data verification audit data. 

EPA's current Enforcement Targeting Tool threshold is 11 points. 

[End of figure] 

Overall, according to our analysis, 73 percent of the water systems 
(or 892) had a different score using the two sets of data. Twenty-
seven percent (333) of the water systems showed no difference between 
the scores calculated using the two sets of data. We found that the 
majority of score differences were the result of unreported monitoring 
violations. The Enforcement Targeting Tool assigns a much lower weight 
to monitoring violations than to health-based violations, but, as 
previously discussed, the number of monitoring violations plays an 
important role in limiting EPA's ability to identify systems with 
serious compliance problems. While most of the 1,225 water systems had 
a higher score with audited data than with SDWIS/Fed data, 2 percent 
(21 systems) had lower scores because EPA found in its audit that the 
violations had not occurred. 

When the SDWIS/Fed data are incomplete, EPA's ability to identify and 
set priorities for enforcement in water systems is compromised. For 
example, because the Enforcement Targeting Tool uses SDWIS/Fed data 
that may be missing violations, some systems may not be assigned 
enough points to exceed EPA's threshold of 10 points for priority 
enforcement action. For some of these systems, one or two additional 
points may be all that are needed to exceed the threshold and in other 
cases, as described below, the point difference for a particular 
system exceeded the threshold by an extraordinary amount. For example, 
we calculated the following for three systems: 

* A 170 point difference: The score we calculated was 3 points for one 
water system in Vermont using SDWIS/Fed data, but 173 when we 
accounted for unreported health and monitoring violations that EPA 
found in its 2009 audit. 

* A 138 point difference: The score we calculated was 0 using 
SDWIS/Fed data for a tribal system in New York that EPA's Region 2 
office oversees as the primacy agency, but 138 when we accounted for 
unreported health and monitoring violations found during EPA's 2009 
audit. 

* A 95 point difference: The score we calculated was 0 using SDWIS/Fed 
data for a system in Utah, but 95 when we used data from EPA's audit. 
The difference was entirely attributable to unreported monitoring 
violations. 

Our analysis echoes concerns voiced by respondents to our survey; 22 
of 41 respondents indicated that the usefulness of EPA's Enforcement 
Targeting Tool is affected by limitations in the SDWIS/Fed 
database.[Footnote 39] One respondent said "missing data will 
significantly affect the usefulness of the results." Another 
respondent said the tool "is hinging on the information recorded in 
SDWIS/Fed" and that "the tool is as good as the data provided." 

Survey respondents and state and EPA officials also reported that 
incomplete or inaccurate data on the resolution of violations could 
result in a water system receiving a higher score for enforcement 
priority than it merits. EPA and state officials told us that states 
do not always indicate in SDWIS/Fed that a violation is resolved, 
perhaps causing the Enforcement Targeting Tool to mistakenly place the 
system on the targeted enforcement list. According to one survey 
respondent, this condition will "confuse states and lead to continued 
poor quality data." Another respondent said that use of the 
Enforcement Targeting Tool "is a waste of time" without steps taken to 
fix this issue. State officials told us that in their regular review 
of the targeted list with EPA regional officials, they can recognize 
when a system has been erroneously included on the targeting list 
because resolved violations were not recorded and they can correct the 
discrepancy. However, EPA officials have told us this data correction 
process is a time-consuming one that places additional demands on 
limited state and EPA enforcement staff. 

Incomplete SDWIS/Fed data can also limit EPA's ability to ensure that 
the states meet the agency's enforcement goal that targeted systems 
have returned, or are returning, to compliance in a timely fashion. 
EPA's Enforcement Response Policy calls for states to work with 
systems to resolve violations or put the system on a "path to 
compliance" within 6 months of when the system becomes a priority 
system on an Enforcement Targeting Tool list. However, unreported data 
on enforcement actions can hamper EPA's ability to determine whether 
states have met that goal. For instance, while states might take an 
enforcement response that leads, or will lead, a water system to 
resolve the violation, states frequently do not enter this information 
into the SDWIS/Fed database or enter the information months or years 
later, according to EPA and state officials we spoke with. Either 
situation hampers EPA's ability to track the timeliness of enforcement 
responses. 

Unreported Violations and Enforcement Data Impede EPA's Ability to 
Monitor and Report Progress Toward Its Strategic Objective of Reducing 
Exposure to Contaminants in Drinking Water: 

Unreported violations and enforcement data impede EPA's ability to 
fully measure and communicate its progress toward meeting the 
strategic objective of reducing human exposure to contaminants in 
drinking water. The agency has established a number of indicators and 
targets that it uses to measure its progress toward meeting that 
objective. However, the unreliable quality of the violations data and 
concerns about the accuracy of enforcement data in SDWIS/Fed make it 
difficult for EPA to reliably communicate the relative public health 
risk posed by community water systems' noncompliance with SDWA and the 
progress made in resolving noncompliance in a timely manner. For 
example: 

* EPA's 2011 national water program guidance contains an indicator for 
the number and percentage of systems serving less than 10,000 people 
with certain repeated health-based violations.[Footnote 40] EPA's 
ability to set and reliably use this type of indicator requires 
complete and accurate data on violations, but as we have shown, the 
SDWIS/Fed data on violations are not reliable. 

* EPA's 2011 national water program guidance also contains an 
indicator for the average time taken for systems serving less than 
10,000 people to return to compliance after committing certain health-
based violations. However, the ability to set and reliably use an 
indicator of this type requires complete and accurate data on 
enforcement actions. As we have previously indicated, EPA and state 
officials we interviewed told us the enforcement data in SDWIS/Fed are 
not reliable. 

Unreliable data quality also limits EPA's ability to introduce or 
modify targets to manage its program and communicate progress in 
meeting the program's goals. Quality data are necessary to accurately 
measure performance relative to strategic targets. Two key EPA 
strategic targets associated with the agency's strategic objective of 
reducing exposure to contaminants in drinking water--the percentage of 
community water systems that met all health-based standards and the 
percentage of the population served by community water systems that 
received drinking water that met all applicable health-based drinking 
water standards--are broad measures of compliance. However, these 
measures do not provide information on the relative severity of the 
violations or account for systems that have multiple health-based 
violations, offering the public a narrow view of the quality of the 
nation's water systems and not clearly communicating the public health 
risk posed by these systems' noncompliance with SDWA. For example, a 
water system with multiple health-based violations is effectively 
"counted" the same as a system with one health-based 
violation.[Footnote 41] Thus, the relative health risk posed by 
different systems' noncompliance is not apparent. Without complete and 
accurate SDWIS/Fed data it is difficult to develop a new measure or 
modify these strategic targets. Similarly, without complete and 
accurate data from the states, EPA will be unable to establish 
reliable measures or targets regarding the rate of reduction in health-
based violations or compliance with monitoring requirements or further 
EPA's core value of transparency. 

Actions EPA and States Are Taking to Improve the Quality of Data in 
SDWIS Have Not Been Fully Successful and More Actions Are Planned: 

EPA and the states have taken actions over many years to identify and 
address the causes of incomplete and inaccurate violations data, but 
those efforts have not been fully successful, according to those we 
surveyed. EPA has conducted audits to assess the quality of state 
violation data in SDWIS/Fed and developed recommendations for 
improving data quality. Survey respondents generally reported that 
those audits have contributed to improvement, but EPA has discontinued 
them. EPA and the states also established work groups to address data 
management and quality. In addition EPA has emphasized the importance 
of specific data quality management tools, although it has not 
required states or regions to use them. More recent EPA initiatives 
include a new strategy for data sharing, plans to redesign SDWIS, and 
a new tool to help the states make and report compliance 
determinations and enforcement actions. 

According to Survey Respondents, EPA's Audit Recommendations 
Contributed to Improving Data Quality, but the Agency Has Discontinued 
Them, at Least Temporarily: 

As described earlier, EPA used its data verification audits to assess 
the quality of the violations data and, to a lesser extent, the 
enforcement data the states have submitted to SDWIS/Fed. The agency 
also used the audits it conducted from 1996 through 2004 to develop 
state-specific and national recommendations for improving data 
quality. EPA and state officials we surveyed had mixed, but generally 
favorable, views about the value of the audits' recommendations with 
regard to improving data quality. Eight respondents said the 
recommendations were very effective in improving data quality, while 
most respondents (26 of 39) said the recommendations were only 
slightly or moderately effective.[Footnote 42] 

According to respondents, the audits pointed out states' 
inefficiencies and poor practices. For example, one respondent said 
that states are able to use the results as a guide to improve training 
for staff and improve data quality. Despite the recommendations 
offered to help states, six respondents indicated that the states or 
regions did not adequately change their practices in response to the 
audit findings. For example, one EPA headquarters manager commented 
that states may incorrectly interpret systemic problems identified 
through the audit as isolated problems to be corrected only at the 
water systems covered by the individual audits. Nonetheless, seven 
respondents stated that the audits' scope or methodology was not 
adequate to determine data quality.[Footnote 43] 

EPA discontinued the audits of violations data in 2010 due to funding 
constraints. According to the Director of the Office of Ground Water 
and Drinking Water, EPA may be able to resume the audits in 2011, but 
at a much reduced number. EPA conducted an average of about 17 audits 
of states, regions, and other primacy agencies in 2007 through 2009, 
but the director told us in December 2010 the agency may be able to do 
4 or 5 in 2011. EPA had not done any 2011 audits as of June 2011. In 
its comments on a draft of this report, EPA said that the Office of 
Water will conduct six to eight audits in 2011. 

EPA and the States Have Established Joint Work Groups to Address Data 
Issues: 

In the 1990s, EPA and the states jointly established two work groups 
charged with providing analysis and recommendations on various aspects 
of data management and formed a third such group in 2010. According to 
most of their members, the two older groups--the Data Management 
Steering Committee and Data Technical Advisory Committee--were 
effective at helping EPA and the states improve data quality.[Footnote 
44] On the other hand, the members of the newer Data Quality Work 
Group were almost evenly divided on whether this new group has been 
effective or not effective.[Footnote 45] 

The Data Management Steering Committee is charged with supporting EPA 
and the states in their cooperative efforts to enhance management of 
drinking water data. In explaining their answers to a question about 
the group's effectiveness, one-third of the members said the committee 
had helped EPA and the states understand the nature of the data 
quality problem and about one-half said it provided direction. 
However, one-third of the members commented on the lack of 
implementation of the committee's recommendations. 

The Data Technical Advisory Committee is responsible for recommending 
ways to obtain the data EPA needs to carry out its PWSS program 
responsibilities. About three-fifths of the members commented that the 
committee had helped EPA and the states understand data problems and 
had provided direction. However, similar to the steering committee, 
advisory committee members had concerns about EPA's implementation of 
recommendations, with close to half saying that the agency's 
implementation had been inadequate. 

The Data Quality Work Group met several times in 2010 and outlined 
draft recommendations for improving data quality, including additional 
training, standard operating procedures for staff managing any new 
drinking water rules, checklists of rule milestone dates for states, 
and quality assurance/quality control checks for SDWIS data. However, 
the Office of Ground Water and Drinking Water diverted the work group 
staff in 2010 to focus their attention on implementing the 
Administrator's Drinking Water Strategy before the group could issue 
final recommendations, according to senior office staff. Perhaps in 
light of that, among the most common comments from members of the 
group was that the work group was too new to evaluate or that its 
activity level had been inadequate. The Office of Ground Water and 
Drinking Water staff noted they would consider the work group's draft 
recommendations in its redesign of SDWIS. 

The Office of Ground Water and Drinking Water Has Emphasized the 
Importance of Data Quality but Not Required States or Regions to Take 
Actions: 

In March 2008, EPA reported the results of the audits it conducted of 
state data in 2002 through 2004. As it had done in its prior reports 
on state audits, EPA included recommendations for improving data 
quality in this report; these recommendations took the form of an 
action plan in its 2008 report.[Footnote 46] The 2008 action plan was 
a joint effort of EPA and the Association of State Drinking Water 
Administrators to provide recommendations for achieving the goal set 
in 2006 of 90 percent complete and accurate data for health-based 
violations, as well as improving the quality of monitoring violations 
data. According to the action plan, the largest challenge was ensuring 
that all data reflecting determinations of violations were entered 
into SDWIS/Fed. The plan called for, among other things, the 
development of new data management tools as well as the implementation 
of tools that EPA has developed over the years to improve compliance 
determinations. However, as discussed below, widespread implementation 
of those tools has not yet occurred. 

In April 2009, the Director of EPA's Office of Ground Water and 
Drinking Water issued a memorandum to EPA's regional water management 
directors calling attention to (1) the incomplete implementation of 
the 2008 action plan and (2) the importance of increasing oversight 
and accountability of the states for the quality of drinking water 
data. Noting that the quality of drinking water data in SDWIS/Fed was 
called into question in the media in the late 1990s and was the 
subject of a 2004 report by EPA's Office of Inspector General, 
[Footnote 47] the director stated that the quality of data continued 
to be too low. She also cited the 2006 agreement between EPA and the 
Association of State Drinking Water Administrators to set a goal that 
90 percent of health-based drinking water violations be completely and 
accurately reported to SDWIS/Fed and said that more than 10 states had 
met the goal, but the overall goal had not been met. 

The Director of the Office of Ground Water and Drinking Water 
Requested Help in Implementing the 2008 Action Plan: 

In her memorandum, the director noted that one of the conditions of 
primacy the states must meet is to report all violations. To improve 
data quality, the director called upon the regions to increase their 
efforts to implement the 2008 action plan. Her memorandum and the 
action plan call for the states to increase their use of several data 
management tools that EPA has developed over the years to improve 
accuracy, including SDWIS/State, electronic data verification (eDV), 
and electronic reporting from laboratories to states. Although EPA has 
developed these tools to improve compliance determinations and data 
flow, the states are not required to use them as a condition of 
primacy or their PWSS grant agreements. We gathered information from 
EPA regarding the current status of these tools and asked survey 
respondents to comment on the factors that have limited the use of 
these tools and the steps they believed should be taken to increase 
the tools' use. 

SDWIS/State: 

EPA reported to us in March 2011 that eight states, one tribe, and one 
territory were not using SDWIS/State at all.[Footnote 48] Of the 
states that do use SDWIS/State, some report using the database only 
for particular drinking water rules. For example, an EPA survey of 
states in mid-2010 found that 20 of the 55 primacy agencies were using 
SDWIS/State to make compliance determination decisions for the surface 
water treatment rule and 35 were using it for the total coliform 
rule.[Footnote 49] 

Due to limitations in the data available to us and inherent 
difficulties in establishing a cause-and-effect relationship, we could 
not determine whether a state's use of SDWIS/State leads to more 
reliable data on violations of particular SDWA rules.[Footnote 50] 
However, 26 of the 41 respondents indicated that more widespread use 
of SDWIS/State would improve data quality; 5 respondents indicated it 
would not; and 10 indicated they did not know. Of the 26 respondents 
who provided detail on why they thought more widespread use of SDWIS/ 
State would improve data quality, 17 indicated it would promote more 
consistent and accurate compliance determinations through automation. 
The most common theme among the respondents' suggestions for what EPA 
should do to address the factors that have prevented full use of SDWIS 
related to the quality, complexity, and ease of use of the system. For 
example, one respondent said that EPA should be aware of the needs of 
drinking water managers, who are the principal users of SDWIS/State, 
rather than database managers. 

Electronic Data Verification: 

According to EPA officials, as of March 2011, only seven states had 
done pilot tests of EPA's electronic quality control tool for SDWIS/ 
State, known as electronic data verification, or eDV. EPA officials 
told us that the eDV tool needs additional refinement to be fully 
compatible with SDWIS/State. eDV could assist states in making 
compliance determinations according to 18 of the 24 survey respondents 
who were familiar with it. They said that using the tool would improve 
data quality by improving states' oversight or auditing capability. 
However, EPA and state officials told us this tool can be used only by 
states that use SDWIS/State to manage all of the drinking water rules, 
and survey respondents noted that the need to fully use SDWIS/State 
was a factor that prevented more states from using eDV. The survey 
respondents' most common recommendation for EPA was to improve the 
quality and ease the use of the tool. For example, one respondent said 
that EPA needs to update the tool to keep pace with changes in 
regulations. At the same time, many respondents suggested that the 
states need to make a greater commitment to using SDWIS/State and eDV 
more. 

Electronic Reporting from Laboratories to States: 

EPA has developed a tool that testing laboratories can use to 
electronically transmit the results of community and other public 
water systems' monitoring directly to the state. However, according to 
EPA's mid-2010 survey, only 19 states were using this tool. Of the 40 
survey respondents who expressed an opinion, 39 believed that more 
widespread use of this tool would improve data quality, and 25 of 
these respondents stated the tool would reduce data entry errors and 
increase accuracy. Another 11 respondents said the tool would increase 
the speed of data exchange between the states and EPA, and 10 said it 
would free up state resources that could be used to improve data 
quality in other ways. For example, one respondent said that, in the 
long run, electronic reporting should save state resources by reducing 
the need for data entry staff but that in the short run, switching to 
electronic reporting requires technical support for the laboratories 
and additional resources. 

According to our survey respondents, the two leading barriers to 
having more states require electronic reporting are (1) laboratories' 
inadequate capability to implement the reporting technology and (2) 
the states' lack of legal authority to make the tool's use a 
requirement. Specifically, many survey respondents said that 
laboratories, particularly small ones, are not always adequately 
equipped or staffed to adopt electronic reporting. Several survey 
respondents said that EPA needs to provide support to laboratories to 
make it easier to adopt the tool. Some respondents also said that as 
EPA's current SDWA regulations do not require electronic reporting and 
some state laws prohibit state agencies from including requirements in 
their PWSS programs that are more stringent than what is required by 
SDWA, the state agencies are unable to require electronic reporting. 
Nine respondents said that EPA should require electronic reporting. 

Several respondents also identified EPA's Cross-Media Electronic 
Reporting Rule as a regulatory barrier to more widespread use of 
electronic reporting.[Footnote 51] This rule provides the legal 
framework for electronic reporting under all of the agency's 
environmental regulations.[Footnote 52] The rule requires states, 
tribes, and local governments that wish to use electronic reporting 
for implementing authorized federal environmental programs to obtain 
EPA approval, which may require modifications to electronic reporting 
systems to meet EPA requirements. One state respondent said that many 
states do not have the financial resources to build a system to 
receive data electronically because of the constraints of this rule, 
noting that the rule places very tight requirements on the security 
required for receiving data electronically. Several respondents called 
on EPA to review the need for the rule or abolish it. 

The Director of the Office of Ground Water and Drinking Water also 
Called for Increased Regional Oversight and Accountability: 

Citing the need for strong regional oversight of the states' PWSS 
programs, the director also requested in her April 2009 memorandum 
that the regions take specific actions over and above those identified 
in the 2008 action plan. Specifically, she called for the regions to: 

* provide documentation of good standard operating procedures and 
lessons learned that may enable EPA to improve SDWIS/Fed data quality, 
among other things; 

* discuss with states annually (or more frequently) the completeness 
and accuracy of the violations data reported to SDWIS/Fed, including a 
review of the state's implementation of recommendations contained in 
previous data verification reports; and: 

* include language in future PWSS grant agreements indicating that the 
state must make compliance determinations that are consistent with 
applicable drinking water regulations, report all violations and 
enforcement actions to SDWIS/Fed in a timely fashion, and otherwise 
comply with 40 CFR §142.15. Also include any corrective action steps 
identified by data verification audits or program reviews in the 
state's annual work plan. 

Although the Director of the Office of Ground Water and Drinking Water 
requested these actions, she told us her office does not have the 
authority to require the regions--which report directly to the EPA 
Administrator--to do so. The director and other drinking water program 
managers we interviewed told us the regions had responded to her 
request, but the office had not assessed and could not document the 
extent to which the regions had complied with the requests to discuss 
data quality with the states or add relevant language to grant 
agreements.[Footnote 53] In its comments on a draft of this report, 
EPA stated that all of the regions have incorporated data quality into 
their discussions with states and data quality has been incorporated 
into grant agreements or state workplans. However, we were unable to 
verify these statements; in response to our request during the comment 
process, EPA said that documentation was not available. According to 
EPA, it made the statements on the basis of e-mail communications and 
discussions between the managers and staff in the Office of Ground 
Water and Drinking Water and regional management and staff. 

The director also requested comments from the regions' water 
management directors on four proposed measures that would assist the 
office in monitoring the regions' oversight of the states' 
performance, including several directly related to the steps discussed 
above: 

* the percentage of states within a region with which the region has 
an annual discussion regarding data quality; 

* the percentage of states that have an action plan to correct 
deficiencies relating to drinking water compliance determinations or 
data reporting that were noted in the most recent EPA data 
verification audit; 

* the percentage of a region's annual PWSS grants that include grant 
conditions requiring the states to make compliance determinations that 
are consistent with drinking water regulations; and: 

* the extent to which a region is achieving EPA's goal that 90 percent 
of health-based violations are completely and accurately reported to 
SDWIS/Fed, when a region is acting as the primacy authority for a 
particular rule in a state. 

According to Office of Ground Water and Drinking Water officials, the 
regions responded to the director's request for comments but did not 
fully support the proposed performance measures, and none of these 
have been implemented as of March 2011. 

EPA Administrator Issued New Strategy for Data Sharing and EPA 
Announced Plans to Redesign SDWIS: 

In March 2010, the Administrator of EPA issued a drinking water 
strategy that called for, among other goals, the agency and the states 
to increase data sharing on water systems.[Footnote 54] As part of 
this strategy, EPA announced it will redesign SDWIS to help to meet 
the Administrator's goals for data sharing.[Footnote 55] According to 
the director of the Office of Ground Water and Drinking Water, 
software for the next generation of SDWIS is only at the beginning 
stages of development; she anticipates it will be ready by 2014, 
depending upon the availability of funding. To help achieve the goals 
of the strategy, EPA signed a memorandum of understanding in November 
2010 with three associations that represent state agencies and 
officials.[Footnote 56] EPA also formed an Implementation Work Group 
comprising agency and state officials to further the data sharing 
goals spelled out in the memorandum. 

The November 2010 memorandum of understanding on data sharing--which 
is a voluntary agreement among the parties--outlines the vision, 
goals, terms, and conditions under which drinking water monitoring 
data are to be exchanged between the states and EPA. The anticipated 
benefits of data sharing include allowing states to more readily 
compare their water system monitoring results with EPA regulations and 
with the SDWIS/Fed data before the states submit the data to EPA. EPA 
intends to use the shared data for a variety of purposes, including 
calculating national and state data completeness, accuracy, and 
timeliness; evaluating differences in state interpretations of EPA 
regulations; and conducting national program oversight. The mission of 
the Implementation Working Group is to recommend ways for states to 
share appropriate compliance monitoring data that eventually will be 
housed in the next generation of SDWIS. 

EPA officials said they expect this redesign of SDWIS--and 
accompanying revisions to state data submission requirements--to 
expand the amount of data that EPA receives electronically from the 
states. With SDWIS/Fed, EPA generally only receives data from the 
states on inventories, violations, and enforcement actions. According 
to the Director of EPA's Office of Ground Water and Drinking Water, 
the next generation of SDWIS would give EPA access to the compliance 
monitoring and enforcement data now collected by the states. She told 
us that having direct access to the states' raw monitoring data would 
improve EPA's ability to better understand national patterns of 
compliance and to diagnose problems faced by states. For instance, 
according to the director, the data could reveal that particular rules 
are hampered by a misunderstanding of the requirements, and EPA could 
use that information to write regulations that are easier to 
understand and report. The director said that a redesigned SDWIS could 
also reveal and address instances when a state has made a compliance 
determination error. However, she also said that some compliance 
determination errors can be addressed with a redesigned SDWIS, but 
others might need to be addressed through better training or writing 
regulations more clearly so that state staff understand what 
constitutes a violation. 

According to the director, under the agency's current position, states 
will continue to have the option to use the next generation of SDWIS. 
EPA will continue to provide ways for states that do not use SDWIS to 
transfer their data to the agency. However, EPA will expect those 
states, like those using SDWIS, to share their compliance monitoring 
data with EPA.[Footnote 57] 

EPA Is Developing a New Compliance Determination and Violations/ 
Enforcement Reporting Tool: 

In November 2010, the Office of Ground Water and Drinking Water 
unveiled an initial version of another tool for improving compliance 
determinations and data quality--the Compliance Determination and 
Violation/Enforcement Reporting Tool. According to the office, the 
tool was designed by Region 5 staff to consolidate, update, and 
supplement EPA guidance on SDWA violations of specific requirements in 
one electronic document. The tool includes violation descriptions, 
compliance determinations, violation reporting instructions, common 
discrepancies from data verifications, related EPA memos, enforcement 
tracking instructions, and return to compliance definitions. The 
target audience for the tool is state and regional compliance, 
enforcement, and data staff. The tool is being developed in modules 
for each National Primary Drinking Water Regulation. The first module 
of the tool was for the lead and copper rule. EPA officials said they 
anticipate all modules will be developed by the end of fiscal year 
2011 if funding is available. 

Conclusions: 

EPA relies on the soundness of state-reported data to ensure that 
community water systems are complying with SDWA and that the states 
and regions are taking appropriate enforcement actions against 
noncompliant water systems. As our analysis of EPA's audit data for 
2007 through 2009 shows, however, states continue to fall short in 
providing accurate and complete data on health-based and monitoring 
violations. EPA's data verification audits after 2004 did not examine 
the quality of data on enforcement actions and systems' return to 
compliance, but EPA officials and survey respondents told us that 
current state data on such actions are also incomplete or inaccurate. 
In addition to discontinuing its audits of enforcement data after 
2004, EPA also discontinued its audits of violation data after 2009 
because of budget constraints. The agency hopes to resume these audits 
of violation data in 2011, but the number of states to be audited 
would be greatly reduced from an annual average of about 17 from 2007 
through 2009 to 6 to 8. Conducting fewer audits of state-reported 
data--both violations and enforcement data--will hamper the 
effectiveness of EPA's oversight of the states and its ability to 
assess its efforts to improve data quality. 

EPA and the Association of State Drinking Water Administrators 
established a goal of completely and accurately reporting 90 percent 
of health-based violations, but EPA has not set a similar goal for 
monitoring violations. Monitoring violations may reflect a wide range 
of circumstances, such as instances in which monitoring was done but 
was not reported to the state in a timely fashion or the potentially 
more serious situation in which required monitoring was not done at 
all. We found, however, that the number of monitoring violations was 
positively and statistically significantly related to the rate of 
health-based violations. Recognizing the importance of having complete 
and accurate data on monitoring violations, a majority of those state 
and EPA officials we surveyed who voiced an opinion supported the idea 
of having a goal for the quality of data on monitoring violations. 

As called for by GPRA, EPA has established several performance 
measures with associated targets and indicators for community water 
systems to assess progress toward the agency's strategic objective of 
reducing exposure to contaminants in drinking water. Each year, EPA 
publicly reports systems' performance levels relative to the targets 
using data from SDWIS/Fed. To be useful and appropriate, these 
performance measures should clearly reflect conditions that directly 
relate to human exposure to contaminants. However, we found that some 
of the measures that EPA relies upon to gauge national compliance 
levels measure how many systems are out of compliance but not the 
extent to which they are out of compliance, which does not clearly 
communicate the public health risk posed by these systems' 
noncompliance with SDWA. In addition, some measures that EPA uses 
depend on data on violations and the status of enforcement actions 
that are unreliable. We found that incomplete and inaccurate data 
could impede EPA's ability to monitor and report progress toward its 
strategic targets for those measures, including having those systems 
return to compliance in a timely manner. 

Recognizing its long-standing problem of receiving incomplete and 
inaccurate state data on violations, EPA has made efforts to improve 
the quality of data reported to SDWIS/Fed. However, many of those 
efforts, including those to implement EPA's 2008 action plan, have not 
been fully successful. In light of the need for more improvement, the 
Director of the Office of Ground Water and Drinking Water requested in 
her 2009 memorandum that the regional water managers take numerous 
steps to implement the action plan--including encouraging the states 
to increase their use of SDWIS/State, electronic data verification, 
and electronic reporting from laboratories to states. Both the action 
plan and additional steps requested in the memorandum have the 
potential to address the factors EPA and state officials identified as 
contributing to unreliable data quality. According to the director, 
however, the states currently are not required to use the data 
management tools that EPA has developed as a condition of primacy or 
their PWSS grant agreements, and many have chosen not to do so. In 
response to our survey, EPA and state drinking water officials 
generally said these tools would help improve data quality but noted 
barriers they believe have prevented more widespread use of them. For 
example, the most common theme among the survey respondents' 
suggestions for increasing the use of the SDWIS/State and electronic 
data verification tools was to address their quality, complexity, and 
ease of use. The Director of the Office of Ground Water and Drinking 
Water also asked the regional water management directors in her 
memorandum to increase their oversight of state programs. For example, 
she asked the regions to include language in future grant agreements 
indicating that the state, among other things, must make compliance 
determinations that are consistent with applicable drinking water 
regulations. However, the director also indicated that her office does 
not have the authority to require the regions to take the actions 
requested in the memorandum, and could not document the extent to 
which the regions had done so. EPA's plan to develop a next generation 
of SDWIS and to increase its access to state data might help the 
agency ensure that it receives higher quality violations and 
enforcement data from the states. However, it is uncertain if and when 
the new system or increased access to data will be available. In the 
meantime, further efforts to overcome the barriers to implementation 
of the 2008 action plan and the director's 2009 memorandum are needed 
to improve state data. 

Recommendations for Executive Action: 

To improve EPA's ability to oversee the states' implementation of the 
Safe Drinking Water Act and provide Congress and the public with more 
complete and accurate information on compliance, we recommend that the 
Administrator of EPA take the following four actions: 

* Resume data verification audits to routinely evaluate the quality of 
selected drinking water data on health-based and monitoring violations 
that the states provide to EPA. These audits should also evaluate the 
quality of data on the enforcement actions that states and other 
primacy agencies have taken to correct violations. 

* Work with the states to establish a goal, or goals, for the 
completeness and accuracy of data on monitoring violations. In setting 
these goals, EPA may want to consider whether certain types of 
monitoring violations merit specific targets. For example, the agency 
may decide that a goal for the states to completely and accurately 
report when required monitoring was not done should differ from a goal 
for reporting when monitoring was done but not reported on time. 

* Consider whether EPA's performance measures for community water 
systems could be constructed to more clearly communicate the aggregate 
public health risk posed by these systems' noncompliance with SDWA and 
progress in having those systems return to compliance in a timely 
manner. 

* Work with the EPA regions and states to assess the progress made in 
implementing the steps called for by the 2008 action plan and the 
Director of the Office of Ground Water and Drinking Water's 2009 
memorandum; identify the barriers that have prevented more widespread 
implementation of the action plan and memorandum; and develop and 
publish a strategy for overcoming those barriers. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to EPA for review and comment. The 
agency provided written comments, which are reproduced in appendix 
III. EPA partially agreed with two of our recommendations, disagreed 
with one, and neither agreed nor disagreed with another. Our responses 
to EPA's comments on our recommendations follow, and our responses to 
EPA's attachment of substantive comments are in appendix III. EPA also 
provided technical comments that we have incorporated as appropriate. 

In its overall comments, EPA said that it recognizes the importance 
and value of high quality data to complement the activities that 
comprise its oversight of primacy agencies. EPA also acknowledged that 
GAO found data quality problems similar to those previously found by 
the agency during data verification audits and that underreporting 
violations data and enforcement actions may limit the public's full 
knowledge of the status of public water system compliance. EPA noted 
that it has implemented a number of activities to improve data quality 
and its ability to oversee the drinking water program. We agree that 
EPA has taken steps to improve data quality and describe many of them 
in our report. 

EPA also noted that complete and accurate data are important in order 
to effectively target enforcement to those systems with the most 
serious compliance problems. The agency added that its 2009 
Enforcement Targeting Tool provides an incentive to the states to keep 
their enforcement data current to ensure that the tool yields accurate 
scores. We agree that the tool underscores the importance to the 
states of keeping enforcement data current. However, the scores 
generated by the tool will also be incorrect if data on the existence 
of violations are incomplete or inaccurate. We believe that whereas 
the use of the tool provides an incentive to the states to improve the 
accuracy of their enforcement data, it does not necessarily provide 
them an incentive to improve the accuracy of their violations data. 

EPA partially agreed with our first recommendation that it resume data 
verification audits of violations and enforcement actions. The agency 
stated that it has found that data verification audits provide 
valuable information on data completeness and accuracy and that it 
plans to conduct six to eight audits during calendar year 2011. 
However, EPA did not commit to conducting data verification audits 
beyond 2011. Instead, EPA said that until the next generation of SDWIS 
is deployed, thus enabling the agency to view compliance monitoring 
data and compliance determinations directly, it will consider using 
data verification audits to evaluate data quality. EPA did not comment 
on how it would evaluate data on enforcement actions taken to correct 
violations. We understand that the next generation of SDWIS may enable 
EPA to more directly monitor water systems data and oversee the 
states' compliance determinations. We note that the Director of the 
Office of Ground Water and Drinking Water said that the new system 
would not be available until 2014, depending on the availability of 
funding. We continue to believe that EPA should commit to, not merely 
consider, conducting data verification audits until the new system is 
available, and that those audits should also evaluate the completeness 
and accuracy of enforcement data. 

EPA did not clearly indicate its agreement or disagreement with our 
second recommendation that it work with the states to establish a 
goal, or goals, for the completeness and accuracy of data on 
monitoring violations. The agency stated that it appreciates the need 
for improved data quality for those types of violations. However, EPA 
neither indicated that it would adopt a goal nor offered any reasons 
for why a goal--such as the one it has for the quality of data on 
health-based violations--would be inappropriate. Instead, the agency 
suggested that along with technology enhancements as part of the next 
generation of SDWIS, (1) it will consider changes to its approach to 
reporting violations data, and (2) will explore the possibility of 
revising the Enforcement Targeting Tool, which could improve its 
oversight capabilities. We are not able to evaluate these changes 
given their speculative nature, and it is not clear how they might be 
relevant to achieving a higher degree of data quality. EPA also stated 
that the regions' annual incorporation of data quality in state grants 
and workplans will improve EPA's oversight capabilities. We agree that 
increased emphasis from the regions is necessary and could lead to 
improved data quality. However, we continue to believe that setting a 
goal for the quality of data on monitoring violations would emphasize 
its importance and encourage the states to make and report correct 
compliance determinations. 

EPA disagreed with our third recommendation that it consider whether 
its performance measures could be constructed to more clearly 
communicate the aggregate public health risk posed by systems' 
noncompliance with SDWA. The agency noted that its program guidance 
currently includes a measure that attempts to address the duration (in 
"person months") of time consumers may be exposed to health-based 
violations. We describe this measure in the background section of our 
report. However, we believe this measure has the same limitation as 
other EPA strategic targets, in that it does not distinguish between 
water systems with multiple health-based violations in a particular 
month and those systems with a single violation in that month. EPA 
also stated that it uses a variety of tools that may convey 
information on risks associated with noncompliance and show progress 
toward returning systems to compliance better than a new performance 
measure would. Among the tools EPA identified is a Web site containing 
detailed information about the violations for individual water 
systems. EPA also said that it recently posted drinking water data to 
its Enforcement and Compliance History Online tool. Similarly, EPA 
said water systems directly convey to their customers information on 
public health risks associated with violations through Public 
Notifications and Consumer Confidence Reports. We acknowledge these 
tools provide the public with details on the violations that states 
and water systems have reported for individual water systems. However, 
our recommendation encourages EPA to consider changes to its 
performance measures to provide Congress and the public a clearer 
understanding of noncompliance with SDWA at a national level and not 
elicit more information about the performance of individual water 
systems. We continue to believe that it is important for EPA to 
develop a national performance measure that helps gauge EPA's overall 
management of the drinking water program. 

Regarding our fourth recommendation that EPA work with the regions and 
states to assess progress in, and develop a strategy for overcoming 
barriers to implementing the 2008 action plan and the Director's 2009 
memorandum, the agency expressed partial agreement by saying it will 
continue to assess the progress of improving data quality. EPA also 
noted that since these documents were issued, the office has worked 
with state and regional staff to understand data quality challenges 
and opportunities for improvement. Specifically, EPA commented that 
all of the regions have incorporated data quality into their 
discussions with states and that data quality has been incorporated 
into grant agreements or state workplans. We agree that those actions 
would signal progress toward implementing the Director's memorandum. 
However, we were unable to verify these statements because EPA told us 
that documentation was not available. EPA told us it made the 
statements on the basis of e-mail communications and discussions 
between the managers and staff in the Office of Ground Water and 
Drinking Water and regional management and staff. EPA also stated that 
the Data Quality Work Group formed by the Director developed a list of 
recommendations to address underlying data quality problems and that 
it will continue to evaluate those recommendations. As we note in the 
report, the recommendations were in draft, and EPA's comments did not 
provide evidence that they have been adopted. EPA identified other 
future actions that it believes will lead to improved data quality. 
For example, EPA emphasized the effect that a next generation of SDWIS 
could have on improving data quality. We do not disagree that these 
actions, if taken, may contribute to improved data quality. However, 
we point out that EPA has already developed tools that states could 
use to improve the quality of their data on violations but that those 
tools have not been widely used. There is no requirement that the 
states use the next generation of SDWIS, if and when it is available. 
Furthermore, EPA did not directly address the recommendation to 
identify the barriers that have prevented more widespread 
implementation of the action plan and memorandum and develop and 
publish a strategy for overcoming those barriers. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution for 30 days 
from the report date. At that time, we will send copies to the 
appropriate congressional committees, the Administrator of EPA, and 
other interested parties. In addition, the report will be available at 
no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. 

If you or your staff members have any questions about this report, 
please contact me at (202) 512-3841 or trimbled@gao.gov. Contact 
points for our Offices of Congressional Relations and Public Affairs 
may be found on the last page of this report. GAO staff who made major 
contributions to this report are listed in appendix IV. 

Signed by: 

David C. Trimble: 
Acting Director: 
Natural Resources and Environment: 

[End of section] 

Appendix I: Scope and Methodology: 

To examine the quality of the Safe Drinking Water Information System/ 
Federal (SDWIS/Fed) data that the Environmental Protection Agency 
(EPA) uses to measure community water systems' compliance with the 
health-based and monitoring requirements in the Safe Drinking Water 
Act (SDWA), we examined the results of audits EPA conducted from 1996 
through 2009 in which it assessed--for a sample of states--the 
completeness and accuracy of violations data those states submitted to 
SDWIS/Fed. We evaluated the methods that EPA used to conduct those 
audits to test the methods' validity and determined that, while 
limited, these methods were sufficiently reliable for the purposes of 
our review. We also conducted our own analysis of EPA's audit data 
from 2007 through 2009 in order to arrive at estimates of the quality 
of the data that states reported to SDWIS/Fed.[Footnote 58] We focused 
our analysis on community water systems only. 

The sample design for the EPA data verification audits consists of a 
nonprobability sample of primacy agencies within a given year and a 
probability sample of community water systems within each selected 
primacy agency. Based on our review of the sample design, we 
determined that it is not appropriate for our purposes to make 
quantitative statements or inferences about the entire nation from the 
selected primacy agencies or comparisons with sampled primacy agency 
data quality results from the previous years. As such, we only 
generated estimates to the states audited within a given year. Table 1 
provides a description of the number of primacy agencies that were 
included in the sample for each year. 

Table 1: Primacy Agencies Covered by EPA's Data Verification Audits, 
by Year: 

2007: 
Arkansas; 
Arizona; 
Georgia; 
Illinois; 
Kansas; 
Maryland; 
Minnesota; 
North Dakota; 
Navajo Nation; 
Nevada; 
Puerto Rico; 
EPA Region 1; 
EPA Region 4; 
EPA Region 7; 
Rhode Island; 
South Carolina; 
Utah; 
Virginia; 
Washington. 

2008: 
Alaska; 
Alabama; 
Iowa; 
Kentucky; 
Louisiana; 
Massachusetts; 
Maine; 
Mississippi; 
Montana; 
New York; 
Ohio; 
Pennsylvania; 
EPA Region 2; 
EPA Region 9; 
Texas; 
Wisconsin; 
West Virginia. 

2009: 
California; 
Connecticut; 
Delaware; 
Florida; 
Hawaii; 
Indiana; 
Michigan; 
North Carolina; 
Nebraska; 
New Jersey; 
New Mexico; 
Oregon; 
Tennessee; 
Vermont. 

Source: EPA. 

[End of table] 

Classification of Violation Types: 

We classified violations into one of two types: health-based 
violations and monitoring violations. By definition, monitoring 
violations include "other" violations such as Public Notification and 
Consumer Confidence Report violations. We reviewed and decided to use 
definitions of violation types provided by EPA to make these 
classifications. We included lead and copper treatment technology 
violations as health-based violations in our analysis. 

Data Quality Estimates: 

We defined three separate measures of data quality: accuracy, 
completeness, and overall quality (a combination of accuracy and 
completeness). These measures are consistent with the measures used by 
EPA in previous years. We reviewed and decided to use definitions 
provided by EPA to calculate these measures. 

To estimate the percentage of violations that were accurate and 
complete, we first created a data set with one observation per 
violation, and then we used a procedure in statistical software that 
appropriately accounts for the stratified cluster sample design. 
[Footnote 59] We calculated point estimates and 95 percent confidence 
intervals. We did not report estimates that have margins of error that 
exceed plus or minus 20 percentage points at the 95 percent confidence 
level. 

To determine whether there was a relationship between monitoring 
violations and health-based violations, we used audit data to estimate 
regression models controlling for size, source and administrative 
control. We tested various specifications of three types of 
statistical models to ensure that the significance and magnitude of 
our estimates were consistent across statistical models.[Footnote 60] 

Estimates of Compliance Determination and Data Flow Errors: 

We conducted a subpopulation analysis to estimate the percentage of 
incomplete (not reported) violations that were either compliance 
determination or data flow reporting discrepancies. We calculated 
point estimates and 95 percent confidence intervals. We did not report 
estimates that have margins of error that exceed plus or minus 20 
percentage points at the 95 percent confidence level. 

The second component of our first objective was to examine the quality 
of SDWIS/Fed data on the status of the states' and EPA regions' 
enforcement actions. Because EPA's recent audits of state data did not 
assess the completeness and accuracy of enforcement data in SDWIS/Fed, 
we examined EPA's national SDWIS/Fed data from 2005 through 2009 to 
determine the percentage of violations the states have identified as 
either resolved (known as returned to compliance), addressed through 
an enforcement action but not yet resolved, or not addressed. We then 
interviewed EPA officials to obtain their views on the completeness 
and accuracy of those data, and also analyzed relevant comments from 
survey respondents. The survey was of EPA and state drinking water 
officials to obtain their views on a range of issues related to data 
quality. (See appendix II for more details on our survey methodology). 

To identify factors that have affected data quality, we analyzed 41 
survey responses representing the views of all 44 members of three 
joint EPA-state work groups that were created to address various 
aspects of data management. 

To examine the ways in which SDWIS data quality could affect EPA's 
management of the Public Water System Supervision (PWSS) program, we 
examined the importance of data quality for two aspects of EPA's 
management of the PWSS program. First, we examined the potential 
impact data quality could have on EPA's Enforcement Response Policy. 
This policy uses a targeting tool that assigns scores to community 
water systems that are a high priority for enforcement action because 
of unresolved violations. To demonstrate the effect that underreported 
health and monitoring violations can have on the Enforcement Targeting 
Tool, we calculated two scores for each of the approximately 1,200 
water systems audited by EPA in 2007, 2008, and 2009. One score was 
based on violations found in the data verification audits, and the 
second score was based on violations found in SDWIS/Fed. We then 
subtracted the two scores for each system to obtain a point 
difference, a result that we used to illustrate the impact of 
incomplete data on the scoring process. We used the same methodology 
EPA uses to create the enforcement score: We assigned point values to 
unresolved violations (acute health violations are worth 10 points, 
nonacute health violations and some monitoring violations are worth 5 
points, and 1 point for all other monitoring and reporting violations) 
and then added these points together to produce an overall score for 
each system. However, the enforcement scores we calculated cannot be 
considered a water system's actual score for three reasons: 

1. EPA's targeting tool scores 5 years of violation data, whereas the 
audits that EPA conducted reviewed state files to identify violations 
that had occurred in shorter periods of time. Those time periods 
typically ranged from 1 to 3 years, depending on the drinking water 
regulation. 

2. Due to limited data in EPA's audit database, our enforcement scores 
only include underreported violation discrepancies and do not include 
any discrepancies related to accuracy. However, underreported 
violations accounted for nearly 97 percent of the discrepancies. 

3. EPA's enforcement score includes an additional penalty that is tied 
to the year of the oldest unaddressed violation. For instance, if a 
violation is 5 years old, EPA adds an additional 5 points to the 
score. Because our analysis considered violations from a shorter 
period of time, we could not duplicate this additional penalty. 

While these three limitations prevent us from duplicating EPA's exact 
targeting tool, our analysis presents a conservative estimate of the 
effect that poor data quality has on the enforcement scoring process. 
It is likely that additional years of underreported violations, plus 
other enforcement penalties, would reveal further distortions of the 
scoring process. 

In addition, we examined survey respondents' views on the impact that 
data quality may have on implementation of the Enforcement Response 
Policy. We also interviewed EPA and state officials to obtain their 
views on the matter. 

Second, we examined the impact data quality could have on the agency's 
ability to inform the public and Congress about water systems' 
compliance with drinking water standards. In particular, we examined 
whether EPA's claims about community water systems' performance 
relative to Government Performance and Reporting Act (GPRA) goals were 
affected by the use of incomplete and inaccurate SDWIS/Fed data. Two 
key GPRA measures are the percentage of community water systems and 
the percentage of population served by those systems that meet all 
health-based drinking water standards (i.e., had no health-based 
violations) in a fiscal year. We used EPA's data verification audit 
data to estimate the percentage of community water systems that met 
all health-based drinking water standards for selected states within 
each audit year. Based on our review of EPA's data verification audit 
sample design, we determined that the sample was not designed to 
produce reliable estimates of the percentage of the population served 
by systems with health-based violations. Therefore, we focused our 
analysis on the percentage of community water systems that met all 
health-based drinking water standards. 

To estimate the percentage of community water systems that met all 
health-based drinking water standards from the data verification audit 
data, we counted the number of health-based violations for each 
community water system in the sample and calculated point estimates 
and 95 percent confidence intervals of the percentage of systems that 
met all health-based drinking water standards after accounting for the 
results of the data validation audits. 

To examine the actions EPA and the states have been taking to improve 
the quality of data in SDWIS/Fed, we interviewed EPA officials and 
obtained and reviewed documentation on steps the agency has taken, or 
is considering, to modernize SDWIS and improve data quality. We also 
examined survey respondents' views on steps that EPA and the states 
could take to address data quality--including the adoption of 
particular data management tools--and ways in which the three EPA-
state work groups could be more effective. For more information on our 
survey and content analysis, see appendix II. 

We conducted this performance audit from February 2010 through June 
2011 in accordance with generally accepted auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient and appropriate evidence to provide a reasonable basis for 
our findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Survey Methodology and Analysis: 

Selection of Survey Respondents: 

To obtain the views of knowledgeable EPA and state drinking water 
officials about SDWIS data management and data quality, we surveyed 
the members of three EPA-state work groups that were formed to address 
various aspects of drinking water data: the Data Management Steering 
Committee, the Data Technical Advisory Committee, and the Data Quality 
Work Group. EPA and the Association of State Drinking Water 
Administrators provided us with the names and e-mail addresses of the 
46 members of these groups. The work group members come from EPA 
headquarters, EPA regions, states, and the Association of State 
Drinking Water Administrators. The Data Management Steering Committee 
had 18 members, the Data Technical Advisory Committee had 15 members, 
and the Data Quality Work Group had 25 members. Nine officials served 
on more than one of the committees. 

Survey Design and Pretesting: 

Our survey asked a range of questions related to the drinking water 
violations data that states and other primacy agencies provide to EPA. 
We asked the respondents to comment on the factors that have 
contributed to data errors, the steps that should be taken to correct 
those errors, the impact that data errors could have on EPA's 
Enforcement Response Policy, and various data management tools. We 
also asked the respondents to evaluate the effectiveness of the work 
group or groups on which they served. 

The practical difficulties of conducting any survey may introduce 
errors, commonly referred to as nonsampling errors. For example, 
respondents may have difficulty in interpreting a particular question 
or may lack information necessary to provide valid and reliable 
responses. In order to minimize these errors, we conducted pretests of 
the draft survey with one EPA headquarters official, one EPA regional 
official, and one state official by telephone. The Chief of the 
Infrastructure Branch of EPA's Office of Ground Water and Drinking 
Water suggested that we conduct the pretest with those individuals. 
During these pretests, we checked whether (1) questions were clear and 
unambiguous, (2) terminology was used correctly, (3) the questionnaire 
did not place undue burden on respondents, (4) the information could 
feasibly be obtained, and (5) the survey was comprehensive and 
unbiased. In addition, the survey was peer reviewed by a GAO senior 
survey methodologist. We made changes to the content and the format of 
the survey based on the feedback we received. 

Survey Administration: 

We administered our survey in August and September of 2010. We first 
phoned each work group member to alert them to our plan to send the 
survey and to request their participation. Through those phone calls, 
we learned that one of the work group members had retired and another 
had transferred to a different position and was no longer a member of 
a work group. As a result, our survey population decreased to 44. 

Prior to fielding the survey, we sent an e-mail to each member of the 
work groups to further explain its purpose. We notified work group 
members electronically when the survey was available, and sent e-mail 
reminders prior to our requested deadline of September 13, 2010. We 
also made phone calls to several survey recipients during the 
extension period to request their participation. In total we received 
41 completed surveys. However, the three work group members from the 
Association of State Drinking Water Administrators collaborated to 
prepare one response and two EPA regional officials collaborated to 
prepare one response. Therefore, the 41 completed surveys represent 
the views of all 44 members of the survey population. 

Data Analysis: 

The survey contained closed-ended questions that asked respondents to 
select from a finite number of options. For example, some questions 
asked respondents to select "Yes," "No," or "Don't Know." Others asked 
respondents to select from a list of factors that may have contributed 
to drinking water violation data errors. Our analysis of the responses 
to these questions simply involved counting the number of responses 
for each option. In the report, there are instances in which we 
identify all of the responses and other instances in which we identify 
the most common response. 

We also asked respondents to evaluate the effectiveness of certain 
data management tools or the EPA-state work groups. For these 
questions, we offered the respondents a four-point range of answers: 
very effective, moderately effective, slightly effective, and not 
effective. Respondents could also answer "Don't Know" to these 
questions. In the report, we sometimes identified the most common 
response while in other instances we combined the number of 
"moderately effective" and "slightly effective" responses because each 
was relatively common. 

Several survey questions asked for opinions on the creation of a goal 
for the percentage of monitoring violations that are completely and 
accurately reported to SDWIS/Fed. We asked those respondents who 
supported the idea of having such a goal to state what they think the 
percentage should be. We summed those percentages and divided by the 
number of respondents who answered that question to arrive at an 
average. 

The responses to the closed-ended questions are provided in this 
appendix. 

Content Analysis: 

The survey also contained open-ended questions that asked respondents 
to provide a narrative response. In order to succinctly summarize the 
open-ended responses, we performed a content analysis in which we 
grouped the responses into a coding structure that represented common 
themes. We decided that the responses to each open-ended question 
would have a coding structure with two dimensions. To explain this, it 
is useful to discuss the link between closed-ended and open-ended 
questions. For example, one closed-ended question asked the 
respondents to select a factor--such as training by EPA or the states 
or guidance from EPA--that they believe has contributed to compliance 
determination errors. The subsequent open-ended question asked them to 
elaborate on why they thought the factor or factors they selected have 
contributed to compliance determination errors. Note that the 
respondents did not necessarily elaborate on each of the factors they 
selected in the prior question. As part of our content analysis, we 
sought to first identify the first dimension code for the response. In 
this example, the first dimension codes mirrored the factors (e.g., 
training or guidance) that the respondent selected to write about. A 
second dimension code provided a more detailed description of what the 
respondent said about the first order code. For example, second 
dimension codes for that question included amount or quality, timing, 
and targeting. 

A team of three GAO analysts jointly reviewed several completed 
surveys to develop an initial draft of a structure for coding the open-
ended responses. To further identify meaningful first and second 
dimension codes for the coding structure, the three GAO analysts 
independently reviewed the open-ended responses for four completed 
surveys. Each analyst made a judgment about appropriate codes that 
described the themes in the open-ended responses. The analysts 
compared their decisions and reconciled any disagreements regarding 
appropriate codes by refining the criteria used to categorize the 
responses. 

After the team agreed upon the coding structure, it continued its 
analysis of the responses to the closed-ended questions. The three GAO 
analysts were each assigned to independently review the responses to 
specific sets of questions. For example, Analyst A and Analyst B 
independently reviewed and coded the open-ended answers to questions 1 
through 4. Analysts A and B then compared their coding decisions and 
reconciled any disagreements. If they could not reconcile a 
disagreement, Analyst C was consulted to achieve agreement. The three 
analysts rotated assignments so that each performed the role of 
"tiebreaker" when the other two could not agree on a coding decision. 

Respondents' Answers to Closed-Ended Survey Questions: 

Our survey of EPA and state drinking water officials contained 
numerous closed-ended questions about various data management issues. 
Table 2 presents those questions and the respondents' answers. 

Table 2: Survey Respondents' Answers to Closed-Ended Questions: 

1. According to EPA's Audits, most of the data discrepancies relate to 
incorrect compliance determinations. Do you think any of the following 
factors contribute to incorrect compliance determinations? 

Information System Structure: 
Yes: 22; 
No: 17; 
Don't Know: 2. 

Training: 
Yes: 35; 
No: 4; 
Don't Know: 2. 

Funding: 
Yes: 26; 
No: 12; 
Don't Know: 3. 

Staffing: 
Yes: 34; 
No: 5; 
Don't Know: 2. 

Guidance: 
Yes: 25; 
No: 13; 
Don't Know: 3. 

Other:
Yes: 21; 
No: 7; 
Don't Know: 2. 

5. According to EPA's Audits, other data discrepancies are related to 
data flow problems. In such cases, a state identified a violation that 
occurred but did not report the violation to SDWIS/Fed. Do you think 
any of the following factors contribute to data flow errors? 

Information System Structure: 
Yes: 28; 
No: 10; 
Don't Know: 2. 

Training: 
Yes: 24; 
No: 12; 
Don't Know: 4. 

Funding: 
Yes: 18; 
No: 13; 
Don't Know: 7. 

Staffing: 
Yes: 25; 
No: 10; 
Don't Know: 4. 

Guidance: 
Yes: 20; 
No: 15; 
Don't Know: 4. 

Other: 
Yes: 10; 
No: 11; 
Don't Know: 4. 

9. Do you think that more widespread use of SDWIS/STATE modules would 
improve data quality? 

Yes: 26; 
No: 5; 
Don't Know: 10. 

10. Do you think any of the following factors preventing more states 
from fully using SDWIS/STATE modules? 

Information System Structure: 
Yes: 31; 
No: 4; 
Don't Know: 6. 

Training: 
Yes: 25; 
No: 10; 
Don't Know: 6. 

Funding: 
Yes: 24; 
No: 9; 
Don't Know: 8. 

Staffing: 
Yes: 24; 
No: 9; 
Don't Know: 8. 

Guidance: 
Yes: 14; 
No: 16; 
Don't Know: 10. 

Other: 
Yes: 12; 
No: 9; 
Don't Know: 3. 

14. Do you think that more widespread use of electronic data reporting 
from labs to states would improve data quality? 

Yes: 39; 
No: 1; 
Don't Know: 1. 

15. Do you think any of the following factors prevent more states from 
requiring electronic reporting from labs to states? 

Capability of Certified Labs: 
Yes: 29; 
No: 6; 
Don't Know: 6. 

State Legal Authority: 
Yes: 28;
No: 4; 
Don't Know: 9. 

Information System Structure: 
Yes: 15: 
No: 18; 
Don't Know: 7. 

Training: 
Yes: 17; 
No: 16; 
Don't Know: 8. 

Funding: 
Yes: 23; 
No: 11; 
Don't Know: 6. 

Staffing: 
Yes: 20; 
No: 14; 
Don't Know: 6. 

Guidance: 
Yes: 15; 
No: 19; 
Don't Know: 6. 

Other: 
Yes: 12; 
No: 11; 
Don't Know: 3. 

19. For states that are using SDWIS/State, do you think that more 
widespread use of electronic data verification would improve data 
quality? 

Yes: 18; 
No: 6; 
Don't Know: 15. 

20. Do you think any of the following factors prevent more states from 
using the eDV tool? 

Information System Structure: 
Yes: 25; 
No: 6; 
Don't Know: 9. 

Training: 
Yes: 16; 
No: 13; 
Don't Know: 9. 

Funding: 
Yes: 13; 
No: 8; 
Don't Know: 15. 

Staffing: 
Yes: 19; 
No: 12; 
Don't Know: 9. 

Guidance: 
Yes: 12; 
No: 15; 
Don't Know: 11. 

Other: 
Yes: 12; 
No: 8; 
Don't Know: 6. 

25. Do you think EPA should establish a data quality goal for the 
percentage of monitoring and reporting violations that are completely 
and accurately reported to SDWIS/Fed? 

Yes: 20; 
No: 14; 
Don't Know: 7. 

26. If yes, what do you think the goal should be? 

Lowest response: 41; 
Average, or mean response: 83.3; 
High response: 100. 

29. Do you think that limitations in the SDWIS/FED database will 
affect the usefulness of the Enforcement Targeting Tool with respect 
to EPA's oversight of enforcement priorities? 

Yes: 1. 22; 
No: 12; 
Don't Know: 7. 

30. In your opinion, how effective were the program review 
recommendations in improving data quality in primacy agencies? 

Very Effective: 8; 
Moderately Effective: 14; 
Slightly Effective: 12; 
Not Effective: 5; 
Don't Know: 2. 

31. Are you a member of the Data Management Steering Committee? 

Yes: 15; 
No: 26. 

32. How effective do you think the steering committee has been in 
helping EPA and the states reach the goal of improving data quality? 

Very Effective: 0; 
Moderately Effective 8; 
Slightly Effective: 4; 
Not Effective: 0; 
Don't Know: 4. 

34. Are you a member of the Data Quality Work Group? 

Yes: 23; 
No: 18. 

35. How effective do you think the work group has been in helping EPA 
and the states reach the goal of improving the accuracy and 
completeness of data on violations reported to SDWIS/Fed? 

Very Effective: 0; 
Moderately Effective: 3; 
Slightly Effective: 7; 
Not Effective: 9; 
Don't Know: 4. 

37. Are you a member of the Data Technical Advisory Committee? 

Yes: 16; 
No: 24. 

38. How effective do you think the Data Technical Advisory Committee 
has been in helping EPA and the states reach the goal of improving the 
accuracy and completeness of data on violations reported to SDWIS/Fed? 

Very Effective: 1; 
Moderately Effective: 14; 
Slightly Effective: 1; 
Not Effective: 0; 
Don't Know: 4. 

Source: GAO survey of members of the Data Management Steering 
Committee, Data Technical Advisory Committee, and Data Quality Work 
Group. 

[End of table] 

Coding Decisions for Responses to Open-Ended Survey Questions: 

Many of the questions in our survey of EPA and state drinking water 
officials asked for open-ended responses. We developed several "coding 
schemes" to categorize the responses to those questions. Our schemes 
generally had first and second dimension codes. For example, question 
2 asked the respondents to explain how factors they selected 
contributed to compliance determination errors. The first dimension 
codes for that question included information system structure, 
training, funding, staffing, and guidance, among others. The second 
dimension codes included amount; targeting; timeliness; quality, 
complexity, or ease of use; automation; and others. A response to 
question 2 might have included a comment that related to information 
system structures and, more specifically, a comment about the quality, 
complexity, or ease of use of information systems. In that situation, 
we would have coded the response as falling into those first and 
second dimension codes. Because some sets of questions generated 
answers that could be similarly coded, we used some coding schemes for 
multiple questions. On the other hand, we used some schemes for only 
one question. 

[End of section] 

Appendix III: Comments from the Environmental Protection Agency: 

Note: GAO comments supplementing those in the report text appear at 
the end of this appendix. 

United States Environmental Protection Agency: 
Office of Water: 
Washington, D.C. 20460: 
[hyperlink: http://www.epa.gov] 

June 8, 2011: 

Mr. David C. Trimble: 
Acting Director: 
Natural Resources and Environment: 
U.S. Government Accountability Office: 
Washington, D.C. 20548: 

Dear Mr. Trimble, 

Thank you for the opportunity to review and respond to GAO's draft 
report entitled, "Unreliable State Data Limit EPA's Ability to Target 
Enforcement Priorities and Communicate Water Systems' Performance." 
Ensuring clean and safe water for all Americans is an important part of
EPA's overall mission to protect human health and the environment. 
Federal and state drinking water regulations of public water systems 
require those systems to monitor and report data to enable government 
agencies to measure performance, address system technical challenges, 
take enforcement actions, and convey to the public information on 
their water systems. 

EPA recognizes the importance and value of high quality data to 
complement the various activities that comprise our oversight of 
primacy agencies. We have made improving the quality of our Safe 
Drinking Water Act (SDWA) data a priority and have worked for over a 
decade with our state partners to understand the many issues involved 
with data collection and management, and then to address the barriers 
to improved data quality. This has included our active participation 
and support of State-EPA data workgroups such as the Data Technical 
Advisory Committee and Data Management Steering Committee, as well as 
updates to our drinking water data system. 

EPA acknowledges that GAO found similar data quality problems to those 
previously found by the Agency during data verification audits. Under-
reporting of violations data and enforcement actions may limit the 
public's full knowledge of the status of public water system compliance.
However, state primacy agencies receive data directly from the public 
water systems and spend a significant amount of time working directly 
with noncompliant water systems with health-based violations to return 
them to compliance. 

EPA has implemented a number of activities over time to improve SDWA 
data quality and our ability to oversee the drinking water program. As 
part of its effort to implement the 2008 action plan to improve data 
quality, EPA formed a workgroup with the states to discuss standard 
operating procedures, best practices, and lessons learned which led to 
recommendations to help improve data quality nationally. These 
recommendations included on-demand training for state and EPA Regional 
staff to help address state turnover and travel limitations. We have 
developed and continue to develop various training tools including 
webinars, interactive computer trainings, and detailed guidance 
documents to assist primacy agencies in implementing the SDWA rules 
and ensuring complete and accurate data. 

Complete and accurate data are important in order to effectively 
target enforcement to those systems with the most serious compliance 
problems. EPA issued the 2009 SDWA Enforcement Response Policy and 
Enforcement Targeting Tool (ETT) to help prioritize unaddressed 
violations for state and EPA attention and bring systems back into 
compliance in a timely manner. Following the release, EPA requested 
that states and Regions update their data to show enforcement actions 
and return-to-compliance where appropriate. EPA believes that use of the
ETT creates an incentive for states to keep enforcement data current 
to ensure that the ETT scores accurately reflect the states' work to 
address violations. The scoring tool provides a list of public water 
systems ranked according to the extent of noncompliance that is 
unaddressed so that resources can be targeted where they will have the 
greatest impact. This is intended to help primacy agencies who are 
implementing the drinking water program and to help EPA fulfill its 
oversight role. 

GAO's Recommendations: 

In the draft report GAO offers four recommendations to improve EPA's 
ability to oversee the states' implementation of the SDWA, and provide 
Congress and the public with more complete and accurate information on 
public water system compliance and primacy agency enforcement actions. 

GAO Recommendation 1: Resume data verification audits to routinely 
evaluate the quality of selected drinking water data on health-based 
and monitoring violations that the states provide to EPA. These audits 
should also evaluate the data on the enforcement actions that states 
and other primacy agencies have taken to correct violations. 

EPA Response: EPA has found that the on-site data verification audits 
provide valuable information on data completeness and accuracy of 
compliance determinations. In Calendar Year 2011, the Office of Water 
will conduct data verification audits of six to eight primacy agencies 
in close coordination with our Regions. Until the next generation of 
EPA's Safe Drinking Water Information System (SDWIS Next Gen) is 
deployed and enables the Agency to view compliance monitoring data and 
compliance determinations directly, we will consider utilizing data 
verification audits to evaluate data quality. 

GAO Recommendation 2: Work with the states to establish a goal, or 
goals, for the completeness and accuracy of data on monitoring 
violations. In setting these goals, EPA may want to consider whether 
certain types of monitoring violations merit specific targets. For
example, the agency may decide that a goal for the states completely 
and accurately reporting when required monitoring was not done should 
differ from a goal for reporting when required monitoring was done but 
not reported on time. 

EPA Response: EPA appreciates the need for improved data quality of 
monitoring and reporting violations, and the important distinctions 
between different types of monitoring and reporting violations. Along 
with technology enhancements that will be part of the SDWIS Next Gen 
data system, EPA will be considering additional changes to the 
reporting of program data such as separately submitting monitoring 
violations and reporting violations to better understand the potential 
risk due to incomplete monitoring. Further, EPA will explore the 
possibility of revising the ETT scoring tool to add more weight to 
violations where a system failed to monitor compared to submitting 
monitoring results late. These activities, along with the Regions 
annually incorporating data quality into state grants and workplans, 
will further improve the Agency's oversight capabilities. 

GAO Recommendation 3: Consider whether EPA's performance measures for 
community water systems could be constructed to more clearly 
communicate the aggregate public health risk posed by the systems' 
noncompliance with SDWA and progress in having those systems return to 
compliance in a timely manner. 

EPA Response: EPA notes that the Office of Water's National Program 
Guidance currently includes a measure (referred to as "person months") 
which attempts to address the duration of time consumers may be 
exposed to health-based violations. In order to communicate risk 
associated with noncompliance and show progress toward returning 
systems to compliance, EPA utilizes a variety of tools that perhaps 
convey information better than a complex new measure. We have provided 
Excel-based GPRA tables on EPA's website, which contain detailed 
information about the violations for individual water systems. One of 
EPA's enforcement goals is to improve transparency by making facility 
compliance information available and accessible to the public. 
Recently, drinking water data has been posted to EPA's Enforcement and
Compliance History Online (ECHO) tool in a user-friendly format along 
with explanations of all terms and data elements. EPA will continue to 
improve this tool based on user feedback. EPA also expects that 
increased scrutiny of these data by the public may improve data 
completeness, accuracy and timeliness. 

As part of the Agency's Drinking Water Strategy, we continue to move 
forward in our effort for states and EPA to share compliance 
monitoring data and provide these data to the public. In November 
2010, EPA signed a memorandum of understanding with the Association of 
State Drinking Water Administrators (ASDWA), the Environmental Council 
of the States (ECOS), and the Association of State and Territorial 
Health Officials (ASTHO) in which the signatories agreed to work 
together to share drinking water monitoring data. 

In addition to EPA's efforts to share information on water system 
compliance with the public, water systems directly convey to their 
customers information on public health risks associated with 
violations through Public Notifications. Further, community water 
systems are required to provide water quality information to their 
customers through annual Consumer Confidence Reports, which include 
information on violations, the duration, and what the system is doing 
to correct the violation. 

GAO Recommendation 4: Work with EPA regions and states to assess the 
progress made in implementing the steps called for by the 2008 action 
plan and the Director of the Office of Ground Water and Drinking 
Water's 2009 memorandum; identify the barriers that have prevented 
more widespread implementation of the action plan and memorandum; and 
develop and publish a strategy for overcoming those barriers. 

EPA Response: The Office of Ground Water and Drinking Water will 
continue to assess the progress of improving data quality. Since the 
action plan and 2009 memo were issued, the Office of Ground Water and 
Drinking Water has worked with state and Regional staff to understand 
various data quality challenges and opportunities for improvement. All 
of the Regions have incorporated data quality into their discussions 
with states, and data quality has been incorporated into grant 
agreements or state workplans as described in the National Water
Program Guidance for FY2011 and 2012. 

In addition, the data quality workgroup formed by the Director 
developed a list of recommendations to address underlying data quality 
problems, such as regular training on rule implementation and 
compliance determinations for new staff. We will continue to follow-up 
on the Regional efforts and continue to evaluate the recommendations 
that were developed. in its regular discussions with primacy agencies' 
enforcement personnel regarding the status of priority systems and 
best practices for effective enforcement programs, EPA will include a 
discussion of ensuring data quality for enforcement actions. 

By updating our SDWIS data system and associated data management tools 
we expect to overcome some of the barriers identified by the workgroup 
and improve SDWA data quality. SDWIS Next Gen will offer a redesigned 
and easier-to-use interface that will reduce data entry issues; 
incorporate more data quality verification tools; and enable primacy 
agencies to use automated Quality Assurance/Quality Control processes 
on their data, thereby addressing many data quality issues at the 
front end of the data processing flow. Further, the integration of 
data flow components (e.g., electronic transmission of laboratory 
results to states) with SDWIS Next Gen will bring further data quality 
benefits by increasing automated rather than manual data entry. 

Although your report focused on community water systems, we are 
working to ensure that the data for all public water systems are 
complete and accurate. 

Please find attached EPA's suggested clarifications and a list of 
technical corrections to this draft report. 

Once again, thank you for the opportunity to respond to this draft 
report. 

Sincerely, 

Signed by: 

Nancy	K. Stoner: 
Acting Assistant Administrator: 

Attachments: 

[End of letter] 

Attachment A — Substantive Comments/Responses: 

Page 1 — "Americans rely on more than 51,000 community water systems 
for safe drinking water." EPA believes that it is important to clarify 
in this report that there are over 152,000 public water systems 
providing clean and safe drinking water to Americans. Not only do the 
National Primary Drinking Water Regulations establish MCL, treatment 
technique, monitoring, reporting, public notification, and 
recordkeeping requirements for these systems, PWSS primacy 
requirements dictate that states oversee this large universe of 
systems and manage the data and enforcement activities associated with 
their violations. This much larger universe and greater amount of data 
place a heavy responsibility on states at a time of declining 
resources, and can impact state data quality. [See comment 1] 

Page 9 second paragraph 1 should clarify that — Monitoring and 
reporting violations include only monitoring or reporting violations. 
Violations that are reported as other should not be included in with 
the monitoring and reporting violations. It would be beneficial to 
also explain what reporting violations are and how they are different 
than monitoring violations. [See comment 2] 

Page 13/28 — Discussion of Small Systems Performance Indicators — 
Please include the following information in a footnote to the report: 
In its National Water Program Guidance for Fiscal Year 2011, EPA added 
several new performance indicators for small community and 
nontransient, noncommunity water systems. This series of indicators 
were included, in addition to several related to the Drinking Water 
State Revolving Fund (DWSRF), in an effort to understand the current 
challenges facing small systems and what improvements may be seen over 
time as a result of EPA and state efforts to address the unique 
challenges faced by small water systems in reliably providing clean 
and safe water. These indicators are intended to be used together to 
convey an overall picture of small system capacity. As EPA continues 
to implement its efforts in partnership with the states, EPA will 
evaluate the utility of these indicators. [See comment 3] 

Page 14 — States' Data Did Not Really Reflect the Frequency of 
Monitoring Violations, Which Are a Predictor of Health-Based 
Violations: 

EPA is unsure of the basis for GAO's statement that monitoring 
violations are predictors of health-based violations. As reported to 
EPA, monitoring and reporting violations include monitoring that was 
conducted but not reported, not reported on time, and where no 
monitoring was conducted. A late report simply can mean that there was 
a problem with a laboratory delivering the results. EPA would 
appreciate the opportunity to evaluate GAO's data analysis. [See 
comment 4] 

It also appears that GAO has presented the health-based violations and 
monitoring/reporting violations for all NPDWRs, rather than separating 
out the Lead and Copper Rule (LCR) violations as EPA does. EPA 
requests that the data quality for the LCR data be presented separately.
[See comment 5] 

Page 30 — According to Survey Respondents, EPA's Audit Recommendations 
Contributed to Improving Data Quality, but the Agency has Discontinued 
Them, at Least Temporarily: 

As noted above, EPA intends to conduct six to eight data verification 
audits in Calendar Year 2011. EPA would appreciate GAO including in 
its report any specific suggestions made by those surveyed on how 
these audits can be improved to address issues associated with data 
quality. [See comment 6] 

Pages 38 & 39 — The Director of the Office of Ground Water and 
Drinking Water Also Called for Increased Regional Oversight and 
Accountability: 

EPA requests that GAO update its discussion to reflect the current 
status of work done by the Office of Ground Water and Drinking Water 
to assess regional responses and document the extent to which the 
Regions had complied with requests to discuss data quality with 
states. As discussed above, EPA notes that the Regions incorporate 
data quality actions, corrective action plans, and other measures in 
work plans and grant agreements, and in discussions with the states. 
We understand that all the Regions include data quality as an issue 
for discussions with their states and as part of their grant 
agreements or state work plans. [See comment 7] 

With respect to proposed measures for tracking regional oversight, EPA 
agrees that the Regions did not fully support new measures as proposed 
by Headquarters. However, a lack of additional performance measures 
has not prevented the Regions (or Headquarters) from continuing to 
work with the states to address data quality challenges. [See comment 
8] 

The following are GAO's comments responding to the comments in 
Appendix A of the Environmental Protection Agency's letter dated June 
8, 2011. 

GAO Comments: 

1. EPA commented that there are over 152,000 public water systems in 
the United States that are also subject to the requirements of the 
National Primary Drinking Water Regulations. We do not disagree, but 
did not modify the report in response to EPA's comment. Footnote 11 of 
the report describes the universe of public water systems. 

2. EPA said that violations that are reported as "other" should not be 
included with monitoring and reporting violations and that it would be 
beneficial to explain how reporting violations differ from monitoring 
violations. Our analysis is consistent with EPA's 2008 analysis of 
data quality which also combined "other" violations, such as 
violations of consumer confidence reporting and public notification 
requirements--with monitoring and reporting violations. Therefore, we 
did not modify our analysis or the report in response to EPA's 
comment. We also did not modify the background section to further 
explain the difference between monitoring and reporting violations 
because we provide examples of different violations in a subsequent 
section of the report. 

3. EPA requested that we provide a footnote in the report describing 
new performance indicators for small water systems. Specifically, EPA 
requested that we explain the agency's intent behind the indicators 
and its plan to evaluate their utility. We did not modify the report 
in response to this comment since we describe two of these indicators 
in the section of the report that addresses EPA's ability to monitor 
and report progress toward its strategic objective of reducing 
exposure to contaminants in drinking water. 

4. EPA said it was unsure of the basis for our statement that 
monitoring violations are predictors of health-based violations. EPA 
also noted some of the variations between monitoring and reporting 
violations and asked for more details regarding our analysis. Our 
statement was based on aggregate regression analyses (negative 
binomial and zero-inflated Poisson models) with limited controls. It 
does not take into account which type of monitoring and reporting 
violation occurred, and cannot differentiate between lack of 
monitoring and monitoring that was not reported or was delivered late. 
The regression was intended to illustrate the link between overall 
counts of monitoring and reporting violations and counts of health-
based violations, and does not provide insight into the nature of the 
link or the reasons that monitoring and reporting violations might 
condition the number of health-based violations. We realize the 
implications of our statement are limited, but believe the correlation 
between overall counts of monitoring violations and health-based 
violations offers useful insight. 

5. EPA requested that we present an analysis of the quality of 
violations data for the Lead and Copper Rule separately from other 
National Primary Drinking Water Regulations, as it has done. We 
acknowledge that there may be value in conducting data quality 
analyses for specific drinking water regulations, as EPA did, for 
example, in its 2008 report on data quality. That report showed that 
the data quality for the Lead and Copper Rule was lower than for other 
types of drinking water regulation. However, because the data 
verification audit data we analyzed was from a sample of community 
water systems from a sample of states, our results included margins of 
error. Analyzing data quality for particular drinking water 
regulations results in larger margins of error than analyzing date 
quality for all health-based violations. In light of that 
circumstance, we decided to conduct our analysis of all health-based 
violations. We did not modify our analysis or report in response to 
this comment. 

6. EPA commented that it intends to conduct six to eight data 
verification audits in calendar year 2011. We have modified the report 
to reflect that comment, but note that EPA's statement concerns audits 
it has yet to conduct. EPA also said that it would appreciate our 
including any specific suggestions made by survey respondents on how 
the data verification audits can be improved. We have added a footnote 
with examples of comments from survey respondents. 

7. EPA requested that we update the report to reflect the status of 
work done by the Office of Ground Water and Drinking Water to assess 
regional responses and document the extent to which the regions had 
complied with requests to discuss data quality with states. EPA went 
on to say it understands that all the regions include data quality as 
an issue for discussions with their states and as part of their grant 
agreements or state work plans. We have modified the report to include 
EPA's statements. However, we were not able to verify the accuracy of 
those statements because EPA did not have supporting documentation. 

8. EPA said that it agrees that the regions did not fully support new 
measures for tracking regional oversight suggested by headquarters but 
that the lack of those measures has not prevented the regions or 
headquarters from continuing to work with the states to address data 
quality challenges. We note that the proposed performance measures 
would assist the Office of Ground Water and Drinking Water to monitor 
the regions' oversight of the states, not the states' performance. We 
assume that EPA headquarters proposed these performance measures 
because it thought they would help encourage the regions to increase 
their oversight. Without them, EPA headquarters may find it more 
difficult to oversee the regions. We did not modify the report in 
response to this comment. 

[End of section] 

Appendix IV: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

David C. Trimble, (202) 512-3841 or trimbled@gao.gov: 

Staff Acknowledgments: 

In addition to the individual named above, Diane B. Raynes, Assistant 
Director; James Ashley; Elizabeth Beardsley; Mark Braza; Ross 
Campbell; Anna Maria Ortiz; Carla D. Rojas Paz; Kelly Rubin; Jerome 
Sandau; Jeffrey Sanders; Carol Herrnstadt Shulman; and Vasiliki 
Theodoropoulos made significant contributions to this report. Robert 
Alarapon, Mick Ray, and Lisa Vojta also made important contributions 
to this report. 

[End of section] 

Footnotes: 

[1] The Environmental Protection Agency defines a community water 
system as a public water system that supplies water to the same 
population year-round. A public water system provides water for human 
consumption through pipes or other constructed conveyances to at least 
15 service connections or serves an average of at least 25 people for 
at least 60 days a year. 

[2] CDC, "Surveillance for Waterborne Disease and Outbreaks Associated 
with Drinking Water and Water not Intended for Drinking--United 
States, 2005-2006," Morbidity and Mortality Weekly Report, 
Surveillance Summaries, September 12, 2008/57(SS09); 39-62. The 2008 
report is the CDC's most recent summary of waterborne disease 
outbreaks. 

[3] The requirements for state primacy are located at 40 CFR 142, 
Subpart B. 

[4] States that generally have primacy responsibility for existing 
drinking water regulations do not automatically have primacy for newly 
issued EPA regulations. To obtain primacy responsibility for a new 
standard (or, a newly issued EPA regulation), the state must apply for 
and receive EPA's determination that it has met specified 
requirements. In this report, we refer to data provided by EPA regions 
that are acting as primacy agencies as state data. 

[5] EPA Office of Inspector General, EPA Claims to Meet Drinking Water 
Goals Despite Persistent Data Quality Shortcomings, Washington, D.C., 
March 2004. 

[6] In 2007, EPA audited SDWIS/Fed data from 14 states, as well as 
Puerto Rico, the Navajo Nation, and 3 EPA regions. In 2008, EPA 
audited data from 15 states and 2 regions. And, in 2009, EPA audited 
data from 14 states. EPA's audits also examined a relatively small 
number of water systems that were under the jurisdiction of an EPA 
regional office rather than a state. When an EPA region has 
jurisdiction over a water system, it is responsible for maintaining 
compliance and enforcement data and for sending those data to 
SDWIS/Fed. Because only about 4.2 percent of the water systems that 
EPA audited in 2007 through 2009 were under regional office 
jurisdiction, for ease of presentation we refer to the audited data as 
state data. 

[7] The enforcement policy also applies to other types of public water 
systems. 

[8] Waterborne disease outbreaks are defined as events in which two or 
more persons are epidemiologically linked by exposure to water in a 
particular location, by time, and by characteristics of illness. 

[9] Colford Jr., John M., Sharon Roy, Michael J. Beach, Allen 
Hightower, Susan E. Shaw, and Timothy J. Wade, "A Review of Household 
Drinking Water Intervention Trials and an Approach to the Estimation 
of Endemic Waterborne Gastroenteritis in the United States," Journal 
of Water and Health, vol. 4, supplement 2, 2006. 

[10] A baby's skin may turn blue if nitrate concentration in his or 
her blood is high enough to impair oxygen delivery to skin tissue. 
Reduced oxygenation can have numerous adverse implications for a baby, 
even resulting in coma and death. 

[11] Other sources of federal funding are available to the states. For 
example, EPA's Drinking Water State Revolving Fund (DWSRF), supported 
by annual appropriations, provides funds to states, which, in turn, 
provide grants and loans to water systems for capital improvement 
projects and other expenses. In the American Recovery and Reinvestment 
Act of 2009 (Pub. L. No. 111-5), Congress provided additional funds 
for DWSRF. See GAO, Recovery Act: Preliminary Observations on the Use 
of Funds for Clean and Drinking Water Projects, [hyperlink, 
http://www.gao.gov/products/GAO-11-642T] (Washington, D.C.: May 4, 
2011). Other federal agencies, including the Department of Agriculture 
and the Department of Housing and Urban Development, also provide 
funding for drinking water projects. States are also authorized to use 
certain set asides from the DWSRF to fund state activities to 
implement the DWSRF and to administer the PWSS program. 

[12] Over 150,000 public water systems, including community water 
systems, are responsible for providing safe drinking water with 
oversight from EPA and the states. In addition to community water 
systems, more than 83,000 transient noncommunity water systems provide 
water in such places as gas stations or campgrounds where people do 
not remain for long periods. More than 18,000 nontransient 
noncommunity water systems regularly supply water to at least 25 of 
the same people at least 6 months per year in places other than their 
residences. Some examples of a nontransient system are schools, 
factories, office buildings, and hospitals that have their own water 
systems. These numbers do not include private water wells, which EPA 
does not have the authority to regulate. Approximately 15 percent of 
Americans rely on private drinking water supplies. 

[13] For example, EPA has set a numerical limit for arsenic in 
drinking water at 0.010 parts per million (10 parts per billion) to 
protect consumers served by public water systems from the effects of 
long-term, chronic exposure to arsenic. 

[14] 40 CFR §142.15 requires that the states submit quarterly reports 
of violations, enforcement actions, and new variances and exemptions 
to EPA in a format prescribed by the Administrator of EPA. By policy, 
EPA requires that the states submit the data in a format that can be 
placed into SDWIS/Fed. 

[15] According to an official in EPA's Office of Enforcement and 
Compliance Assurance, the agency is coordinating its implementation of 
the Enforcement Response Policy with its 1984 policy for the 
administration of environmental programs on Indian reservations. That 
policy, in conjunction with 2001 guidance on enforcement at tribal 
facilities, spells out EPA's procedures for taking enforcement actions 
at tribal facilities in order to protect human health and the 
environment. 

[16] The EPA regions may have complete primacy responsibility or 
partial responsibility in a state that has not yet obtained primacy 
for a particular drinking water regulation. 

[17] For example, a formal action could be an emergency administrative 
or compliance order, or a civil or criminal case for judicial referral 
or filing. 

[18] U.S. Environmental Protection Agency, 2006-2011 EPA Strategic 
Plan: Charting Our Course, September 30, 2006; and Fiscal Year 2011- 
2015 EPA Strategic Plan: Achieving Our Vision, September 30, 2010. The 
plans contain the goals "Clean and Safe Water" and "Protecting 
America's Waters," respectively, which include the objective to reduce 
exposure to contaminants in drinking water. 

[19] GAO, The Results Act: An Evaluator's Guide to Assessing Agency 
Annual Performance Plans, [hyperlink, 
http://www.gao.gov/products/GAO/GGD-10.1.20] (Washington, D.C.: April 
1998). 

[20] For this performance measure, EPA had a 2009 target that 87 
percent of the population in Indian Country would be served by 
community water systems that met all health-based standards. 

[21] Person months for each community water system are calculated as 
the number of months in the most recent four-quarter period in which 
health-based violations overlap, multiplied by the retail population 
served. 

[22] U.S. Environmental Protection Agency, Fiscal Year 2009 
Performance and Accountability Report (EPA-190-B-09-001), Washington, 
D.C., Nov. 16, 2009. 

[23] EPA conducted audits of state data in 2007, 2008, and 2009, but 
told us it had not reported on those audits as of March 2011. We used 
original data from EPA's audits to estimate the quality of the states' 
data. We chose not to report estimates from 2008 because the margin of 
error for that estimate exceeded plus or minus 20 percentage points. 

[24] In 2007, EPA audited 14 states, as well as Puerto Rico, the 
Navajo Nation, and 3 EPA regions. See table 1 in appendix I for a list 
of the audited entities. 

[25] As stated, EPA has established a goal of having 90 percent of 
health-based violations completely and accurately reported to SDWIS/ 
Fed. Based on our analysis of EPA's 2007 audits, the estimated 
percentage of health-based violations reported accurately or 
completely for states audited in 2007 had a 95 percent confidence 
interval that ranged from 65 percent to 96 percent. The 95 percent 
confidence interval for the estimate of the number of violations the 
states did not report or inaccurately reported ranges from about 165 
to about 921 violations. 

[26] The 95 percent confidence interval for the 2009 estimate ranged 
from 12 percent to 40 percent. The 95 percent confidence interval for 
the estimate of the number of violations the states did not report or 
inaccurately reported ranges from about 250 to about 1,306 violations. 

[27] EPA's 2000 report examined audit data from 1996 through 1998, the 
2004 report examined audit data from 1999 through 2001, and the 2008 
report examined audit data from 2002 through 2004. As of March 2011, 
EPA had not published a report summarizing the results of the audits 
it conducted in 2005 through 2009. 

[28] The 95 percent confidence interval for the estimated percentage 
of health-based violations completely and accurately reported ranged 
from about 37 percent to 61 percent. 

[29] EPA's 2008 report estimated the quality of the data states 
provided on violations of several categories of health-based 
standards: maximum contaminant levels for total coliform; maximum 
contaminant levels for other contaminants; treatment technologies for 
the surface water treatment rules; and treatment technologies for the 
lead and copper rule. EPA did not estimate the quality of the data 
concerning violations of these health-based standards by community 
water systems in particular. However, EPA reported that its estimates 
of the quality of data for violations of maximum contaminant levels 
and surface water treatment rule treatment technologies were not 
significantly different between community water systems and other 
types of public water systems. 

[30] The 95 percent confidence interval for the estimated percentage 
of health-based violations that were compliance determination errors 
in 2009 ranged from 78 percent to 100 percent and for data flow errors 
the range was from 0 to 22 percent. 

[31] The data on monitoring violations from the 2008 audits were 
acceptable for our analysis because the margin of error was less than 
plus or minus 20 percentage points. 

[32] The 95 percent confidence interval for the estimate of monitoring 
violations that states failed to completely and accurately report 
ranged from about 45,400 to about 63,800. The 95 percent confidence 
interval for the estimate of the percentage of violations that the 
states did not report or inaccurately reported ranged from about 81 
percent to about 87 percent. 

[33] Due to limitations in the data, we could not use the results of 
EPA's audits to estimate the total number of monitoring violations 
committed by community water systems nationwide. 

[34] Our analysis applies to systems covered by the data verification 
audits for 2007 through 2009 and is not necessarily representative of 
the relationship between monitoring violations and health-based 
violations across all community water systems. Although our 
statistical models controlled for system size, water source, and 
administrative control, they did not directly address complexities in 
the data collection and reporting processes, such as whether systems 
delayed or skipped monitoring in order to avoid having a recorded 
health-based violation. 

[35] EPA, Providing Safe Drinking Water in America: 2007/2008 National 
Public Water Systems Compliance Report, Office of Enforcement and 
Compliance Assurance (2201A) Washington, D.C., 20460, EPA Document 
Number 305R10001, June 3, 2010. 

[36] The 95 percent confidence interval for the estimated percentage 
of enforcement data completely and accurately reported ranged from 63 
percent to 83 percent. 

[37] Our survey did not ask respondents to identify the factors that 
have contributed to incomplete data on enforcement actions. However, 
some provided relevant information in response to other questions. 

[38] We could not duplicate the scoring calculation used by the 
Enforcement Targeting Tool because of limited data in EPA's audit 
database. However, we generated the score for purposes of comparison 
using a methodology similar to the one EPA used. Had we been able to 
more closely duplicate EPA's scoring calculation, it is likely that 
additional discrepancies would have further distorted the Enforcement 
Targeting Tool scoring process. For a full description of the process 
we followed, see appendix I. 

[39] Twelve respondents said that limitations in SDWIS/Fed would not 
affect the usefulness of the Enforcement Targeting Tool, while 7 said 
they did not know. 

[40] National Water Program Guidance: Fiscal Year 2011, Office of 
Water, Environmental Protection Agency, April 2010. The guidance 
describes the key actions needed to accomplish the public health and 
environmental goals in the EPA strategic plan. 

[41] We found that EPA's reports of progress toward the GPRA target 
for the percentage of community water systems meeting all health-based 
standards were not statistically different when using the more 
accurate data from EPA's verification audits. Given that the corrected 
data had more health-based violations, this was unexpected but 
illustrates the limitations of this measure. We believe the results 
reflect insensitivity in the GPRA measure as systems that have more 
than one violation count the same as those with multiple violations in 
estimating the percentage of systems that met all health-based 
standards. While this measure is "conservative" in that it counts 
every system with a violation, it does not communicate information 
regarding the number of systems with multiple violations or the 
relative severity of those violations. 

[42] Our survey contained several questions in which we asked the 
respondents to evaluate the effectiveness of a particular activity or 
group. We asked the respondents to select from very effective, 
moderately effective, slightly effective, not effective, and don't 
know. Two respondents to this question answered don't know. 

[43] For example, several respondents said the samples of audited 
water systems were too small. One respondent said the audit teams 
reviewed the compliance process for increasingly complex drinking 
water rules with "insufficient depth." Another suggested that EPA's 
audits use a scoring system similar to that of the Enforcement 
Targeting Tool to differentiate between violations. 

[44] Specifically, 8 of the 15 members of the steering committee who 
responded to the survey said it was moderately effective and 4 said it 
was slightly effective. The remaining 3 said they did not know. One of 
the 17 advisory committee members who responded said it was very 
effective, 13 said it was moderately effective, 1 said it was slightly 
effective, and 2 said they did not know. 

[45] Three members of the work group said it was moderately effective, 
7 said it was slightly effective, 8 said it was not effective, and 3 
said that they did not know. 

[46] The action plan, although titled "2006 Drinking Water Data 
Reliability Improvement Action Plan," was part of EPA's March 2008 
audit report. 

[47] EPA Office of Inspector General, EPA Claims to Meet Drinking 
Water Goals Despite Persistent Data Quality Shortcomings, Washington, 
D.C., March 2004. 

[48] The 10 that do not use SDWIS/State are: Florida, Massachusetts, 
Minnesota, New Hampshire, Pennsylvania, South Dakota, Washington, 
Wisconsin, the Navajo Nation, and American Samoa. In addition, EPA 
Regions 1 and 9 do not use SDWIS/State to manage their data on tribal 
programs. 

[49] The surface water treatment rules seek to prevent waterborne 
diseases caused by viruses, Legionella, and Giardia lamblia. These 
disease-causing microbes are present at varying concentrations in most 
surface waters. The rules require that water systems filter and 
disinfect water from surface water sources, as well as groundwater 
under the influence of surface water, to reduce the occurrence of 
unsafe levels of these microbes. The total coliform rule seeks to 
prevent the presence of pathogens that may harm human health. EPA 
considers total coliform to be a useful indicator of harmful 
pathogens. The absence of total coliform in the distribution system 
minimizes the likelihood that fecal pathogens are present. Thus, total 
coliform is used to determine the vulnerability of a system to fecal 
contamination. 

[50] We did not attempt to assess any relationship between data 
quality and the use of SDWIS/State because EPA's data verification 
audits were not designed to generate state-level estimates, and 
because we lacked information on multiple other factors that likely 
contribute to the quality of data from a particular state, such as 
funding, staffing levels, effectiveness of training, and management 
priorities. To adequately calculate the influence that using 
SDWIS/State has on data quality would require having a way to account 
for the presence of these, and possibly other, factors. 

[51] EPA adopted the Cross Media Electronic Reporting Rule in 2005, 
which amended various sections of its environmental regulations, to 
establish the framework for the agency to accept electronic reports 
from regulated entities as well as delegated state primacy agencies in 
satisfaction of certain document submission requirements in EPA's 
regulations. See 70 Fed. Reg. 59,848 (Oct. 13, 2005.) 

[52] The rule applies to: (a) regulated entities that submit reports 
and other documents to EPA under Title 40 of the Code of Federal 
Regulations, and (b) states, tribes, and local governments that are 
authorized to administer EPA programs under Title 40. 

[53] Under 40 CFR §1.49, the Assistant Administrator for Water is 
responsible for the evaluation of regional water activities. 

[54] The Administrator's specific goals related to data sharing are to 
(1) promote the use of advanced information technology to facilitate 
information and data exchange capability between states and EPA; (2) 
enhance compilation and analyses of public water system information to 
strengthen the review of potential drinking water public health 
concerns without additional information collection burden and requests 
on states; (3) share powerful data analysis tools with states to 
target public health issues, program oversight, compliance assistance, 
and enforcement to areas where risk to public health may be high; and 
(4) implement a range of interactive communication tools to enable 
states, the drinking water industry, and consumers to learn more about 
their drinking water and obtain timely information about the quality 
of drinking water and performance of drinking water systems. 

[55] In its comments on a draft of this report, EPA said that it is 
replacing SDWIS in response to federal government data system 
management requirements to review data systems and conduct 
alternatives analysis. In addition, EPA said it conducted an 
alternatives analysis in 2009 on SDWIS to determine whether it should 
maintain the system or replace it and decided to replace SDWIS with a 
new system that best meets cost and data quality objectives. 

[56] The three associations are the Environmental Council of the 
States, the Association of State & Territorial Health Officials, and 
the Association of State Drinking Water Administrators. 

[57] EPA can request and obtain such data from a state, but the 
current SDWIS/Fed does not include these data and hence they are not 
automatically shared electronically. See 40 C.F.R. §§ 142.14 
(generally specifying data records to be maintained by primacy 
states), 142.14(g) (requiring such records to be available to EPA upon 
request) (2011). 

[58] EPA's audits also examined a relatively small number of water 
systems that were under the jurisdiction of an EPA regional office 
rather than a state. When an EPA region has jurisdiction over a water 
system, it is responsible for maintaining compliance and enforcement 
data and for sending that data to SDWIS/Fed. Because only about 4.2 
percent of the water systems that EPA audited in 2007 through 2009 
were under regional office jurisdiction, for ease of presentation we 
refer to the audited data as state data. 

[59] We used PROC SURVEYFREQ in SAS. 

[60] The three models were logistic, negative binomial, and zero- 
inflated Poisson models. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: