This is the accessible text file for GAO report number GAO-06-595 
entitled 'Human Health Assessment: EPA Has Taken Steps to Strengthen 
Its Process, but Improvements Needed in Planning, Data Development, and 
Training' which was released on , 2006. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Chairman, Committee on Environment and Public Works, U.S. 
Senate: 

United States Government Accountability Office: 

GAO: 

May 2006: 

Human Health Risk Assessment: 

EPA Has Taken Steps to Strengthen Its Process, but Improvements Needed 
in Planning, Data Development, and Training: 

GAO-06-595: 

GAO Highlights: 

Highlights of GAO-06-595, a report to the Chairman, Committee on 
Environment and Public Works, U.S. Senate 

Why GAO Did This Study: 

Over 100,000 chemicals, pollutants, and toxic substances are used in 
the United States and regulated by the Environmental Protection Agency 
(EPA). EPA uses risk assessment to determine the health risk from 
exposure to these substances, collectively referred to as contaminants. 
In the last 12 years, independent reviewers have examined this process 
and made recommendations for how it could be improved. GAO was asked to 
(1) identify the significant recommendations that have been made to 
improve human health risk assessment; (2) describe what EPA has done to 
modify its human health risk assessment process; (3) determine the 
effects these past modifications have had on the preparation of risk 
assessments; and (4) identify any additional actions experts believe 
EPA could take to improve its process, and the barriers it would face 
in doing so. 

What GAO Found: 

Since 1994, independent reviewers recommended that EPA better plan its 
risk assessments. In doing so, they said EPA should better utilize 
scientific data it has and identify other data it needs on the 
potential adverse effects from exposure to contaminants, and prioritize 
and support research to meet those needs. Furthermore, reviewers 
recommended that EPA better evaluate the analytic tools it uses and 
employ more powerful tools when appropriate. Reviewers also recommended 
that EPA better analyze and characterize the sources of uncertainty in 
its risk assessments. Finally, they recommended that EPA enhance its 
analysis of variability in exposure to contaminants and in 
susceptibility to harm from exposure, and improve how it considers the 
effects of exposure to multiple contaminants and through many sources. 

EPA has strengthened its risk assessment process since 1994 and 
improvement efforts are ongoing. For example, EPA has increased 
planning for assessments and has initiated actions to develop missing 
or incomplete scientific data. EPA has also begun to embrace new 
methodologies, such as ones to predict how the body will react to a 
contaminant. Furthermore, EPA now uses a tiered approach to conducting 
uncertainty analysis, employing more sophisticated analysis as 
warranted. Finally, EPA has made progress in characterizing variability 
due to differences in both exposure and susceptibility of exposed 
individuals and has begun to take steps to address exposure to multiple 
contaminants and through multiple sources. 

According to EPA’s risk assessors, the modifications EPA has made have 
generally helped improve risk assessments. Many EPA risk assessors 
believe that agencywide guidance has helped them prepare risk 
assessments and have resulted in greater consistency across program 
offices. Furthermore, while most assessors report collaboration with 
internal and external entities is effective and has improved the 
quality of risk assessments, some said conflicting priorities and poor 
communication hindered collaboration among some EPA offices. Finally, 
while risk assessors said training has helped them gain skills and 
knowledge, over 70 percent said that more in-depth or relevant training 
would improve their risk assessment abilities. 

Experts identified additional actions EPA could take to further improve 
its risk assessment process, recognizing that it may face barriers in 
doing so. Experts said EPA could improve its planning process by better 
focusing on scientific data needs and involving stakeholders early to 
obtain their concurrence with EPA’s approach. Experts also said EPA 
could more thoroughly evaluate methods and models, transparently 
document its analytic choices, and enhance internal review. Finally, 
experts said EPA could provide additional training for risk assessors, 
managers, and stakeholders on the risk assessment process. Experts, 
however, said that the scientific complexity of risk assessment, the 
difficulty of obtaining and applying data, and a cultural resistance to 
deviating from established methods could act as obstacles to 
successfully making such changes. 

What GAO Recommends: 

GAO recommends that EPA enhance early planning of each risk assessment, 
identify and communicate data needs to the public and private research 
community, and support development and implementation of in-depth 
training for risk assessors and managers. EPA neither agreed nor 
disagreed with our findings and recommendations. However, the agency 
provided specific technical comments, which we incorporated as 
appropriate. 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-595]. 

To view the full product, including the scope and methodology, click on 
the link above. To view the results of GAO’s survey of EPA’s risk 
assessors, click [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-
637SP]. For more information, contact John Stephenson at (202) 512-6225 
or stephensonj@gao.gov. 

[End of Section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Independent Reviewers Have Recommended Improvements to EPA's Risk 
Assessment Process Since 1994: 

EPA Has Strengthened Many Facets of Its Risk Assessment Process Since 
1994, and Efforts Are Ongoing: 

EPA Risk Assessors Responding to Our Survey Reported That Process 
Modifications Have Helped Them Prepare Better Risk Assessments but That 
Collaboration and Training Limitations Hamper Further Progress: 

Enhanced Risk Assessment Planning, Improved Analysis and Review, and 
Added Training Could Further Improve EPA's Process, but Barriers Could 
Limit Progress: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments: 

Appendix I: Objectives, Scope, and Methods: 

Appendix II: GAO Contact and Staff Acknowledgments: 

Figures: 

Figure 1: Typical Sequence of Risk Assessment and Risk Management 
Processes: 

Figure 2: Helpfulness of Office-Specific Guidance: 

Figure 3: Risk Assessors That Generally or Strongly Agree That 
Organizations Help Fill Data Needs: 

Figure 4: Areas in Which Training Has Moderately or Greatly Improved 
Risk Assessor Knowledge and Skills: 

Figure 5: Types of Training Taken by Risk Assessors: 

Figure 6: Modifications in Training That Would Moderately or Greatly 
Improve the Preparation of Risk Assessments: 

Abbreviations: 

ASPH: Association of Schools of Public Health:
ATSDR: Agency for Toxic Substances and Disease Registry:
BOSC: Board of Scientific Counselors:
Commission: Presidential/Congressional Commission on Risk Assessment 
and Risk Management:
CREM: Council for Regulatory Environmental Modeling:
EPA: Environmental Protection Agency:
FQPA: Food Quality Protection Act:
GIS: Geographic Information System:
HEI: Health Effects Institute:
HPV: High Production Volume:
ILSI-RSI: International Life Sciences Institute--Risk Science Institute:
IRIS: Integrated Risk Information System:
NAS: National Academy of Sciences:
NCCT: National Center for Computational Toxicology:
NCEA: National Center for Environmental Assessment:
NCER: National Center for Environmental Research:
NERL: National Exposure Research Laboratory:
NHEERL: National Health and Environmental Effects Research Laboratory:
NHEXAS: National Human Exposure Assessment Survey:
NIEHS: National Institute for Environmental Health Sciences:
NRMRL: National Risk Management Research Laboratory:
OAQPS: Office of Air Quality Planning and Standards:
OMB: Office of Management and Budget:
OP: Office of Pesticides:
OPPT: Office of Pollution Prevention and Toxics:
ORD: Office of Research and Development:
OW: Office of Water:
STAR: Science to Achieve Results:
TRIM: Total Risk Integrated Methodology:
TSCA: Toxic Substances Control Act: 

United States Government Accountability Office:
Washington, DC 20548: 

May 31, 2006: 

The Honorable James M. Inhofe: 
Chairman: 
Committee on Environment and Public Works: 
United States Senate: 

Dear Mr. Chairman: 

Chemicals play an important role in people's everyday lives. Consumers 
use products containing chemicals, such as cleansers and pesticides, 
and companies use a variety of toxic substances as solvents or 
additives to their industrial processes. Although these substances have 
beneficial uses and are important in producing goods and services, some 
may adversely affect human health. Over 100,000 chemicals, pollutants, 
and toxic substances are used in the United States and regulated by the 
Environmental Protection Agency (EPA). These substances, whether 
produced in the United States or imported, include 187 hazardous air 
pollutants, nearly 20,000 pesticide products on the market, and a 
growing number of substances used in industrial processes--over 82,000 
as of December 2005. EPA uses a process known as risk assessment to 
estimate the health risk from exposure to these substances, 
collectively referred to in this report as contaminants.[Footnote 1] 
While highly technical by nature, risk assessments, along with other 
relevant information, serve as a basis for regulatory decisions that 
protect human health. EPA's risk assessments are shaped by available 
scientific information and by provisions in major environmental 
statutes, such as the Clean Air Act, and they affect a range of 
stakeholders, including regulated industries, federal agencies, 
environmental advocacy groups, academic and other researchers, and the 
public.[Footnote 2] 

In 1994, as part of a congressionally mandated review of the methods 
used by EPA to estimate the risk of developing cancer from exposure to 
hazardous air pollutants, the National Academy of Sciences (NAS) issued 
a report entitled Science and Judgment in Risk Assessment. This report 
focused primarily on hazardous air pollutants and provided an overview 
assessment of EPA's risk assessment methods.[Footnote 3] In providing 
this overview, the NAS committee identified several themes, largely 
focusing on the quality and availability of essential information, that 
serve as a useful framework for discussing the risk assessment process: 

* Implementation. EPA faces certain overarching, institutional issues 
that affect its implementation of the risk assessment process. 

* Data needs. Data necessary to complete scientifically plausible risk 
assessments are often unavailable on such topics as the actual levels 
of exposure to contaminants and how those exposures affect human 
health. 

* Default options. In the absence of convincing scientific knowledge or 
data, EPA relies on assumptions, often conservative in nature, about 
such questions as how exposure to low doses of a contaminant affects 
human health. 

* Method and model evaluation. The predictive accuracy of methods and 
models, such as those used to predict how a contaminant will be 
processed once it enters the body, is not always known. 

* Uncertainty. The lack of precise knowledge about the type, 
likelihood, and extent of adverse effects from exposure to a 
contaminant results in uncertainty in risk assessment that can be 
reduced only by advances in scientific understanding or the collection 
of better data. 

* Variability. Variability, in exposure or in the biological 
differences among humans that determine how exposure to contaminants 
affects health, can be better characterized with more data but cannot 
be reduced or eliminated. 

* Aggregate and cumulative effects. Aggregate effects are the results 
from exposure to a single contaminant by any combination of means-- 
including inhalation, contact with the skin, and ingestion; cumulative 
effects are the results from exposure to multiple contaminants by any 
combination of means. 

Science and Judgment in Risk Assessment was one of several reports by 
NAS that discussed risk assessment in general and made recommendations 
for ways to improve it.[Footnote 4] In addition, the Presidential/ 
Congressional Commission on Risk Assessment and Risk Management issued 
a report in 1997 that reviewed risk assessment, as well as risk 
management, in regulatory programs.[Footnote 5] Furthermore, the Office 
of Management and Budget (OMB) issued guidance related to peer review 
and ensuring the quality, objectivity, utility, and integrity of 
information released to the public.[Footnote 6] More recently, in 
January 2006, OMB issued a proposed bulletin that advocates minimum 
standards for the scientific quality of risk assessments to enhance 
their technical quality and objectivity. 

In the context of these prior reviews, you asked us to (1) identify the 
significant recommendations to improve human health risk assessment 
that have been made since 1994; (2) describe what EPA has done to 
modify its human health risk assessment process over the same period; 
(3) determine the effects these past modifications have had on the 
preparation of risk assessments; and (4) identify any additional 
actions experts believe EPA could take to improve its risk assessment 
process in the future, and the barriers EPA would face in doing so. 

To identify significant recommendations to improve human health risk 
assessment since 1994, we reviewed EPA documents, including those 
produced by EPA's Risk Assessment Forum, Science Policy Council, 
Council for Regulatory Environmental Modeling, and Science Advisory 
Board, as well as each of EPA's program offices. We also reviewed our 
own reports and documents produced by NAS and the Presidential/ 
Congressional Commission on Risk Assessment and Risk Management. We 
also spoke with experts in the risk assessment field, who identified 
many of these documents in the course of our discussions and provided 
insight into some of the recommendations made therein. To describe what 
EPA has done to modify its human health risk assessment process, we 
interviewed program office managers from EPA's Office of Air and 
Radiation, Office of Pesticide Programs, Office of Pollution Prevention 
and Toxics, Office of Water, and Office of Research and Development 
(ORD). While we recognize that EPA's regional offices and the Office of 
Solid Waste and Emergency Response conduct risk assessments, for the 
most part their work focuses on site-specific assessments and thus was 
not included in this effort. Within ORD, we interviewed managers in two 
of EPA's laboratories (the National Health and Environmental Effects 
Laboratory and the National Exposure Research Laboratory) and three of 
EPA's research centers (the National Center for Environmental 
Assessment, National Center for Environmental Research, and National 
Center for Computational Toxicology). Furthermore, we attended various 
EPA and stakeholder group training sessions and meetings. 

To determine the effects these modifications have had on the 
preparation of risk assessments, we surveyed risk assessors from the 
program offices mentioned above. We sought their views on the 
usefulness of many of these modifications, as well as on aspects of 
EPA's risk assessment process, including guidance documents, training, 
organizational structure, and collaboration. The survey and results can 
be viewed at [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-
637SP]. In addition, we interviewed experts in the risk assessment 
field and obtained their perspectives on the extent to which these 
modifications were helpful to EPA's risk assessment process. To 
identify additional actions experts believe EPA could take to improve 
its process and to identify barriers to such actions, we contacted 
experts representing a range of stakeholders in the process. 
Specifically, we contacted risk assessment scientists; toxicologists; 
scientific advisers to EPA; state officials; and representatives from 
regulated industries, government agencies, and environmental advocacy 
groups who have an expertise in risk assessment. We used an iterative 
process (often referred to as the "snowball sampling" technique) to 
identify these knowledgeable experts and selected for interviews those 
who would provide us with a broad and balanced range of perspectives on 
EPA risk assessment practices. We used a standard set of questions to 
interview each of these experts to ensure we consistently discussed 
each aspect of EPA risk assessment policies and practices. (See app. I 
for a more detailed description of the methodology we employed.) In 
addition, we consulted with NAS's Board on Environmental Studies and 
Toxicology at its June 2005 meeting to obtain input on the areas in 
which EPA has made the most progress in improving its risk assessment 
practices and areas EPA will need to focus on in the future. We used 
the experts present at this meeting as a starting point for our 
snowball sampling technique used to identify subsequent experts. We 
conducted our work from February 2005 through March 2006 in accordance 
with generally accepted government auditing standards. 

Results in Brief: 

Independent reviewers, including NAS, have made a number of significant 
recommendations to improve EPA's human health risk assessment process 
since 1994 that can be grouped and discussed in terms of the seven 
themes originally identified by NAS in 1994. First, reviewers have said 
that EPA should improve its overall implementation of the process with 
such steps as more thorough planning of its risk assessments and 
greater use of independent reviews by scientists (called peer reviews). 
Second, to improve the quantity and quality of the data upon which risk 
assessments are based, reviewers have recommended that EPA better 
utilize the scientific data it has and identify additional data needs 
on the potential adverse effects from exposure to contaminants, and 
prioritize and support research to meet the data needs identified. 
Third, because EPA lacks data to fully assess the health risk from 
exposures, reviewers have recommended that when the agency relies on 
assumptions--known as default options--it should, among other things, 
more clearly indicate when it relies on default options and how it 
chooses them. Fourth, to improve the accuracy of EPA's risk 
assessments, reviewers have stated that EPA should better evaluate the 
methods and models used in its analysis and incorporate newer, more 
powerful tools when appropriate. Fifth, to address the inevitable 
uncertainties associated with gaps in scientific knowledge and general 
unknowns about model and data accuracy, EPA should, according to 
reviewers, more explicitly analyze and characterize the sources of 
uncertainty in its risk assessments and, when possible, discuss the 
uncertainties both descriptively (qualitatively) and numerically 
(quantitatively). Sixth, reviewers have also recommended that EPA 
enhance its analysis of variability in levels of exposures to 
contaminants, as well as differences in individual reactions to 
exposure. Finally, because people are typically exposed to a mixture of 
contaminants through a variety of means, such as contact with skin and 
breathing air, reviewers have recommended that EPA improve how it 
considers the effects of combinations of contaminants and all possible 
means of exposure. 

EPA has strengthened many aspects of its risk assessment process since 
1994, and improvement efforts are ongoing. First, EPA has improved 
implementation of its risk assessment process by, for example, 
beginning to improve risk assessment planning and creating scientific 
leadership positions, such as the Office of the Science Advisor. 
Second, EPA has initiated actions to develop missing or incomplete 
scientific data on the potential adverse effects from exposure to 
contaminants. For example, EPA officials told us they have begun to use 
the planning and review processes to determine what data are needed and 
communicate these needs to both EPA and outside researchers. Third, EPA 
has cited the need for risk assessments to be more transparent about 
their use of default options. For example, EPA has recently issued 
guidance describing default options that are appropriate for certain 
purposes and directing risk assessments to disclose the default options 
used in a particular risk assessment. Fourth, EPA is enhancing its use 
of models and embracing new methodologies. For example, EPA has 
established a group--the Council for Environmental Regulatory Modeling-
-to review models and provide guidance in model selection. Furthermore, 
some program offices now use more advanced models to predict how a 
contaminant will be processed once it enters the body. Fifth, EPA 
generally characterizes uncertainty descriptively, but has begun to 
incorporate quantitative techniques into its uncertainty analyses. For 
example, EPA typically uses a tiered approach to conducting uncertainty 
analysis, starting as simply as possible, describing uncertainty 
qualitatively, and sequentially employing more sophisticated analysis, 
such as probabilistic analysis, as warranted. Sixth, EPA has made 
progress in describing variability due to differences in both the 
exposure and the susceptibility of exposed individuals to contaminants. 
For example, when determining a level of exposure that is unlikely to 
be harmful, EPA includes adverse effects for people who might be at 
increased risk because of their age or the state of their health. 
Finally, EPA has begun to take steps to consider the combined effects 
of exposure to multiple contaminants through multiple means of 
exposure. For example, EPA has developed a framework to assess the 
combined, or cumulative, risk and has directed its program offices to 
include cumulative risks when planning major risk assessments. 

EPA risk assessors believe the modifications EPA has made over the past 
10 years--particularly issuing additional guidance--have generally 
helped them improve risk assessments, but collaboration and training 
could be improved. Most EPA risk assessors believe that agencywide 
guidelines and policy and reference documents have helped them prepare 
risk assessments by, for example, providing useful frameworks for 
evaluating potential harm from chemicals, and have resulted in greater 
consistency among risk assessments prepared by different offices within 
EPA. Furthermore, EPA's collaboration with external researchers, 
including other federal research entities, academia, and industry, has 
improved EPA's ability to conduct risk assessments by providing 
expertise and research not always available within the agency, 
according to risk assessors. In addition, risk assessors reported that 
internal collaboration among EPA offices is moderately to very 
effective, but could be improved. For example, a few risk assessors 
commented that the program offices' knowledge of the regulatory context 
in which research will be used helps the researchers structure their 
work. However, risk assessors also reported that collaboration is 
hindered within EPA by, for example, conflicting priorities among the 
various offices and the poor communication between some of them. 
Finally, while risk assessors said improved training has also helped 
them gain relevant skills and knowledge, over 70 percent of the risk 
assessors responding to our survey stated that more in-depth or 
relevant training would improve their ability to prepare risk 
assessments. In addition, they believe training for risk assessors and 
managers in specific technical and scientific areas, such as emerging 
scientific issues and the use of newer models, is lacking. 

While the experts we spoke with said the modifications EPA has made 
over the past 10 years have been beneficial overall, they identified 
additional actions EPA could take to improve its risk assessment 
process, recognizing that EPA may face barriers to doing so. 
Specifically, experts said EPA could improve the planning it undertakes 
prior to starting a risk assessment by better focusing on what data are 
needed for the assessment and by involving stakeholders early in this 
planning process. For example, several experts said that increased 
involvement with a broad range of stakeholders early in the planning 
process would help identify alternative methods and models and obtain 
stakeholder concurrence with the agency's approach. In addition, 
experts said EPA could more thoroughly evaluate methods and models, 
transparently document its analytic choices, and enhance internal 
review. For example, several experts said that EPA should more 
transparently communicate which default assumptions were used in risk 
assessments, why the defaults were chosen, and what judgments EPA was 
making when it employed certain methods. Finally, experts said EPA 
could provide additional training for risk assessors, managers, and 
stakeholders on all elements of the risk assessment process, such as 
how to use and apply models and how to interpret data from emerging 
scientific fields. While these efforts would further improve the risk 
assessment process, experts pointed out inherent barriers that EPA may 
face in carrying them out, such as the scientific complexity of risk 
assessment, the difficulty of obtaining and applying data, and a 
cultural resistance to deviating from established methods. For example, 
several experts said that EPA's risk assessments have grown more 
technically challenging and require risk assessors and managers to have 
different skills from what they had in the past. 

To further improve the risk assessment process, GAO recommends that EPA 
enhance early planning of each risk assessment, identify and 
communicate data needs to the research community, and support 
development and implementation of in-depth training for risk assessors 
and managers. 

We provided EPA with a draft of this report for review and comment. EPA 
neither agreed nor disagreed with our findings and recommendations. 
However, the agency provided specific comments to improve the report's 
technical accuracy, which we incorporated as appropriate. 

Background: 

Risk assessment, as used in public health and environmental settings, 
is inherently a complex and highly technical process that provides a 
systematic scientific description of potential adverse effects from 
exposure to contaminants. Risk assessments are conducted to estimate 
whether and how much harm can be expected from exposure to a given 
contaminant or mixture of contaminants and to help determine whether 
the harm is significant enough to require regulatory or other 
corrective action. The adverse effects from exposure can be acute, such 
as respiratory illnesses or poisonings, or chronic, such as cancer or 
reproductive or genetic abnormalities. The result of a risk assessment 
is a statement of the likelihood that an exposed population will be 
harmed and to what extent. EPA's human health risk assessment process 
generally consists of the four stages recommended in 1983 by the 
National Academy of Sciences (NAS): (1) hazard identification 
(determining whether a substance could cause adverse affects and what 
those adverse affects would be), (2) dose-response assessment 
(determining the relationship between the magnitude of exposure to a 
contaminant and the probability and severity of adverse effects), (3) 
exposure assessment (identifying the extent to which exposure is 
predicted to occur), and (4) risk characterization (combining the 
information from the preceding analyses into a conclusion about the 
nature and magnitude of risk). 

Risk assessment plays a critical role in EPA's regulatory decisions, 
but the characterization of risk that results from a risk assessment is 
only one piece of information used to inform decisions on regulatory 
standards for protecting human health. This analysis is integrated with 
other information, such as economic information on the costs and 
benefits of mitigating the risk, technological information on the 
feasibility of managing the risk, and the concerns of various 
stakeholders. The combined analysis informs agency officials who 
ultimately make regulatory decisions. This overall process is generally 
called risk management. (See fig. 1.) According to NAS, the risk 
assessment process should be conceptually distinguished from how the 
results of that process are used in the risk management process. 
However, in practice, maintaining this separation is difficult because 
the two processes are fundamentally linked by the complementary needs 
of the risk assessors and the risk managers. Specifically, risk 
assessors need to be aware of the context in which the assessment will 
be used and to communicate their key findings, as well as their 
confidence in them, in a way that risk managers can understand and 
apply. Risk managers need to ensure that the risk assessment has been 
properly performed and can be integrated with other information to make 
and justify a sound regulatory decision. 

Figure 1: Typical Sequence of Risk Assessment and Risk Management 
Processes: 

[See PDF for image] 

Source: EPA. 

[End of figure] 

EPA's human health risk assessment process is limited to a great extent 
by the amount of scientific data available on contaminants and by the 
level of scientific understanding about how contaminants act in the 
body to produce adverse effects. Data are often unavailable, and 
science cannot always provide definitive answers. The complex and 
sophisticated analytic tools used in risk assessment cannot overcome 
basic scientific unknowns. As a result, EPA's risk assessments almost 
always include assumptions about potential adverse effects. Some 
assumptions are conservative--that is, they are intended to help ensure 
that the agency does not underestimate health risks. Some critics of 
EPA's risk assessment practices believe EPA uses assumptions that are 
unjustifiably conservative, given new scientific data and methods, and 
thereby produces estimates that overstate actual risks. Other 
stakeholders, however, criticize some agency assumptions for not being 
conservative enough in the face of scientific uncertainties. These 
criticisms often result in legal challenges to EPA's decisions. 

EPA's human health risk assessment process is also greatly dependent on 
its degree of knowledge about the population's level of exposure to 
contaminants. This knowledge includes the extent to which people are 
exposed to potentially harmful contaminants in their daily lives, the 
chemicals to which they are most often exposed, the levels of such 
exposures, how exposures change over time, and the sources of exposure. 
Risk assessors, researchers, and policymakers must often rely on 
estimates of human exposure that are often derived from data showing 
the extent the chemicals are found in the air, water, food, or other 
environmental media and assumptions about how and at what rate the body 
absorbs the chemicals it contacts. In addition to estimates from 
models, extrapolations from experiments involving animals, and 
measurements of chemicals in the environment, EPA also relies on more 
direct methods to measure exposure and more accurately assess exactly 
how much of a contaminant has been absorbed in the body. For example, 
EPA uses population activity models--models based on actual human 
behavior, such as the time spent outdoors or, for children, the amount 
of time spent on the floor--to better estimate an individual's true 
exposure. 

Risk assessment activities at EPA are carried out by both the agency's 
Office of Research and Development (ORD)--its principal scientific and 
research arm--and its program and regional offices, including the 
Office of Air and Radiation, Office of Pesticide Programs, Office of 
Pollution Prevention and Toxics, Office of Solid Waste and Emergency 
Response, and Office of Water. ORD carries out all steps of highly 
complex, precedent-setting risk assessments for specific contaminants, 
such as dioxin. In addition, ORD often has responsibility for the first 
two steps of the risk assessment process--hazard identification and 
dose-response assessment--in support of the program offices. In such 
cases, the last two steps--exposure assessment and risk 
characterization--are the responsibility of the various program 
offices. Three notable exceptions exist to this division of 
responsibility. The Office of Pesticide Programs and the Office of 
Pollution Prevention and Toxics often conduct all steps of risk 
assessments independently, in part, because the relevant statutes for 
these program offices place strict time frames on decision making and 
the confidential nature of data provided to the agency under these 
programs. Furthermore, the Office of Water also does all of the stages 
for some of the assessments for purposes of the Safe Drinking Water 
Act. 

EPA's approach to risk assessment varies across program offices, often 
as a result of different regulatory and legal requirements. For 
example, a branch of the Office of Pollution Prevention and Toxics 
assesses data submitted by industry applicants on approximately 2000 
new chemicals annually under the Toxic Substances Control Act (TSCA) of 
1976. TSCA generally requires EPA to evaluate the chemicals within 90 
days, but does not require all applicants to conduct laboratory tests 
on the potential hazards and risks of the chemicals. In contrast, the 
Clean Air Act requires EPA to establish National Ambient Air Quality 
Standards, to review the scientific basis for those standards at least 
every five years, and to revise the standards as appropriate. As part 
of this process, ORD summarizes the most current scientific information 
on the pollutant in question. ORD's findings and conclusions are then 
combined with other exposure and risk analyses to determine what, if 
any, revisions should be made to the standards. 

Risk assessment has been the center of numerous reports, analyses, and 
regulations over the years. One of the earliest was the National 
Academy of Sciences' 1983 report Risk Assessment in the Federal 
Government: Managing the Process, often referred to as the "Red Book" 
because of the color of its cover. Subsequently, NAS released several 
related reports, including Science and Judgment in Risk Assessment 
(1994), Understanding Risk: Informing Decisions in a Democratic Society 
(1996), and Strengthening Science at the U.S. Environmental Protection 
Agency (2000). In addition, Congress, via the Clean Air Act Amendments 
of 1990, created the Presidential/Congressional Commission on Risk 
Assessment and Risk Management (Commission) and required it to 
investigate the policy implications and appropriate uses of risk 
assessment and risk management in various regulatory programs designed 
to protect people from cancer and other chronic health effects that may 
result from exposure to hazardous substances. The Commission published 
its two-volume final report in 1997. 

In addition, the Office of Management and Budget (OMB) has issued 
guidelines, reports, and bulletins that have affected the practice of 
risk assessment. For example, in October 2001, OMB issued its 
Guidelines for Ensuring and Maximizing the Quality, Objectivity, 
Utility, and Integrity of Information Disseminated by Federal Agencies, 
which required agencies to issue their own guidelines to ensure the 
quality of information being disseminated. Furthermore, in September 
2003, OMB issued a report to Congress on the costs and benefits of 
federal regulation. As part of this effort, OMB sought public comment 
on the practice of risk assessment, many of which were directed at, and 
passed along to, EPA. OMB also issued the Final Information Quality 
Bulletin for Peer Review in December 2004 that provided guidance to 
federal agencies on what information is subject to peer review and 
defined a planning process for peer review. Most recently, on January 
9, 2006, OMB released a draft bulletin on procedures agencies should 
use to conduct risk assessments and information that should be 
included. Public comment on this draft will be accepted until June 15, 
2006. 

Independent Reviewers Have Recommended Improvements to EPA's Risk 
Assessment Process Since 1994: 

Independent reviewers, including the National Academy of Sciences (NAS) 
and the Presidential/Congressional Commission on Risk Assessment and 
Risk Management have made a number of significant recommendations to 
improve EPA's human health risk assessment process since 1994. These 
recommendations cover a range of actions, including improving planning 
of what will be required to complete a particular risk assessment and 
what key stakeholders need to be involved, supporting development of 
new scientific data on the potential adverse effects from exposure to 
the contaminant under review, and enhancing analysis of the potential 
risk and its uncertainties. All of them can be summarized and discussed 
in terms of the seven themes--implementation, data needs, default 
options, method and model evaluation, uncertainty, variability, and 
exposure to multiple contaminants and routes of exposure--reported in 
NAS's report Science and Judgment in Risk Assessment.[Footnote 7] 

Improve Planning and Review Process and Ensure an Appropriately Skilled 
Workforce to Conduct Risk Assessment: 

Independent reviewers made a number of recommendations to EPA to 
improve the overall implementation of its risk assessment process. The 
reviewers recommended that EPA improve its planning of risk assessments 
before beginning the process and adopt an iterative risk assessment 
approach. Reviewers also stressed that EPA should increase its use of 
risk assessment reviews by independent scientists, known as peer 
reviews; increase stakeholder involvement in the risk assessment 
process; and ensure it has the workforce needed to conduct quality risk 
assessments. Finally, the reviewers recommended that EPA improve the 
way it characterizes the risk it finds and strengthen the link between 
risk assessment and risk management. 

Improve Planning and Adopt an Iterative Risk Assessment Approach: 

In 1996, NAS reviewers stressed the importance of planning from the 
beginning of a risk assessment to incorporate the perspectives and 
knowledge of interested and affected parties.[Footnote 8] Because of 
the large number of stakeholders interested in the results of a risk 
assessment, it is important to ensure that the risk assessment 
addresses all of the stakeholders' needs. NAS pointed out that one way 
to do this is to be aware of the stakeholders' concerns from the outset 
and incorporate them into the analysis and characterization of risk. By 
involving stakeholders early, risk assessors can ensure that they ask 
the right questions, make appropriate assumptions, and determine the 
best way to summarize information, thereby potentially making the 
resulting message more credible to these parties. 

In addition to improved planning, NAS suggested in 1994, and the 
Commission concurred, that EPA develop the ability to conduct risk 
assessments iteratively--that is, begin with a screening analysis to 
ascertain the potential risk and refine that analysis as appropriate. 
Such an approach would start with relatively inexpensive screening 
techniques and move to more resource-intensive data gathering and 
modeling as the particular situation warranted. To guard against the 
possibility of underestimating risk, these screening techniques should 
be constructed to err on the side of caution where there is 
uncertainty. 

Increase Peer Review and Stakeholder Involvement in the Risk Assessment 
Process: 

Each of the NAS reports we reviewed, as well as the report by the 
Commission, recommended ways EPA could improve its peer review process 
and better involve stakeholders in risk assessment. These independent 
reviewers said peer review is critical to evaluate the accuracy and 
appropriateness of technical information, scientific observations, and 
interpretations used in regulatory decision making. The Commission 
observed that peer review can also help in the planning stages of a 
risk assessment to put a problem in context and draw on the knowledge 
of experienced researchers, public health officials, and scientists. 

In particular, NAS recommended in 2000 that EPA change its peer review 
policy to ensure greater independence of these reviews from the 
control, or appearance of control, of the program managers.[Footnote 9] 
EPA's Science Advisory Board had expressed concern about potential 
conflicts of interest because agency policy allowed the same individual 
to serve as both project manager for a particular work product and peer 
review leader.[Footnote 10] The NAS committee concurred with this 
concern. 

In addition, NAS's 1994 report recommended that EPA collaborate more 
with outside parties to improve the overall risk assessment process. 
For example, EPA could use external advisory groups, such as its 
Science Advisory Board, to help ensure that risk assessment decisions 
use the best science and incorporate full public discussion and 
participation from the scientific community. Such collaboration could 
also include a process for public review and comment. The Commission 
also said that agencies should use advisory groups, composed of 
stakeholders, to periodically evaluate the use of technical 
information. In addition, the Commission recommended that agencies 
establish clear, written guidelines for peer review and match the use 
of peer review to the importance of the decision to be made. 

In 1996, NAS commented on the importance of stakeholder involvement in 
the risk assessment process. NAS suggested that the risk assessment 
process involve the spectrum of interested and affected parties, 
including decision makers and specialists in risk analysis, at each 
step of the process. According to NAS, such involvement will help 
ensure that as much important, relevant knowledge as possible enters 
the process and that the process remains open and inclusive. NAS also 
mentioned another benefit of involving these stakeholders in the 
process: the participation of a spectrum of people helps ensure that 
the process will be framed correctly and the resulting risk 
characterization will be accurate, balanced, and informative. 

Ensure EPA Has the Skilled Workforce Needed to Conduct Quality Risk 
Assessments: 

Each of the NAS reports we reviewed contained recommendations stressing 
the importance of an organization's ability to ensure that it has the 
expertise and leadership needed to conduct risk analyses. In addition, 
according to current workforce models, agencies need to ensure that 
they have effective training and the capability to identify what skills 
and competencies the employees and the organization need. In 1994, NAS 
recommended that EPA acquire additional expertise, as needed, to better 
interpret data and reach sound conclusions about the risks to human 
health from a contaminant. For example, many of the conclusions 
regarding the potential risks from a particular contaminant rely upon 
risk assessment models based on animal data or, when available, human 
evidence from epidemiological studies. The quality of the risk 
assessment will depend upon how well these data are interpreted to 
predict health effects in human populations. NAS recommended that EPA 
acquire staff with specialized skills in fields such as toxicology to 
successfully complete this type of analysis. 

NAS also recommended in 2000 that EPA continue to place high priority 
on ORD's graduate fellowship and postdoctoral program. The report 
stated that to achieve scientific and technical excellence, EPA must 
first attract, retain, and properly support a dedicated professional 
staff. NAS also noted that while EPA has many outstanding scientists 
and engineers, ORD's workforce is aging and many staff may retire in 
the relatively near future. Periodic hiring freezes, combined with high 
competition from the private sector, had made it difficult for ORD to 
recruit the new talent needed to sustain and enhance the research 
workforce. NAS concluded that ORD's graduate fellowships and 
postdoctoral programs brought a stream of fresh scientific and 
technical talent into EPA's research program and helped train future 
researchers in environmental science, engineering, and other 
disciplines. NAS urged EPA to continue to place a high priority on 
these programs. 

In 2000, NAS made a series of recommendations related to scientific 
leadership and talent at EPA. Specifically, NAS recommended that EPA 
establish a new position--deputy administrator for science and 
technology--to address concerns that science at EPA was not perceived 
to be strong and that many EPA scientists believed their scientific 
knowledge and resources were not effectively used. Traditionally, EPA's 
most senior science official had been the assistant administrator for 
research and development, but that official lacked agencywide 
responsibility or authority to oversee the scientific and technical 
basis for regulatory and policy decision making. In addition, EPA's 
program offices were not required to follow scientific advice from ORD. 
Consequently, the NAS panel concluded that EPA needed an appropriately 
qualified science official at a sufficiently high level with both the 
authority and responsibility for agencywide scientific performance. 

Improve Characterization of Risk and More Closely Link Risk Assessment 
with the Needs of Risk Managers: 

Finally, reviewers recommended ways EPA could improve its 
characterization of risk and better link its risk assessment and risk 
management processes to harmonize the scientific aspects of risk 
assessment and the broader policy objectives of risk management. NAS's 
1996 report, which primarily focused on risk characterization and the 
role it plays in risk assessment, stressed that successful risk 
characterization can result in better and more widely accepted risk 
assessment and risk management decisions. The Commission echoed NAS's 
position and recommended that risk characterizations should include 
information useful to everyone in the risk management process, such as 
information on who is at risk, how they might be affected, what the 
severity of an adverse effect might be, and how confident the risk 
assessors are about their predictions. 

Reviewers also suggested that EPA better link its risk assessment and 
risk management processes. EPA had tried to separate these processes 
entirely to avoid the possible perception that EPA made scientific 
judgments on the risk posed by a contaminant on the basis of its 
willingness to regulate the substance rather than on the science. 
However, NAS pointed out that while some degree of judgment is 
necessary, the science-policy judgments EPA makes in the course of risk 
assessment should more clearly be informed by the agency's risk 
management priorities and goals. According to NAS, better linkage 
between risk assessment and risk management will help ensure that the 
risk assessments will be more accurate and relevant to risk managers. 
The Commission concurred, stating that risk assessment should be guided 
by an understanding of the issues important to risk managers' decisions 
about how to protect public health and the environment. 

Review Existing Data, Identify Data Gaps, Prioritize Research Needs, 
Foster Development of New Data, and Improve Strategic Research Planning 
and Collaboration: 

In 1994, NAS found, in general, that EPA did not have the full range of 
data needed to conduct adequate risk assessments to estimate the health 
risks associated with most contaminants--for example, data related to 
the effect a contaminant has on the body and the extent of exposure to 
the contaminant. These data are essential to ensure the accuracy and 
precision of the risk assessment. In addition, NAS recommended that EPA 
increase its collaboration with researchers and improve the 
transparency of its research priorities. 

To help address EPA's dearth of data, NAS recommended, among other 
things, that EPA review its existing databases, such as the Integrated 
Risk Information System (IRIS), and identify data gaps.[Footnote 11] As 
part of its database review, NAS recommended that EPA compile an 
inventory of the chemical, toxicological, and epidemiological 
literature in the databases to determine what gaps exist. NAS further 
suggested that EPA prioritize its future research on the basis of the 
significance of the missing data to risk assessments, convey its data 
needs to other public and private researchers, such as the Agency for 
Toxic Substances and Disease Registry,[Footnote 12] and develop 
incentives to expedite the generation of needed data by these entities. 

Furthermore, in 2000, NAS recommended, among other things, that EPA 
expand the multiyear research planning by its Office of Research and 
Development (ORD). For example, ORD had developed plans for research on 
topics such as particulate matter, endocrine disruptors,[Footnote 13] 
and drinking water. The NAS committee pointed out that these planning 
efforts will contribute to research program continuity and the 
achievement of strategic goals. In addition, NAS recommended that the 
plans be both continued for problem-driven research--research targeted 
at a particular environmental problem--and expanded to include core 
research--broader, more generic research that will help improve current 
and future understanding of a scientific issue. Research should be 
balanced between these two types, since core research will help ORD 
better understand and anticipate environmental risks and will enable 
ORD to provide better problem-specific research and technical 
assistance to the agency. 

NAS has also made recommendations to EPA in the area of research 
accountability. Specifically, NAS recommended that EPA improve the 
documentation and transparency of the decision-making process ORD uses 
to set research and technical assistance priorities and allocate funds. 
For example, NAS pointed to an EPA Science Advisory Board finding that 
ORD's lack of transparency in its process for setting research 
priorities made it difficult to evaluate the adequacy of proposed 
budgets. NAS also recommended that EPA expand on its efforts to create 
an inventory of science projects and programs across EPA by documenting 
and publishing a comprehensive and detailed inventory. The inventory 
should include information such as goals and objectives of each 
project, milestones, schedules, and staff allocations and should be 
used to ensure that science activities are properly coordinated through 
the agencywide science planning and budgeting process and are 
appropriately peer reviewed. 

Once the research strategy has been established, research 
collaborations can help execute it. NAS suggested that EPA should 
recognize the limits of its own research capabilities and develop a 
strategy to obtain outside support in research areas where ORD is not 
pre-eminent. Specifically, NAS recommended EPA enhance its research 
collaborations by developing and implementing a strategy to stimulate, 
acquire, and apply the results of research conducted or sponsored by 
other federal agencies, state agencies, universities, and industry in 
this country and abroad. NAS said such collaboration is especially 
important given the budget constraints EPA and the federal government 
face and the breadth of knowledge required to conduct the full range of 
risk assessments. In addition, NAS recommended that EPA develop 
additional mechanisms to promote and facilitate research interactions 
among grantees in EPA's Science to Achieve Results (STAR) grant program 
and ORD research staff.[Footnote 14] For example, NAS suggested that 
grant applicants could identify in their proposals how their research 
might complement or supplement ongoing or planned research in the ORD 
laboratories. NAS also recommended that EPA increase its efforts to 
disseminate ORD's research products, explain their significance, and 
assist others inside and outside the agency in applying the research. 
NAS noted that while EPA's policy and regulatory work receives a great 
deal of public attention, the agency's science research typically 
receives a similar degree of attention only when the scientific basis 
for a decision is questioned. They also pointed out that even internal 
EPA offices may be unaware of important benefits from ORD's research 
program. 

More Clearly Indicate Use of Default Options and How They Were Chosen: 

When EPA needs to rely on assumptions--known as default options-- 
because it lacks data to fully assess the potential health risks from 
exposure to a contaminant, reviewers said EPA should, among other 
things, more clearly indicate that it relied on these defaults and how 
they were chosen. EPA's defaults, which are based on general scientific 
knowledge and policy judgment, are, for the most part, 
conservative.[Footnote 15] One example of a commonly used default 
option is that laboratory animals are a surrogate for humans in 
assessing health risks: An adverse effect in the animals is taken as 
evidence of a chemical's potential to harm humans. 

Independent reviewers agreed with EPA's use of default options as a 
reasonable way to cope with the lack of available data. However, NAS 
recommended that EPA be clearer about the scientific and policy basis 
for each default option and when it uses a default. Furthermore, 
reviewers said that EPA should fully explain under what conditions they 
would depart from these default options. According to NAS, giving 
greater formality to the criteria for a departure would result in 
greater guidance for the public and lessen the possibility of 
undocumented departures that would undercut the scientific credibility 
of a risk assessment. The Commission concurred, stressing that the 
defaults used in risk assessments and the uncertainty associated with 
their results should be clearly identified and justified. 

Better Evaluate Available Methods and Models and Incorporate More 
Advanced Tools When Appropriate: 

The overall accuracy of a risk assessment largely depends on the 
validity of the various analytic methods and models EPA uses to assess 
the toxicity of, and exposure to, a particular contaminant. However, 
according to NAS, EPA often does not clearly understand the extent to 
which the methods or models it chooses accurately predict the toxicity 
or exposure to a contaminant. For example, because of limited 
scientific information on how contaminants actually work in the body to 
produce adverse effects in humans, EPA frequently uses a method that 
relies on studies involving laboratory animals to understand the 
toxicity of a substance. The concentrations of the contaminant 
introduced into the animals as part of these studies are higher and 
administered for shorter periods of time than humans would normally 
experience. Consequently, to determine the expected response in people, 
EPA extrapolates the response from laboratory animals to humans. While 
such extrapolations are useful to predict potential harmful effects of 
a contaminant, different analytic methods may better predict the 
effect. 

In 1994, NAS recommended, among other things, that EPA evaluate the 
accuracy of its methods and models for assessing toxicity and, when 
appropriate, incorporate more advanced tools. For example, models that 
are based on the underlying mechanisms at the cellular or molecular 
level can more accurately estimate the dose of the contaminant that 
would have an adverse biological effect on a specific part of the body. 
Regarding the risk of cancer from a particular contaminant, NAS 
recommended that EPA continue to both use methods involving animal 
studies to evaluate the possibility of adverse effects in humans and 
explore, when appropriate, mechanistic models. Furthermore, to better 
convey the cancer risk associated with a particular substance, NAS 
recommended that EPA develop a classification scheme that provides 
narrative statements regarding the hazards posed by carcinogens and a 
descriptive evaluation of the strength and nature of the evidence used 
to estimate the substance's potential for causing cancer. 

Regarding exposure, NAS noted that EPA had traditionally characterized 
exposure according to two criteria: exposure of the total population 
and exposure of a specified highly exposed subpopulation. While these 
two criteria can help assess whether any particular exposure might 
occur above a regulatory threshold, only considering the highly exposed 
subpopulation is likely to overestimate the exposures of most of the 
population. Consequently, NAS recommended, and the Commission 
concurred, that EPA consider the entire range of a population's 
exposure, rather than just the exposures of a highly exposed 
subpopulation. The Commission also recommended that EPA identify and 
evaluate highly exposed populations separately. Moreover, NAS 
recommended that EPA use population-activity models--models based on 
actual human activity patterns--to better estimate an individual's true 
exposure and expand efforts to use personal monitoring data to better 
understand actual exposures and variances across the population. 

More Explicitly Analyze and Characterize the Sources of Uncertainty in 
Risk Assessments: 

To address the inevitable uncertainties associated with gaps in 
scientific knowledge and general unknowns about model and data 
accuracy, reviewers recommended that EPA more explicitly analyze and 
characterize the sources of uncertainty in its risk assessments. 
Numerous gaps in scientific knowledge exist regarding the health 
effects of various contaminants, such as the exact amount of exposure 
to a particular contaminant that can cause an adverse effect or the 
biological effect of a contaminant on the body. In addition, knowledge 
is often lacking about which model or method might be most appropriate 
to estimate risks to human health from a particular contaminant. NAS 
recommended that EPA develop guidelines for how to analyze and report 
the different types of uncertainty, both for the overall assessment and 
for the different stages of the risk assessment, such as hazard 
identification and exposure assessment. For example, during hazard 
identification, uncertainty can be related to the quality, type, and 
results of scientific studies; however, during exposure assessment, 
uncertainty can be related to model choice or available data. 

NAS also recommended in 1994 that EPA conduct uncertainty analysis of 
the risk estimate and present the identified uncertainties as 
explicitly, accurately, and fully as feasible. Analysis of the effects 
of uncertainties can help inform EPA decision makers and the public 
about the extent of uncertainty associated with the risk assessment. 
The analysis can also show where additional research might resolve 
major uncertainties and where it might not. NAS recommended that the 
uncertainty analysis be presented both descriptively (qualitatively) 
and, where possible, numerically (quantitatively). For example, some 
sources of uncertainty, such as those related to estimating exposures, 
can be reduced through the use of more advanced statistical methods. 
Other types of uncertainty, such as those associated with extrapolating 
data from animal testing to predict the effect on humans, are more 
difficult to quantify. The Commission concurred and recommended that 
risk characterizations include narrative descriptions of the primary 
reasons for uncertainty, as this information is likely to be more 
understandable and useful than quantitative estimates or model results. 

Enhance Analysis of Variability in Exposure Levels and Health Risks to 
Exposed Individuals: 

Variability, which refers to the natural diversity in a population, can 
be better understood or described, but not reduced. To address the two 
main categories of variability--one related to differences in levels of 
individual exposures to contaminants and the other related to 
differences in individual reactions to exposure--reviewers recommended 
that EPA enhance its analysis of both types and carefully state in each 
risk assessment what assumptions it made about what is and is not 
accounted for. Specifically, variability related to different exposures 
depends on the various concentrations of a contaminant as it disperses 
in the environment, different breathing rates, and different food 
consumption and personal activity patterns. For example, infants and 
children generally consume more fruits, vegetables, and fruit juices 
per body weight than adults, and some people, such as agricultural 
workers, are more exposed to pesticides through breathing and skin 
contact. The Commission recommended that risk assessments identify 
groups of people who are likely to have higher exposures to 
contaminants and consult these groups in the early stages of an 
assessment to obtain information about all known sources of exposure. 

Reviewers also recommended that EPA revise the way it estimates how 
long a person is exposed to a contaminant. NAS and the Commission 
recommended that EPA move away from estimates of exposure based on a 
hypothetical "maximally exposed individual," who was assumed to be the 
person at greatest risk in a worst-case scenario, because these 
estimates do not account for a number of other factors that may affect 
exposure patterns and rates, such as the time the person spends indoors 
or going to work. Furthermore, estimates based on the hypothetical 
maximally exposed individual likely overestimate the exposures of most 
of the population and underestimate the exposures of subpopulations, 
such as agricultural workers, who may be more highly exposed than the 
general population. While EPA's 1992 exposure assessment guidelines 
suggest the use of ranges and high-end exposure estimates chosen from 
the high end of those ranges, according to NAS, EPA had not 
sufficiently documented the reliability of such estimates when data are 
limited. 

The second type of variability--differences in human susceptibility--is 
related to inherent differences among people, such as age, physiologic 
characteristics, lifestyle, genetics, sex, and ethnicity. Reviewers 
found that EPA's approach for reducing risks associated with chemical 
exposures generally did not include information on differences in 
individual susceptibility or encourage gathering evidence to identify 
these differences. The reviewers recommended, among other things, that 
EPA consider this "interindividual" variability and adopt a default 
option for differences in susceptibility among humans. In addition, NAS 
recommended that EPA assess risks to infants and children whenever it 
appears that their risks might be greater than those of adults. For 
example, the developing brains of infants and young children have an 
increased susceptibility to contaminants that harm the nervous system, 
such as lead. NAS specifically recommended that EPA sponsor research to 
examine the causes and extent of interindividual variability in 
susceptibility to cancer and the possible connection between 
susceptibility and age, race, ethnicity, and sex. The Commission 
generally concurred, adding that risk assessments should also identify 
especially susceptible subpopulations, such as people with asthma who 
may have an increased responsiveness to allergens and respiratory 
irritants. The Commission also stated that, where possible, available 
information about the range of the population's susceptibility should 
be considered and used in place of default assumptions. 

Better Consider the Human Health Effects of Exposure to Multiple 
Contaminants and Routes of Exposure: 

Reviewers recommended that EPA improve how it considers the effects of 
combinations of contaminants (cumulative exposure) and all possible 
paths of exposure to a single contaminant (aggregate exposure). People 
are typically exposed to a mixture of contaminants through a variety of 
pathways, such as contact with skin or eating food, each of which might 
be associated with an increased probability of one or more health 
effects. However, most risk assessments address a single contaminant 
and often focus on a single pathway of exposure, such as inhalation. As 
a result, NAS recommended that EPA should consider all possible 
exposure pathways. For example, when assessing risk from mercury, EPA 
should consider the risk to residents from inhaling mercury emitted 
from a nearby industrial smoke stack, as well as the possibly greater 
health risk of consuming mercury that has accumulated in the tissue of 
fish that are caught and eaten locally after mercury from the smoke 
stack was deposited into water.[Footnote 16] 

When assessing the risks of exposure to chemical mixtures, most risk 
assessments estimate the risks from individual contaminants, then 
calculate the combined risk by simple addition. However, this method 
ignores potentially synergistic interactions that may make the effects 
more damaging to human health than anticipated or antagonistic 
interactions that may make the effects less damaging than anticipated. 
Consequently, this method could either under-or overestimate the total 
risk. NAS said that simple addition of the risks from multiple 
contaminants may be appropriate for screening-level risk estimates. 
However if a more refined quantitative estimate is needed, EPA should 
consider using statistical procedures to combine the risks from 
exposures to multiple contaminants, which would help produce a more 
comprehensive estimate of risk. The Commission concurred but stressed 
that combining risks may not always be feasible; the risk analyses for 
exposure to each contaminant may not be compatible because the risk 
assessments may differ in accuracy. Further, the Commission recommended 
that for risk assessments involving exposures to low concentrations of 
multiple chemicals, the risks from each exposure should be added in the 
absence of information on exactly how the chemicals affect the body. 
However, if the multiple chemicals affect the body in different ways-- 
for example, if one chemical affected development while another 
affected the nervous system--the impact of each chemical on the body 
should be considered independently and not added together. 

EPA Has Strengthened Many Facets of Its Risk Assessment Process Since 
1994, and Efforts Are Ongoing: 

EPA has modified its human health risk assessment process since 1994 in 
several ways. First, the agency enhanced implementation of its risk 
assessment process by, for example, issuing guidance and realigning 
staff resources. In addition, EPA has taken steps to identify the 
scientific data it has on the potential adverse effects from exposure 
to various contaminants and has established collaborative relationships 
with external researchers to foster the development of needed 
additional data. Furthermore, EPA has begun to improve its use of 
default options, enhance its modeling capabilities, and explore new 
methodologies. EPA has also begun to characterize uncertainty 
quantitatively and analyze and communicate variability more thoroughly. 
Finally, EPA is more often considering the combined effects of exposure 
to multiple contaminants through multiple pathways. Most of these 
efforts are ongoing and can be discussed in terms of the themes 
presented in the previous section. 

EPA Has Enhanced Implementation of Key Aspects of Its Risk Assessment 
Process: 

EPA has taken a number of steps to improve implementation of its risk 
assessment process. Specifically, EPA has developed guidance and policy 
documents at the agency and program office levels; built scientific 
capacity; modified components of its approach to risk assessment; and 
refined its peer review and quality-assurance practices. 

EPA Has Issued Many Agencywide and Program Office-Specific Guidance and 
Policy Documents to Improve Risk Assessment Practices: 

The guidance and policy documents EPA has issued over the past decade 
were intended to help staff develop and use risk assessments and to 
provide basic information to the public about EPA's risk assessment 
methods. While some of these documents have remained unchanged over 
time, many have been revised, or will be revised, as science, 
knowledge, and analytic methods have improved. Among these documents 
are the guidelines issued by EPA's Risk Assessment Forum, a committee 
of senior EPA scientists established to promote agencywide consensus on 
risk assessment issues, which cover such topics as 
neurotoxicity,[Footnote 17] exposure assessment, and carcinogenic risk 
assessment. Each of the original five guideline documents created in 
1986 has been updated at least once, and some are slated to be revised 
again. For example, the exposure guidelines were revised in 1992 and 
revisions are currently being planned. In addition, the forum has 
issued two entirely new sets of guidelines since 1994--Guidelines for 
Reproductive Toxicity Risk Assessment and Guidelines for Neurotoxicity 
Risk Assessment--and has developed a number of policies on, among other 
things, risk characterization, peer review, and evaluating risk to 
children. EPA has also issued interim policy memorandums and position 
papers on scientific issues such as genomics and endocrine disruption. 

Similarly, much of EPA's agencywide guidance issued since 1994 has 
undergone revision and has its origins in earlier policy documents. For 
example, EPA's Risk Characterization Handbook, issued in 2000, has its 
roots in the 1995 Policy for Risk Characterization. The handbook 
stresses that risk characterization should be transparent, clear, 
consistent, and reasonable. EPA's policy on the use of peer review was 
originally issued in 1994 and was followed up by issuance of peer 
review handbooks in 1998 and again in 2000.[Footnote 18] The peer 
review policy was recently updated in January 2006. EPA has also issued 
technical guidance, such as its review of the processes to estimate a 
daily or continuous exposure to humans that is likely to be without 
appreciable adverse effects during a lifetime. 

In addition to the EPA-wide guidance, many of the program offices have 
also issued guidance documents that support their particular risk 
assessment efforts. In general, the office-specific guidance documents 
provide risk assessors with analytic tools and exposure scenarios 
pertinent to the statutory responsibilities of the office. For example, 
the Office of Pollution Prevention and Toxics created guidance in the 
form of an analytic tool to screen chemicals in the absence of data, 
which frequently occurs because of the lack of a requirement for 
industry to develop extensive data on new chemicals or new uses of 
existing chemicals. In addition, the Office of Pesticide Programs has 
issued pesticide-specific guidance documents. For example, to help 
ensure consistency in pesticide chemical risk assessments, the Office 
of Pesticide Programs issued guidance for developing residential 
exposure assessments and developed a template for making and 
documenting registration eligibility decisions. 

EPA Has Built Scientific Capacity through Increased Focus on Scientific 
Leadership, Greater Reliance on Research Advisory Groups, and 
Development of Future Scientific Talent: 

EPA enhanced its scientific leadership through the creation of the 
position of science advisor in 2002 and the Office of the Science 
Advisor in 2003, the increased reliance on research advisory groups 
composed of senior EPA scientists and external experts, and the 
continuation of its research fellowship programs. The overarching 
responsibility of the science advisor is to coordinate and oversee the 
scientific activities of the program offices at EPA to ensure the best 
use of science. The Office of the Science Advisor provides further 
leadership by establishing specific mechanisms to ensure that 
scientific results, combined with technical evaluation and peer review, 
play a prominent role in regulatory decisions and that EPA staff 
interpret and enforce regulations consistent with the science 
supporting them. 

In addition to enhancing its scientific leadership, EPA has also 
increased its reliance on research advisory groups since 1994. The 
Science Policy Council and the Risk Assessment Forum play key roles in 
advancing the practice of risk assessment at EPA. The council reviews 
the adequacy of existing policies, establishes science policy as 
needed, and coordinates EPA efforts related to methods, modeling, risk 
assessment, and environmental technology. The Science Policy Council 
staff facilitate ad hoc work groups, encourage communication and 
consensus building within the agency, and participate in technical work-
group activities and deliberations. 

The Risk Assessment Forum is a standing committee of senior EPA 
scientists established to promote agencywide consensus on difficult and 
controversial risk assessment issues and to ensure that this consensus 
is incorporated into guidance. According to an agency official, the 
forum is designed as a venue where staff can meet and discuss common 
risk assessment issues across program offices. One of the forum's main 
contributions to risk assessment at EPA has been the issuance of a 
series of risk assessment guidelines. The forum is currently working on 
new guidelines, such as one related to adverse effects on the immune 
system. When more specificity is needed on an existing guideline, the 
forum issues companion pieces, known as "purple books" because of the 
color of their cover, that provide additional or updated information. 

The Board of Scientific Counselors (BOSC) provides objective and 
independent advice, information, and recommendations about ORD's 
research program to ORD's assistant administrator. BOSC is composed of 
scientists and engineers from academia, industry, and environmental 
organizations who are recognized as experts in their fields. In 1998, 
BOSC completed a peer review of ORD's laboratories and 
centers.[Footnote 19] BOSC completed a second review of the 
laboratories and centers in 2002 and 2003 that identified key 
accomplishments of the laboratories and centers, as well as areas for 
future improvement. In addition, after EPA's Office of the Science 
Advisor issued its 2004 staff paper,[Footnote 20] it asked BOSC to host 
a workshop for EPA staff and other interested stakeholders, such as 
industry, environmental groups, and researchers, to provide feedback to 
refine EPA's current practices and to suggest alternative approaches 
for specific aspects of risk assessment. 

EPA has also worked to foster scientific excellence and enhance the 
skills of its existing workforce through its graduate and postdoctoral 
fellowship programs. One such program, the EPA/ORD Postdoctoral 
Fellowship Program, began in 1998, and, as of May 2003, 205 individuals 
had participated in the program. One benefit of the program is that it 
helps provide a ready pool of talented candidates for EPA vacancies. In 
fact, according to a study of the program conducted by the National 
Council for Science and the Environment, nearly half of the former 
postdoctoral participants had taken permanent positions at EPA. The 
Science to Achieve Results (STAR) fellowship program is designed to 
encourage masters and doctoral students to pursue careers in an 
environmental field. The STAR fellowship program has provided new 
environmental research in the biological and health sciences, two 
fields related to the development of human health risk assessment. EPA 
has also partnered with the Association of Schools of Public Health 
(ASPH) to offer 1-year placements of graduates in EPA laboratories, 
centers, and program offices to work on public health issues. In the 
announcement of opportunities for the 2006 ASPH Fellows Program, a 
dozen fellowships are being offered in areas related to human health 
risk assessment. For example, the National Center for Environmental 
Assessment is offering a fellowship to develop health assessments for 
various chemicals, and the National Center for Environmental Research 
is offering a position for someone interested in working on developing 
models or analyzing uncertainty in risk assessments. 

EPA Has Begun to Incorporate Planning Activities and Stakeholder Input 
into Its Risk Assessment Approach: 

To improve planning, which is a part of all risk assessments to some 
degree, EPA has issued various guidance documents and held workshops 
for staff. One of EPA's earliest related guidance documents,[Footnote 
21] released in 1997, was designed to help risk assessors and risk 
managers plan and document the scope of risk assessments and to 
consider input from appropriate stakeholders and experts, especially in 
those assessments involving the effects of combinations of 
contaminants.[Footnote 22] EPA followed up this guidance with workshops 
to help staff apply it in risk assessments. In January 2002, EPA issued 
a handbook to reflect some of the lessons learned from implementation 
of the 1997 guidance to make risk assessments more useful to decision 
makers and other stakeholders. These lessons, conveyed through case 
studies, include that planning can be particularly valuable when the 
assessment is complex, controversial, or precedent setting, and that 
explaining uncertainty to stakeholders can help develop trust, 
credibility, and support for the decision-making process. 

The 2000 Risk Characterization Handbook also strongly advocates the use 
of planning and presents a number of topics for both risk managers and 
risk assessors to consider, such as identifying the stakeholders in the 
process; scope of the effort; relevant management goals, issues, and 
policies; available data; and data needs. For example, during planning, 
risk assessors, risk managers, and stakeholders need to identify the 
key data gaps and discuss how best to fill them, such as whether to use 
existing data or conduct additional short-or long-term tests to 
evaluate exposure and effects. The Office of Air and Radiation 
recognized the need for planning and developed planning guidance as 
part of its Air Toxics Risk Assessment Reference Library, issued in 
2004. EPA acknowledged in its 2004 staff paper that it needs to 
continue to stress the importance of concerted and conscious planning 
with risk assessors and risk managers before a risk assessment is 
started. According to EPA, risk assessors need to outline early in the 
development of a risk assessment what will and will not be addressed 
and how they will develop the risk assessment. 

Stakeholders and the public play a key role in the planning, as well as 
at later stages in the development of a risk assessment. Stakeholders, 
at various levels and in various forms, can help ensure better 
understanding of the risk assessment results and may promote support 
for the selected risk reduction strategies. Program offices involve 
stakeholders in various ways. For example, the branch of the Office of 
Air Quality Planning and Standards (OAQPS) responsible for setting 
certain air quality standards for six principal pollutants solicits 
input from stakeholders in the planning phase of its periodic updates 
to the standards it sets.[Footnote 23] In addition, the public may 
officially comment on draft air quality standards once they are 
publicly released. The Office of Water pursues stakeholder and public 
involvement that includes working with the environmental community, 
industry, trade associations, risk assessor organizations, states, and 
bordering countries. In addition, the office's periodic reviews of 
water quality standards and other nonregulatory actions, such as health 
advisories, are all open processes that allow for public input on 
various stages of the analysis. 

For risk assessments involving the reregistration of pesticides, the 
Office of Pesticide Programs (OPP) established a process that provides 
several opportunities for public participation.[Footnote 24] Depending 
on the potential health risks posed by a pesticide product, the public 
has anywhere from one to four separate opportunities to comment. For 
example, if risk assessors estimate that the product poses little risk 
to human health, the public will have one opportunity to comment before 
OPP decides whether to approve the pesticide product. For higher-risk 
products, the public will have as many as four opportunities to 
comment. The first opportunity to comment occurs after OPP has 
completed a preliminary risk assessment. This preliminary assessment 
contains all of the elements of a risk assessment and has undergone 
internal review, but is not yet finalized. Notice of the opportunity to 
comment is distributed to people who have elected to sign up for such 
notifications, as well as through a "notice of availability" published 
in the Federal Register. The public can also comment on risk 
assessments prepared by the Office of Pesticide Programs through the 
office's Science Advisory Panel--which holds periodic public meetings 
on pesticide-related risk assessment issues, such as methods to assess 
skin sensitivity to exposure to pesticides or models used to estimate 
dietary exposures. 

EPA has also adopted an iterative approach to many of its risk 
assessments. An iterative approach begins with a screening assessment 
and progressively grows in depth and scope in relation to the estimated 
risks to human health. When a screening assessment identifies a 
potential for a nontrivial risk, EPA decides if pursuing that risk is 
appropriate based on its current priorities and available resources. If 
EPA decides to pursue the risk, a more detailed, refined risk 
assessment is performed. The degree of refinement is based on the type 
of decision, the available resources, and the needs of the risk 
manager. After refinement of the estimate, EPA reviews it to see if it 
will be sufficient to answer the questions posed. Refinements proceed 
iteratively until the assessment provides an adequate answer for the 
decision maker within the resources available. Both the revised cancer 
guidelines and EPA's 1995 Policy for Risk Characterization support an 
iterative approach to risk assessment. Some program offices have also 
adopted an iterative--or tiered--approach to risk assessment. For 
example, the air toxics program follows a tiered approach, beginning 
with an analysis that includes few data and many conservative 
assumptions. If this analysis indicates that the risk may be relatively 
high, assessors pursue more intensive analysis to determine if the risk 
is realistic or an artifact of the lower tier's conservative 
assumptions. Despite this move toward greater use of an iterative 
approach, EPA acknowledges it could be clearer about when it is taking 
such an approach. For example, EPA could be more transparent about when 
and why it makes a risk management decision based on a screening-level 
assessment rather than a more detailed assessment. 

EPA Has Refined Its Peer Review and Quality-Assurance Practices: 

In the years since the issuance of Science and Judgment in Risk 
Assessment, EPA has made strides to improve its peer review practices. 
EPA uses peer review to help ensure the quality of its risk assessments 
and keep them as objective and consistent as possible. EPA's Peer 
Review Policy states that scientifically and technically based work 
products related to agency decisions should be peer reviewed. In 2000, 
EPA issued its revised Peer Review Handbook, an update of the original 
1998 edition. In the intervening 3 years, EPA's Science Advisory Board, 
EPA's Office of Inspector General, the National Research Council, and 
GAO scrutinized the peer review process. In response, in part, to 
recommendations made by many of these groups, EPA issued the current 
edition of the handbook. Among other things, it instructs EPA to 
balance peer review panels in terms of expertise and biases to help 
ensure a reasonable and scientific review. 

In addition to issuing the Peer Review Handbook, EPA has undertaken a 
number of actions to help ensure the quality of its data and 
information. In May 2000, EPA established an agencywide quality- 
assurance system and issued the EPA Quality Manual for Environmental 
Programs. Key components of this system include assigning a quality- 
assurance manager to conduct independent oversight of data quality, 
developing a management plan, and conducting an annual assessment of 
the quality system. In addition, the system calls for an assessment of 
the data EPA used to support agency decisions to verify that they were 
of sufficient quantity and quality for their intended use. In 2002, EPA 
developed its information-quality guidelines in response to those 
issued by OMB, which stated that agencies must ensure the quality, 
objectivity, utility, and integrity of information released to the 
public.[Footnote 25] EPA's guidelines outline its policy and procedures 
to ensure and maximize the quality of the information it disseminates 
and describe the mechanisms by which EPA reviews information prior to 
dissemination. EPA issued a complement to these guidelines in 2003 to 
raise awareness among the public about EPA's ongoing interest in high- 
quality data and to serve as an additional resource for staff as they 
evaluate the quality and relevance of information.[Footnote 26] 

EPA Has Enhanced Its Strategic Planning and Refocused Staff Resources 
to Initiate Review of Existing Data and Development of Needed Data: 

Since 1994, EPA has initiated a number of actions to develop missing or 
incomplete scientific data on the potential adverse effects from 
exposure to contaminants, including refocusing and expanding the Office 
of Research and Development human health risk assessment program, 
improving strategic research planning, increasing collaboration with 
outside researchers, and enhancing databases. 

EPA Restructured and Augmented Its Research Office: 

In 1995, EPA restructured ORD to more effectively generate and gather 
information needed for the risk assessment process, combining 12 
existing laboratories into 3 national laboratories and 2 national 
centers--the National Health and Environmental Effects Research 
Laboratory (NHEERL), National Exposure Research Laboratory (NERL), 
National Risk Management Research Laboratory (NRMRL), National Center 
for Environmental Research (NCER), and National Center for 
Environmental Assessment (NCEA).[Footnote 27] Specifically, one 
laboratory, NHEERL, conducts research on the effects of various 
exposure routes and rates, dose levels and duration, and cumulative 
doses on health outcomes. Its main research areas, all of which help 
improve risk assessments, target (1) the use of mechanistic 
information--how a substance causes an effect at a biological level--to 
reduce uncertainties in risk assessment; (2) the cumulative risks posed 
by exposure to contaminant mixtures; (3) the health risks to 
particularly susceptible populations; and (4) the evaluation of public 
health outcomes to determine the effectiveness of actions designed to 
reduce health risks. This research is developed in coordination with 
the program offices to target issues in risk assessment for air 
pollutants, safe foods and pesticides, water, and contaminated lands. 

A second laboratory, NERL, researches and develops improved methods and 
models to assess and predict human exposures to harmful contaminants in 
air, water, soil, and food. For example, it developed methods to better 
characterize pollution sources, and models to quantify the effects on 
exposure from various individual behaviors. NERL works closely with the 
program offices to set research priorities and help ensure that its 
results are useful to the program offices. For example, NERL 
coordinated with OAQPS to format data from several studies on airborne 
particulate matter in a comparable manner that allowed NERL and OAQPS 
staff to develop more powerful analytic results than would have been 
possible from the individual studies alone. NERL also developed tools 
to enhance the exposure assessment portion of the risk assessment, such 
as a database of human activities by age, sex, and location, to better 
characterize exposure risk based on personal activities. The third 
laboratory, NRMRL, focuses its research more on ways to minimize 
exposure to contaminants that cause health risks than on ways to 
improve the preparation of risk assessments. For example, NRMRL 
researches sources of chemicals that disrupt the endocrine system and 
strategies to minimize exposure to such chemicals. 

The two centers, NCER and NCEA, serve as focal points for external 
researchers and risk assessors, respectively. NCER funds innovative 
environmental research by academic scientists to reduce uncertainty in 
risk assessment. Specifically, NCER has sponsored research to develop 
data for use in models, thereby helping to reduce the model's 
uncertainties. NCER also coordinates with EPA's laboratories and 
program offices to develop its research topics. The Science to Achieve 
Results (STAR) program, one of NCER's grant programs, funds competitive 
research proposals and graduate fellowships in environmental science 
and engineering fields to complement ORD's research in its strategic 
and research plans, such as the health effects of contaminants on 
airborne particulate matter and in drinking water, and, more generally, 
on children's health. For example, the STAR program has funded research 
to generate data on human exposures that will improve risk estimates. 
NCER has also established academic research centers in such areas as 
children's health. NCEA, EPA's national resource center for human 
health risk assessment, supports EPA's work in three main ways. First, 
NCEA conducts risk assessments of national significance--for example, 
assessments of dioxin and diesel emissions--and prepares the air 
quality criteria documents that reflect the state of the science for 
six principal air pollutants. Second, it develops scientifically sound, 
defensible risk assessment methods to improve the use of science in 
risk assessment, such as software to estimate a benchmark dose--the 
dose that produces change in the risk of an adverse effect. Third, it 
provides guidance and support to risk assessors and risk managers 
through such means as its management of the Integrated Risk Information 
Systems (IRIS), a database of the potential human health effects of 
exposure to various chemicals in the environment. 

In addition to the six labs and centers, EPA in 2005 established the 
National Center for Computational Toxicology (NCCT) to coordinate and 
implement EPA's research on computational toxicology, a cutting-edge 
field that uses mathematical models to predict adverse effects and to 
better understand the mechanisms through which a given contaminant 
causes harm. Given advances in such newly emerging disciplines as the 
study of genes and their functions, computational toxicology offers the 
potential for scientists to develop a more detailed understanding of 
the risks posed by a much larger number of chemicals than is currently 
possible. NCCT research is designed to develop tools to conduct 
quantitative risk assessments more rapidly and to improve the 
identification of chemicals that may pose substantial health risks. 
NCCT staff have begun to collaborate with other ORD laboratories and 
centers to effectively target their research efforts. 

EPA Has Enhanced Strategic Research Planning: 

Since 1994, EPA has undertaken a number of strategic activities to more 
closely link the data needs of program offices to research agendas of 
EPA and ORD.[Footnote 28] In connection with the goals presented in 
EPA's and ORD's strategic plans, ORD defined 16 high-priority research 
areas such as human health, endocrine disruption, airborne particulate 
matter and other air pollutants, and safe pesticides. Research 
strategies for each area are either in place or under development. For 
example, EPA's Human Health Research Strategy identifies four broad, 
overarching research areas to guide ORD's human health research over 
the next 5 to 10 years.[Footnote 29] 

To carry out its 16 research strategies, ORD began in 2000 to develop a 
multiyear implementation plan for each. The first plans emerged in 
2001, with most finalized by 2003 following widespread participation 
and input from many stakeholders as well as review by senior ORD 
managers. ORD invites input on the plans from ORD and program office 
staff, federal research partners, and outside peer groups, such as 
EPA's Science Advisory Board, the Board of Scientific Counselors, and 
the National Research Council. These plans establish the short-and long-
term goals and timelines required for ORD's laboratories or centers to 
implement each of the strategies. The National Program Director, a 
newly established position for each major ORD research area including 
human health, helps ensure that ORD's time and staff resources are used 
strategically and that the overall planning effort links to the needs 
of the program offices. 

Program offices use various planning and review approaches to determine 
data needs. For example, the Office of Air Quality Planning and 
Standards develops a research-needs paper at the conclusion of each 
periodic review of the air quality standards it establishes to inform 
the research agenda for the next review, which occurs about every 5 
years. Each paper helps ensure that research in key areas will be 
available for the next review. In contrast, the data needs of the 
Office of Pesticide Programs are defined by law.[Footnote 30] 
Applicants who wish to register a pesticide product must submit the 
data defined in the statute and regulations, and OPP staff determine 
whether the data are of sufficient quality and quantity to assess the 
risk from the pesticide product.[Footnote 31] This list of required 
data is currently undergoing revision and additional data requirements 
may be added. 

EPA Program Offices Have Established Collaborative Relationships with 
External Researchers: 

Since 1994, EPA has strengthened and formalized collaboration with a 
range of other federal researchers to better leverage its limited 
research dollars and foster the development of data to improve human 
health risk assessments. Specifically, EPA has developed relationships 
with agencies such as the National Institute for Environmental Health 
Sciences (NIEHS) and the Agency for Toxic Substances and Disease 
Registry (ATSDR). For example, in 1998, EPA established a cooperative 
agreement with NIEHS to develop a body of research on the relationship 
between exposures and children's health. This collaboration jointly 
funded Children's Environmental Health Research Centers at seven U.S. 
universities and one medical center to research children's asthma and 
other respiratory diseases, as well as ways to reduce farm children's 
exposure to pesticides. In addition, EPA works closely with ATSDR to 
help fill research gaps and develop chemical-specific toxicological 
assessments used in risk assessments. In 2004, EPA and ATSDR entered 
into a formal agreement to ensure close collaboration to avoid 
duplicating efforts to fill data gaps. Under the agreement, the two 
agencies formed a work group to coordinate their efforts to develop 
toxicological assessments for ATSDR's work at specific highly 
contaminated locations and for EPA's Integrated Risk Information System 
(IRIS) database. 

EPA, NIEHS, and ATSDR also jointly develop and annually review a list 
of approximately 275 hazardous substances commonly found at the 
nation's highly contaminated sites and for which ATSDR will prepare 
toxicological assessments. At each annual review, agency staff may add 
chemicals to the list and identify priority research to fill gaps in 
knowledge. Of these 275 chemicals, approximately 150 have been 
identified by EPA as high-priority needs. Each toxicological assessment 
contains almost everything that is known about the chemical, including 
its potential to harm human health or the environment. A key difference 
between these toxicological assessments and the ones in EPA's IRIS 
database is that ATSDR includes chronic cancer and noncancer effects, 
as well as acute effects, while IRIS generally includes only chronic 
cancer and noncancer effects. 

In addition, EPA has begun to establish collaborative relationships 
with scientific and industry-related researchers. For example, EPA has 
cooperative agreements with the International Life Sciences Institute's 
Risk Science Institute (ILSI-RSI), an organization that researches 
critical scientific issues in risk assessment, such as the development 
of risk assessment methodologies.[Footnote 32] These cooperative 
agreements were specifically designed to engage the scientific 
community and bring together scientists from different affiliations 
(including academia, other parts of government, and the private sector 
including industry) to address risk assessment issues. Under one 
agreement, ILSI-RSI is to research risk assessment approaches for 
cumulative and aggregate exposures. In addition, EPA has used research 
provided by CIIT Centers for Health Research, a chemical research 
laboratory funded by EPA, industry, and other federal agencies, to 
provide information for its formaldehyde IRIS assessment. Furthermore, 
EPA and industry jointly fund the Health Effects Institute (HEI)--an 
organization that researches the health effects of various air 
pollutants, including airborne particulate matter and ozone. HEI has 
provided data for risk assessments and convened panels of experts to 
review and issue reports related to risk assessment, recently on diesel 
exhaust. In its 2004 staff paper, EPA noted that it needs to continue 
to encourage development of the specific data necessary to more 
accurately assess potential risks, particularly with researchers 
responsible for generating appropriate data, such as those seeking 
approval to manufacture or use a chemical. 

The Office of Pollution Prevention and Toxics has two programs to work 
with industry to develop data on contaminants that can be used to 
better understand risks. The first is the High Production Volume (HPV) 
Challenge Program. This program was officially launched in late 1998 to 
ensure that a baseline set of data would be made available to the 
public on approximately 2,800 chemicals that are manufactured or 
imported in amounts greater than 1 million pounds per year. Diverse 
stakeholders, including the American Chemistry Council, Environmental 
Defense, and the American Petroleum Institute participate in the 
program. The HPV Challenge Program provides an opportunity for all 
stakeholders, including the public, to comment on the tests and data 
summaries from the chemical sponsors--companies and consortia that 
volunteered to make publicly available screening-level data that allow 
EPA, industry, and other stakeholders to more effectively gauge the 
potential hazards of HPV chemicals. All comments are publicly available 
on the World Wide Web. As of January 2006, EPA had commitments from 
industry sponsors to provide data for 2,247 of the chemicals. The 
second program, the Voluntary Children's Chemical Evaluation Program, 
is designed to provide data that will allow the public to better 
understand the potential health risks to children associated with 
certain chemical exposures. EPA asked companies that manufacture or 
import 23 chemicals that have been found in human tissues in various 
biological monitoring programs to voluntarily sponsor the evaluation of 
specific chemicals in a pilot program. Thirty-five companies and 10 
consortia volunteered to sponsor 20 chemicals. This program was 
developed only after considering comments and concerns from 
stakeholders. Of the 23 chemicals chosen for this pilot, data gathering 
has been completed for 9 and is under way for another 11. The remaining 
3 chemicals in the pilot program have no sponsors. 

EPA Maintains Databases of Information Related to Risk Assessment: 

EPA maintains both an agencywide and several program-specific databases 
of information to help in the development of risk assessments. The 
primary database used by risk assessors in the program offices is the 
Integrated Risk Information System (IRIS), an electronic database of 
descriptive and quantitative information on human health effects that 
may result from exposure to various chemicals in the environment. 
Program office staff combine the IRIS data with exposure information 
they develop to characterize the health risks of a given chemical in a 
given situation. 

Since 1994, EPA has changed the IRIS assessment process in several 
ways. For example, each IRIS file now contains a discussion of the key 
studies, as well as a description of the decisions and default 
assumptions used in the assessment. EPA has also expanded the review 
that IRIS assessments undergo. For example, internal peer reviewers, 
including EPA senior health scientists representing program offices and 
regions, review the IRIS summary and accompanying detailed technical 
information. After this review, ORD releases the document for external 
peer review. EPA makes draft assessments available to the public at 
this time and, following peer review, the IRIS assessment discusses the 
key issues reviewers raised and EPA's response. In addition, EPA has 
added a tracking system that allows IRIS users to readily determine 
where an individual assessment is in its development. 

In September 2003, EPA completed a congressionally requested review to 
assess the need to update information in IRIS, based on concerns that 
EPA and state regulators rely on potentially outdated scientific 
information. Input from EPA program and regional offices, the public, 
and other stakeholders indicated that EPA should, among other things, 
increase the number of new or updated assessments completed each year 
to 50. To date, EPA has fallen considerably short of this goal. 
According to a program official, EPA completed 8 IRIS assessments in 
2005, plans to complete 16 in 2006, and has approximately 75 
assessments under way. EPA officials said a number of factors, such as 
the complexity of the assessment process, resource limitations, and 
extensive peer review, had limited EPA's ability to complete more 
assessments last year. EPA has increased the number of staff working on 
IRIS assessments from 6 to 23 and may ultimately increase the number to 
29. The review also indicated that EPA needs to assign staff to develop 
health assessments for IRIS, and provide funding for extramural 
research and contracts to develop IRIS files and subject them to 
external peer review. 

EPA also changed how it sets priorities for which chemicals need new or 
updated IRIS assessments.[Footnote 33] Annually, EPA asks its program 
offices, regions, and the public to identify contaminants for which it 
should develop or revise IRIS assessments. EPA publishes the list in 
the Federal Register and requests the public and scientific community 
to submit any relevant data on substances undergoing review. EPA is 
currently reviewing ways to increase coordination with other 
governmental agencies that develop chemical assessments, outreach to 
stakeholders earlier in the development of IRIS assessments, and 
consultation with independent external reviewers. In 2004, the IRIS 
program also initiated a review of available scientific literature for 
the 460 chemicals in the database that are not under active 
reassessment to determine whether a reassessment based on new 
literature could significantly change existing toxicity information. 
For 63 percent of the chemicals reviewed, no major new health effects 
studies were found. Such literature reviews will be conducted annually 
and the findings noted in the IRIS database. 

In addition, some program offices maintain databases to enhance their 
risk assessments. For example, the Office of Air Quality Planning and 
Standards (OAQPS) maintains a database of dose-response values 
developed by various sources, including IRIS, ATSDR, and the California 
Environmental Protection Agency, as an aide for its risk assessors. 
OAQPS staff update this database as better data become available. As 
part of its National Air Toxics Assessment--an ongoing comprehensive 
evaluation of hazardous air pollutants in the United States--EPA 
assessed 32 air pollutants plus particulate matter in diesel exhaust in 
1996. The national assessment is designed to identify air pollutants 
with the greatest potential to harm human health, and the results will 
help set priorities for collecting additional data. As part of its 
assessment, EPA compiled a national emissions inventory of hazardous 
air pollutants from outdoor sources, estimated population exposures to 
the pollutants, and characterized the potential cancer and noncancer 
health risks from breathing the pollutants. 

ORD also maintains personal monitoring data on the chemicals in the 
air, foods and beverages, water, and dust in an individual's personal 
indoor and outdoor environments. For example, in its National Human 
Exposure Assessment Survey (NHEXAS) program, which was completed in 
1998, ORD collected human exposure data from hundreds of subjects from 
several areas of the country. NHEXAS provided data on background levels 
of total exposure to environmental contaminants that can be used as a 
baseline in exposure and risk assessments to estimate whether specific 
populations are exposed to increased levels of environmental 
contaminants. 

EPA Has Improved Its Choice and Communication of Default Options: 

EPA has explicitly stated the need for risk assessors to identify when 
they relied on a default option, why they chose it, and when they 
departed from using a standard default option, but the agency 
acknowledges more could be done. To carry out its mission to protect 
human health, EPA's risk assessment procedures, including its default 
options, are protective of human health. In three recent guidance 
documents--the 2004 staff paper, the 2005 cancer guidelines, and the 
risk characterization handbook--EPA advocated more transparency in the 
choice of default options. These documents summarize a significant 
change in EPA's approach. Specifically, EPA first critically examines 
all relevant and available data to assess health risks, then uses the 
default options only in the absence of adequate contaminant-specific 
data. EPA also states in its staff paper that it bases its default 
assumptions on peer reviewed published studies, empirical observations, 
extrapolation from related observations, and scientific theory. 
Moreover, the cancer guidelines include an appendix that defines the 
basis for each of the default options that may be used in a cancer risk 
assessment. The Risk Characterization Handbook notes that risk 
assessments should describe the full range of default options that were 
used, including ones to address uncertainty. Moreover, the handbook 
states that when defaults are used, the risk assessment should 
reference the relevant EPA guidance that explains them. 

EPA program offices also advocate greater transparency when using 
default options. Specifically, the majority of IRIS assessments 
completed since 1997 describe the defaults used in the analysis and any 
departures from those defaults. The Office of Air Quality Planning and 
Standard's Air Toxics Risk Assessment Reference Library contains a 
number of references to defaults that should be used in the course of 
preparing a human health risk assessment.[Footnote 34] For example, to 
estimate an individual's exposure to an air pollutant, the guidance 
presents the default option to use for exposure in a screening-level 
assessment, namely that the individual remains at a single location and 
continuously breathes polluted air. 

Despite the increased focus on more transparency in the use of 
defaults, EPA acknowledges it could more consistently describe how the 
default was developed and explain why it is a reasonable assumption. In 
its staff paper, EPA acknowledges it needs to ensure that the defaults 
are supported by the best available data and should look for 
opportunities to increase certainty and confidence in the defaults and 
extrapolations used. EPA also acknowledges it may need to re-examine 
older risk assessments that relied on defaults that can now be replaced 
with relevant data. To a large degree, the use of defaults is 
intertwined with EPA's ability to get the data it needs. As was 
discussed previously, EPA has targeted research, both within EPA and 
through its grant programs, to understand variability and uncertainty 
in the data derived from studies of laboratory animals, and this 
research may further reduce EPA's need to rely on default options. 

EPA Has Taken Steps to Enhance Its Modeling Capabilities and Embrace 
New Methodologies: 

Since 1994, EPA has taken several steps to enhance its modeling 
capabilities and embrace new methodologies for risk assessment through 
improved guidance and workshops. To help improve its models, EPA's 
Agency Task Force on Environmental Regulatory Modeling published a 
report that concluded a need existed for, among other things, training 
and technical support, agency guidance on external peer review of 
environmental regulatory modeling, and creation of a Committee on 
Regulatory Environmental Modeling.[Footnote 35] Also in 1994, EPA's 
Risk Assessment Forum developed a draft protocol to evaluate models for 
exposure assessments.[Footnote 36] In 1997, ORD and program offices 
conducted an agencywide conference, called the Models 2000 Workshop, to 
facilitate adherence to existing guidance on modeling, to define and 
implement improvements in how the agency developed and used models, and 
to recommend an implementation plan for improving modeling within the 
agency. 

EPA followed up these activities in 2000 by creating the Committee on 
Regulatory Environmental Modeling (CREM) to promote consistency and 
consensus within the agency on modeling issues (including modeling 
guidance, development, and application) and to enhance internal and 
external communications on modeling activities. CREM supports and 
enhances the existing modeling activities in the program offices and 
provides EPA with tools to support environmental decision making. CREM 
also provides the public and EPA staff with a central point of inquiry 
about EPA's use of models. In 2000, CREM launched agencywide activities 
designed to enhance the development, use, and selection of regulatory 
environmental models at EPA. One such activity--a workshop to 
facilitate discussion of good modeling practices--resulted in the 
development of modeling guidance. 

In 2003, CREM developed guidance and created a database--called the 
Models Knowledge Base--of the models most frequently used in 
EPA.[Footnote 37] The guidance recommends best practices to help 
determine when a model, despite its uncertainties, can be appropriately 
used to inform a decision. Specifically, it recommends that model 
developers and users subject their model to credible, objective peer 
review, assess the quality of the data they use as inputs, and perform 
sensitivity and uncertainty analysis to determine which of the model 
inputs has the greatest impact on the modeled results. 

EPA has also incorporated efforts to improve models in its research 
strategies and implementation plans. For example, in its plan for 
research on hazardous air pollutants, EPA established a long-term goal 
to reduce uncertainties in risk assessments through methods, data, and 
models of acute and chronic exposures and exposures through multiple 
pathways at both the national and regional levels. In addition, one of 
ORD's laboratories established an exposure modeling research branch and 
develops population exposure models, such as the Stochastic Human 
Exposure and Dose Simulation model for inhalation and exposures of 
general and sensitive subpopulations through multiple pathways. EPA has 
also begun to use geographic information systems (GIS) to present risk 
information spatially. For example, a GIS system is being developed 
that maps all of the drinking water intakes in the United States and 
their associated watersheds, so that the agency can better assess risks 
to drinking water supplies stemming from activities in the related 
watershed. For risk assessments of hazardous air pollutants, GIS can 
display and analyze data during planning, scoping, and problem 
formulation, during the exposure assessment, and during the 
characterization of risks. GIS can also help communicate information to 
risk managers and other stakeholders. 

In addition to models, EPA is beginning to embrace such new risk 
assessment methodologies as probabilistic risk assessment and mode of 
action analysis. Probabilistic risk assessment characterizes the 
variability or uncertainty in risk estimates as the range or 
distribution of the number of times each possible outcome will occur. 
In probabilistic risk assessment, one or more variables in the risk 
equation, such as the exposure rate, is defined as a distribution 
rather than as a single number. A primary advantage of probabilistic 
risk assessment is that it provides a quantitative description of the 
degree of variability or uncertainty. EPA's 1997 policy states that 
probabilistic techniques, such as Monte Carlo analysis, can be viable 
statistical tools to analyze variability in risk assessments, when they 
are based on adequate supporting data and credible 
assumptions.[Footnote 38] The guidance presents a general framework and 
broad set of principles to ensure the use of good scientific practices 
when conducting probabilistic analyses of variability and uncertainty. 
EPA currently uses a number of models that include probabilistic 
analyses and is developing a new modeling framework, known as the 
Multimedia Integrated Modeling System, that will further enhance the 
agency's ability to probabilistically model uncertainty. 

EPA's recently revised cancer guidelines advocate the use of a mode of 
action analysis--based on the sequence of biological events that must 
occur to produce a harmful effect--to improve the accuracy of risk 
assessments. As a general rule, EPA assumes that toxic responses 
observed in laboratory animals indicate that the same responses are 
likely to occur in people even though differences in such areas as 
metabolic rates can result in different sensitivities between 
laboratory animals and humans. Mode of action analysis will more 
clearly indicate whether a difference exists between animals and humans 
in their response to contaminants. In addition, the guidelines present 
a new cancer characterization system consisting of five summary 
descriptors, to be used in conjunction with narrative, to describe the 
extent to which available data support the conclusion that a 
contaminant causes cancer in humans and to justify the summary 
descriptor selected. For noncancer risk assessments, EPA has used, and 
continues to refine, the benchmark dose methodology, which identifies 
the dose or concentration of a contaminant that slightly increases the 
likelihood of an adverse effect. 

EPA Is Introducing More Quantitative Characterization of Uncertainty 
into Its Risk Assessments: 

Uncertainty is inherent in all phases of risk assessment, from hazard 
identification through risk characterization. Over the years, EPA has 
relied more on qualitative, or descriptive, characterizations of 
uncertainty and less on quantitative, or numeric, characterizations. 
EPA's practice now is to use a tiered approach to analyze uncertainty. 
That is, EPA starts with a simple description of uncertainty and 
sequentially employs more sophisticated quantitative analysis, such as 
sensitivity analysis, provided the additional analysis reduces the 
uncertainty. To characterize risk quantitatively, EPA has typically 
used approaches that produce a single number to characterize the risk 
in terms of the level of a contaminant that does not cause harm, as 
opposed to presenting a range of possible values. 

Although EPA is beginning to use probabilistic approaches in exposure 
assessments, and has done so for six principal air pollutants, it does 
not typically do so to analyze uncertainty in its dose-response 
analyses, though its Science Advisory Board encouraged development of 
such approaches in 2000. EPA acknowledged in its 2004 staff paper that 
probabilistic risk assessment could be used more frequently and could 
provide useful information beyond screening-level assessments. However, 
the staff paper said probabilistic analysis may not be appropriate in 
all situations and the accuracy of the analysis will depend largely on 
the availability and quality of the data used in the analysis. 

Another quantitative approach to uncertainty is the use of uncertainty 
factors to account for such unknowns as variation in sensitivity among 
members of the human population or the appropriateness of extrapolating 
animal data to humans. EPA routinely uses uncertainty factors when it 
estimates the daily exposure to the human population that is likely to 
be without appreciable risk of adverse effects during a lifetime. This 
daily exposure estimate is called a reference dose for contaminants 
that are consumed and a reference concentration for inhaled 
contaminants. EPA states in its 2004 staff paper that it applies 
uncertainty factors in health assessments based on available data and 
the scientific judgment of EPA risk assessors and peer reviewers. 
According to EPA, most IRIS toxicological assessments, which contain a 
chemical's reference dose and reference concentration, provide 
justifications for the uncertainty factors applied to a particular 
chemical. Moreover, the factors undergo rigorous internal, and 
independent, external scientific peer review before being used in IRIS 
assessments. 

EPA has issued a number of documents that delineate the need to clearly 
and consistently characterize uncertainty in risk assessments. In 1995, 
EPA issued a risk characterization policy that stated that the risk 
assessment should fully, openly, and clearly characterize risks, and 
should disclose the scientific analyses, uncertainties, assumptions, 
and policies underlying the decisions. This policy was followed in 
December 2000 by the Risk Characterization Handbook, which includes 
guidance on how to address, among other things, uncertainty in risk 
assessment and describes the need and methods to present the sources 
and magnitude of uncertainty to the risk manager. More recently, the 
2005 revised cancer guidelines discuss each of the major uncertainties, 
such as model uncertainty or uncertainty related to human variation, 
and stress that assessments should discuss the significant 
uncertainties encountered in the analysis. For example, the guidance 
calls for the assessments of hazard, dose-response, and exposure to 
have accompanying technical characterizations covering the strengths 
and limitations of data and a discussion of uncertainty. 

EPA Better Analyzes and Communicates Variability in both Exposure and 
Susceptibility: 

EPA has made progress in describing variability due to differences in 
both the exposure individuals receive and the susceptibility of exposed 
individuals to adverse effects. A key document EPA risk assessors use 
to account for variation in exposure is its Exposure Factors Handbook, 
originally issued in 1989 and revised in 1997. The handbook summarizes 
data on human behaviors and characteristics that influence exposure to 
environmental contaminants and recommends values to account for those 
factors in assessing risk. Specifically, the handbook contains a series 
of over 150 data tables that provide information on how much time 
individuals spend at various activities and in various environments. 
Assessors can use these data to develop exposure duration estimates for 
exposure scenarios. For example, the tables contain statistics--broken 
down by age, gender, race, education, and some medical conditions, such 
as asthma or emphysema--for time spent in various outdoor locations. 
The handbook also provides general guidance to risk assessors on the 
types of variability relevant to a risk assessment and ways variability 
can be analyzed and addressed. 

All program offices address exposure variability in their risk 
assessments, although they do so in different ways. For example, risk 
assessors in the Office of Air Quality Planning and Standards who set 
certain air quality standards for six principal pollutants said they 
consider individual activity patterns for sensitive populations like 
children or asthmatics in exposure modeling by including a distribution 
of breathing rates to reflect variability inherent in the population. 
Furthermore, by modeling to protect the most sensitive or at-risks 
groups, they are assured of protecting the rest of the population. 
Variability in exposure to the six principal pollutants is generally 
described qualitatively in scientific summaries for each pollutant. The 
Office of Water includes an analysis of risks to various subpopulations 
and a narrative discussion of the strengths and weaknesses of the 
studies it used to estimate exposure, but generally does not include a 
quantitative analysis. The Office of Pesticide Programs considers 24 
different population subgroups in its exposure estimates, including 
differences in age, gender, ethnicity, and geographic dispersion. When 
data allow, the Office of Pesticide Programs develops a distribution of 
exposures and risks for its more refined risk assessments. 

To further its understanding of variability in exposure, EPA has 
undertaken a number of research projects. For example, one of ORD's 
laboratories conducted the National Human Activity Pattern Survey to 
provide detailed human exposure information for specific populations 
and allow EPA to better understand actual human exposure to pollutants 
in real-world situations. The survey results are stored in the 
Consolidated Human Activity Database to help risk assessors estimate 
the time that exposed people spend in various environments and their 
inhalation, ingestion, and dermal absorption rates while in those 
environments. This laboratory also conducts research to define, 
quantify, and reduce the uncertainty associated with the exposure and 
risk assessments, to develop improved methods to more accurately 
measure exposure and dose, and to develop technical information and 
quantitative tools to predict the nature and magnitude of human 
exposures to environmental contaminants. A recent EPA study was 
designed to identify chemicals commonly used in homes or day care 
centers, and whether children in these environments encountered the 
chemicals in the course of their daily activities.[Footnote 39] The 
research sought to identify the major routes (i.e., breathing and 
ingestion) and sources (i.e., dust, food, air, soil, and water) through 
which children come into contact with chemicals. 

Variability also exists with regard to susceptibility to adverse 
affects because of inherent differences among humans. EPA most recently 
addressed variability in susceptibility in the 2005 revision of its 
cancer guidelines, which describe the importance of separate risk 
assessments for all potentially sensitive life stages, including adults 
and children. The supplementary cancer guidelines for children address 
issues pertaining to cancer risks associated with early-life exposures. 
Legislation can also require EPA to consider potentially susceptible 
populations and life stages. For example, the Safe Drinking Water Act 
Amendments mandate that EPA consider risks to groups within the general 
population that are at greater risk of adverse health effects, 
including children, the elderly, and people with serious illnesses. In 
addition, the Food Quality Protection Act contains special provisions 
for the consideration of risks to children from pesticides. In 1995, 
EPA's Science Policy Council called for EPA to consider the risks to 
infants and children consistently and explicitly as part of its risk 
assessments. In 1997, the White House issued an executive order that 
required EPA and other federal agencies to identify and assess 
environmental health and safety risks that may disproportionately 
affect children and to ensure that policies, programs, activities, and 
standards address such disproportionate risks.[Footnote 40] 

In its 2004 staff paper, EPA acknowledges that characterizing 
variability for susceptible populations and life stages is an area 
where it could improve, although the absence of data limits its 
efforts. In 2002, EPA pointed out that data are limited to identify 
susceptible populations and life stages for many contaminants.[Footnote 
41] In these situations, EPA typically relies on default options, such 
as the use of uncertainty factors to account for variations in 
susceptibility. Many of the exposure assumptions and default values 
used to assess plausible current and future exposure scenarios can be 
found in EPA's 1997 Exposure Factors Handbook, and recent updates to 
the handbook are available online. 

Another way EPA addresses variability is through research. One of ORD's 
four strategic research directions in its Human Health Research 
Strategy is designed to improve the understanding of why some people 
and groups are more susceptible and highly exposed than others. 
According to this strategy, ORD's research on subpopulations will focus 
on three factors--life stage, genetic factors, and pre-existing 
diseases--that have been identified by a program office and the 
scientific community as having a high priority for risk assessment. In 
2000, ORD released its Strategy for Research on Environmental Risks to 
Children to strengthen the scientific foundation of risk assessment and 
management decisions that affect children and guide EPA's research 
needs and priorities over the following 5 to 10 years. Approximately 75 
percent of the funding for this strategy will be dedicated to research 
grants under the STAR program, such as those designed to evaluate 
children's exposure to pesticides. 

EPA Has Begun to Consider the Combined Effects of Exposure to Multiple 
Contaminants through Multiple Pathways: 

To help risk assessors analyze the health effects of exposure to 
multiple contaminants (cumulative exposure) and through multiple routes 
(aggregate exposure), EPA has issued guidance, developed methods and 
models, and supported research. In 1997, EPA's Science Policy Council 
issued guidance on cumulative risk assessment. This guidance directs 
each office to consider cumulative risk in planning major risk 
assessments and, where relevant data are available, to broaden the 
scope of the assessment to integrate multiple sources, effects, 
pathways, stressors, and populations for cumulative risk analyses. The 
guidance also highlights the need to ensure that the public and other 
stakeholders have an opportunity to help define the way EPA assesses an 
environmental or public health problem and calls for ongoing 
communication and coordination among EPA's risk assessors, risk 
managers, economists, engineers, and other technical experts. 

In 2000, EPA updated its 1986 guidance on chemical mixtures, to 
generate a consistent agencywide approach to assess health risks from 
exposures to multiple chemicals.[Footnote 42] The guidance is organized 
according to the type of data available to risk assessors, ranging from 
data-rich to data-poor situations, to help risk assessors select an 
appropriate methodology. For example, if data are of poor quality or 
quantitative data are very limited on chemical mixtures, the risk 
assessor may choose to perform a qualitative analysis of the potential 
human health impacts from exposure to the mixture. The guidance also 
contains procedures to develop toxicity equivalency factors, based on 
the toxicity of components of the mixture, to assess the risk from 
mixtures in the absence of data on the specific mixture. 

In 2003, EPA's Risk Assessment Forum developed a simple, flexible 
framework to help risk assessors consistently conduct and evaluate 
cumulative risk assessments.[Footnote 43] The framework is conceptually 
similar to the one used in human health assessments of a single 
contaminant in that it follows a three-stage approach of (1) planning, 
scoping, and problem formulation; (2) analysis; and (3) risk 
characterization. In addition, the document also highlights needed 
areas of research and methods development that may be important to the 
evaluation of cumulative risks, such as understanding how the sequence 
and timing of exposure may influence the ultimate risk for effects. 

EPA risk assessors use a number of models and methodologies to analyze 
multiple routes of exposure. For example, the branch of the Office of 
Air Quality Planning and Standards that regulates hazardous air 
pollutants employs the Multiple Pathways of Exposure model to assess 
and predict the movement and behavior of chemicals in the environment. 
For example, the Multiple Pathways of Exposure model includes 
procedures to estimate human exposures and health risks that result 
from the transfer of pollutants from the air to soil and surface water 
bodies and the subsequent uptake of the pollutant by plants, animals, 
and humans. The model specifically addresses exposures from breathing; 
consuming food, water, and soil; and contact with skin. More recently, 
EPA developed the Total Risk Integrated Methodology (TRIM) and created 
the TRIM Fate, Transport, and Ecological Exposure model that describes 
the movement of air pollutants emitted from any type of stationary 
source as well as their transformation over time in water, air and 
soil. 

Some program offices have also taken steps to explicitly consider the 
risks associated with more than one route of exposure or more than one 
chemical. Specifically, the Office of Pesticide Programs issued 
guidance in 2001 and 2002 in response to statutory requirements to 
assess the risk of aggregate exposure--exposure to a single chemical by 
multiple pathways and routes.[Footnote 44] The first set of guidance 
focuses on how to assess aggregate risk in those cases where more 
extensive data and more sophisticated exposure assessment methods and 
tools are available; this guidance also emphasizes, when data are 
available, the use of distributional data--aggregate exposures of many 
individuals in the population of interest--for all pathways of 
exposure.[Footnote 45] This approach allows the risk assessor to more 
fully evaluate exposure and resulting risk across the entire 
population, rather than the exposure of a single highly exposed 
individual. The second set of guidance describes a framework to assess 
potential human health risks from all pathways of exposure to multiple 
pesticides that share a common mechanism of toxicity--that is, the 
pesticides produce a similar toxic effect on the same organ or 
tissue.[Footnote 46] 

In addition, the branch of the Office of Air Quality Planning and 
Standards that regulates hazardous air pollutants developed the 
Integrated Urban Air Toxics Strategy, which is used to consider 
cumulative risks presented by exposures to hazardous air pollutants 
emitted from various sources. Staff can assess risk at both a national 
and an urban or a neighborhood scale. Furthermore, the hazardous air 
pollutant office developed guidance on multipathway risk assessments 
that are particularly important for hazardous air pollutants, such as 
mercury and dioxins, because human exposure occurs both from breathing 
air containing the toxins and from consuming plants, water, and soil 
where the pollutants were deposited. Moreover, these pollutants persist 
in the environment for long periods of time and may also accumulate in 
the tissues of commonly consumed plants and animals to levels that are 
harmful to humans.[Footnote 47] The guidance, maintained in the Air 
Toxics Risk Assessment Reference Library, describes how to plan, scope, 
and formulate the problem, conduct the analysis, and characterize the 
risk for such cases. 

The extent to which program offices assess the effects of cumulative 
and aggregate exposures is related to the regulatory responsibilities 
of each office and by the availability of data. For example, the 
hazardous air pollutant office routinely analyzes a mix of chemicals 
from various emitting sources, such as petroleum refineries, to 
regulate hazardous air pollutants.[Footnote 48] Similarly, as mentioned 
above, the Office of Pesticide Programs is required to consider 
exposure to pesticides from various pathways, such as food, drinking 
water, and residential uses, and various routes, such as eating, 
breathing, and contact with skin. In contrast, the Toxic Substances 
Control Act does not require the Office of Pollution Prevention and 
Toxics to assess the risks of a new chemical that may occur through its 
interaction with other chemicals. The office also assesses the risks of 
existing chemicals but cannot conduct cumulative risk assessment for 
classes of chemical that share a common mode of action because no data 
exist. Program managers hope such data will become available in the 
future. 

In its 2004 staff paper, EPA commented that while it has increased its 
emphasis on evaluating cumulative risks, it needs to expand on 
approaches to do so, and it needs to produce a rigorous scientific base 
to support such evaluations. To that end, one of ORD's four strategic 
research directions in its Human Health Research Strategy is to improve 
assessments of aggregate and cumulative risks. Specific research 
objectives are to develop exposure models and methods, provide a 
scientific basis to predict interactive effects of contaminants in 
mixtures, and determine the most appropriate approaches to combine 
effects and risks from mixtures. 

EPA Risk Assessors Responding to Our Survey Reported That Process 
Modifications Have Helped Them Prepare Better Risk Assessments but That 
Collaboration and Training Limitations Hamper Further Progress: 

EPA risk assessors responding to our survey reported that some 
modifications to its risk assessment processes, such as new or updated 
EPA guidance issued over the last 10 years, have been helpful. They 
also said that although collaboration among internal and external 
researchers has improved, problems remain with communication and 
coordination. Finally, risk assessors said that the training they've 
taken in the last 5 years has been beneficial, but they need additional 
training on analytic tools, such as modeling, and on other scientific 
disciplines related to risk assessment. 

EPA Has Issued Numerous Guidance Documents That Have Been Generally 
Helpful to Risk Assessors: 

At least two-thirds of risk assessors responding to our survey who 
reported using guidelines or reference documents indicated that these 
documents were moderately to very helpful in preparing risk 
assessments.[Footnote 49] In addition, between one-third and two-thirds 
of respondents who reported using policy documents said these documents 
were moderately to very helpful in preparing risk assessments. More 
specifically, many risk assessors said agencywide guidelines and 
reference documents provide a framework to assess risks to human health 
that help make risk assessments more consistent. For example, some risk 
assessors noted the usefulness of agency reviewed or approved 
procedures to support their assessments. In addition, some risk 
assessors said the guidelines and reference documents helped clarify 
issues, and several assessors said they were a good source for data 
needed to conduct assessments. Risk assessors responding to our survey 
cited the Guidelines for Carcinogen Risk Assessment as the document 
most frequently used when preparing human health risk assessments. More 
specifically, several risk assessors noted that the carcinogen 
guidelines provide a useful framework for preparing risk assessments. 
Many risk assessors commented that agencywide guidelines and reference 
documents are helpful or provide useful examples. For example, a few 
risk assessors stated that the Exposure Factors Handbook helps provide 
consistency among EPA offices that conduct exposure assessments because 
it defines standard values for exposure, and the rationale behind those 
values. Another assessor said that the Review of the Reference Dose and 
Reference Concentration Processes provides comprehensive guidance on 
setting reference values and contains a case study that serves as a 
model for concise and well-written hazard identification. Although risk 
assessors responding to our survey reported that guidance documents are 
generally helpful, many expressed concerns about them. For example, 
some risk assessors consider the documents too general or too difficult 
to decipher. 

In addition, 82 percent of the risk assessors whose offices have office-
specific guidance said that the guidance is very or moderately helpful 
with regard to preparing risk assessments. (See fig. 2.) According to 
many risk assessors, office-specific guidance provides information in a 
format relevant to each office's specific needs. For example, the 
Office of Pesticide Programs periodically issues "hot sheets" that 
describe how to apply general guidance to pesticide product risk 
assessments. In addition, the Office of Air and Radiation created the 
Air Toxics Risk Assessment Reference Library that provides information 
on how to analyze the risks from hazardous air pollutants. Over 65 
percent of risk assessors reported that EPA and program offices were 
moderately to very effective at disseminating guidance. 

Figure 2: Helpfulness of Office-Specific Guidance: 

[See PDF for image] 

Source: GAO analysis of survey data. 

Note: Percentages do not total to 100 because of some risk assessors 
did not respond to this question. 

[End of figure] 

Collaboration Has Improved EPA's Ability to Conduct Risk Assessment, 
but Problems Remain: 

To prepare risk assessments, EPA relies on external peer review and 
scientific data from a variety of sources on the potential health risks 
from exposure to contaminants. Collectively, this collaboration has 
improved EPA's ability to conduct those assessments. For example, 35 
percent of risk assessors responding to our survey reported that 
external peer review, which is often conducted by independent 
researchers, has definitely helped improve the overall quality of EPA's 
risk assessments, with an additional 48 percent noting that it has 
probably helped improve the overall quality. Furthermore, at least 32 
percent noted that peer review has definitely helped ensure that the 
science used in risk assessments is appropriately characterized, helped 
advance the use of new scientific methods or models, and helped EPA 
produce risk assessments that are more defensible. Another 38 percent 
said that peer review has probably helped with these improvements. In 
addition, responding EPA risk assessors rely primarily on other federal 
research entities, academia, and industry, and to a slightly lesser 
extent, ORD's research laboratories, to meet their offices' research 
needs. (See fig. 3.) 

Figure 3: : 

[See PDF for image] 

[End of figure] 

Figure 4: Risk Assessors That Generally or Strongly Agree That 
Organizations Help Fill Data Needs: 

[See PDF for image] 

Source: GAO analysis of survey data. 

[End of figure] 

More specifically, 63 percent of risk assessors generally or strongly 
agreed that they relied on other federal research entities, such as the 
National Toxicology Program; [Footnote 50] the National Institute for 
Environmental Health Sciences (NIEHS) and the National Cancer 
Institute--both within the National Institutes of Health; and the 
Agency for Toxic Substances and Disease Registry (ATSDR) to help fill 
their offices' needs for scientific data. EPA has established formal 
collaborative agreements with both NIEHS and ATSDR to research 
children's health and to develop toxicological data useful to both 
agencies, respectively. In addition, EPA has also collaborated with the 
U.S. Geological Service to identify contaminants in ground and surface 
waters. 

In addition to federal research entities, over 57 percent of 
respondents generally or strongly agreed that their offices rely on 
research from academia and industry to meet their research needs. For 
example, EPA has formal agreements with the International Life Sciences 
Institute's Risk Science Institute and the Health Effects Institute to 
develop research on approaches to analyze cumulative and aggregate 
exposures and the health effects of various air pollutants, 
respectively. EPA also relies heavily on industry-generated research on 
specific chemical substances. For example, under the laws that govern 
registration of pesticide products and new chemicals, applicants must 
supply specific data for relevant EPA offices to review when deciding 
whether to approve the pesticide products or chemicals in question. 

Furthermore, 46 percent of risk assessors said they relied on ORD's 
laboratories to generate research that helps fill scientific data 
needs. For example, some risk assessors said scientists from ORD's 
laboratories provide useful technical guidance on scientific issues or 
the risk assessment process. Another risk assessor commented that ORD's 
expertise is very useful to help interpret unusual findings or to 
advise on emerging issues. One risk assessor stated that ORD helped 
develop a specific model to use in probabilistic risk assessments. 
Other types of collaboration with ORD's laboratories include help to 
develop models for assessing dose and response relationships, to 
interpret toxicity data, to conduct epidemiological studies, and to 
develop scientific summaries for risk assessments of priority air 
pollutants. 

Overall, of the risk assessors who said they often or always 
collaborate with other EPA offices, at least 46 percent said that the 
collaboration was very effective. A few risk assessors commented that 
collaboration has become more effective in recent years, in part 
because staff in the various offices have more contact with one other, 
established scientist-to-scientist relationships, or learned whom to 
contact to address a particular question. For example, one risk 
assessor pointed out that cross-agency workgroups help to facilitate 
agencywide collaboration. Furthermore, a few risk assessors commented 
that the program offices' knowledge of the regulatory context in which 
research will be used helps ORD's researchers structure their work. For 
example, NERL collaborated with the Office of Air to fund studies that 
coordinated the format of data produced by different researchers to 
enhance the consistency of research approaches, which created more 
powerful results and made the data more useful to program offices. 
Finally, because some chemicals may be assessed by more than one 
office, collaboration across program offices helps ensure the 
consistency of risk assessments across EPA. For example, the Office of 
Pesticide Programs and the Office of Water may both prepare risk 
assessments for certain contaminants that may be found on food and in 
drinking water. 

Despite the improvements to collaboration at EPA, some risk assessors 
pointed out two barriers that limit collaboration. Specifically, 
assessors noted that conflicting priorities or goals among EPA offices 
and poor communication between some offices hinder the effectiveness of 
collaboration. For example, although some chemicals are studied by more 
than one office within EPA, the approaches and timelines differ among 
offices because the laws and responsibilities for each program office 
can differ significantly. As a result, what may be a priority chemical 
in one program office may not be a priority in another, thereby 
hindering timely collaboration. Furthermore, a couple of risk assessors 
found collaboration challenging because they could not find the right 
person in another office to communicate with on a specific issue. 

Several risk assessors suggested ways to improve and increase 
communication among program offices, ORD, and non-EPA organizations. 
For example, some risk assessors suggested more interagency work groups 
or meetings as a way to address research needs and foster information 
exchange on the development of methods. A few risk assessors suggested 
that a central library of risk assessment information would facilitate 
collaboration and avoid duplicating work already done by others. 
Specifically, one risk assessor said EPA could provide centralized 
databases of work conducted by different agencies and organizations, 
such as chemical-specific toxicity data, specific exposure or other 
values, and points of contact at each office. 

EPA's Training Has Been Helpful, but Risk Assessors and Managers Lack 
Sufficient Training on Analytic Tools and Emerging Issues: 

At least 80 percent of risk assessors responding to our survey said 
that the training they received, whether on the job, self-directed, 
office specific, or agencywide, was moderately to very useful. 
Moreover, over half of these risk assessors said that training improved 
their ability to prepare risk assessments. Nevertheless, risk assessors 
surveyed and agency officials interviewed reported that both risk 
assessors and managers would benefit from more in-depth training on 
subjects such as analytic tools and emerging scientific issues. 

Risk Assessors Have Enhanced Their Skills through Training in Various 
Areas: 

Over half of the risk assessors reported that training had moderately 
or greatly improved their abilities in at least seven different risk 
assessment skill and knowledge areas. More specifically, over 75 
percent of risk assessors reported that training has helped them learn 
about a particular risk assessment method or model, enhance the quality 
of risk assessments or risk characterizations they prepare, and 
maintain or refresh their existing knowledge (see fig. 4). To a 
slightly lesser extent, training also helped risk assessors apply EPA 
guidance and policies, more effectively prepare risk assessments, and 
understand EPA's four-stage paradigm for risk assessment. 

Figure 5: Areas in Which Training Has Moderately or Greatly Improved 
Risk Assessor Knowledge and Skills: 

[See PDF for image] 

Source: GAO analysis of survey data. 

[End of figure] 

Over three-quarters of risk assessors reported that they participated 
in on-the-job (82 percent) or self-directed (77 percent) training to 
enhance their skills. (See fig. 5.) According to risk assessors in the 
office that reviews new chemical applications, training is primarily on 
the job, largely because the office makes risk assessment decisions 
under tight time frames, which limits the time available for formal 
training, according to program officials. Similarly, workload 
constraints affect opportunities for formal training at the Office of 
Water, which instead relies primarily on self-directed or on-the-job 
training. Some on-the-job training takes the form of mentoring, such as 
in the Office of Research and Development, where senior staff are 
expected to mentor newer staff. 

Figure 6: Types of Training Taken by Risk Assessors: 

[See PDF for image] 

Source: GAO analysis of survey data. 

[End of figure] 

In addition, over 70 percent of risk assessors who responded to our 
survey reported that they participated in office-specific training to 
enhance their risk assessment skills. For example, the Office of 
Pesticide Programs (OPP) has a comprehensive risk assessment training 
program focusing mostly on scientific issues, but also on other issues 
to improve the overall quality of risk assessments. This training 
features speakers, including some from outside OPP, such as from other 
EPA offices, academia, and industry. These biweekly sessions are 
broadcast live over the office's internal computer system and taped for 
future use as well as broader distribution, so that staff can access 
them as needed. Moreover, OPP's training program includes having 
experienced risk assessors nearing retirement share their knowledge 
with other staff. Of the risk assessors responding to our survey that 
work in OPP, over 90 percent took advantage of its office-specific 
training. A similarly high percentage of staff in the hazardous air 
pollutant branch of the Office of Air and Radiation reported that they 
took that office's training on preparing risk assessments. 

In addition, 50 percent of risk assessors said they attended external 
training, such as professional society meetings sponsored by the 
Society for Risk Analysis, Health Effects Institute, and the Society of 
Toxicology and university courses related to scientific methods and 
disciplines. Finally, 46 percent of risk assessors said they 
participated in agencywide training. According to an EPA official, the 
agency offers a broad array of agencywide risk assessment training. For 
example, EPA's Risk Assessment Forum develops training to accompany the 
agency's risk assessment guidelines. More specifically, forum staff 
conducted briefings and orientation sessions on the 2005 cancer risk 
assessment guidelines. In addition, the forum sponsors colloquia 
approximately twice a year for staff to exchange information on risk 
assessment issues. Recent colloquia dealt with dermal exposure 
assessment and whether additional guidance is needed on the use of 
Monte Carlo analysis. Colloquia usually result in reports that 
summarize the findings and may inform future guidance documents. 

Additional Training Is Needed: 

While risk assessors reported taking advantage of and benefiting from 
various forms of training, they also said additional training would 
improve their ability to prepare risk assessments, a belief echoed by 
agency officials we spoke with. Over 70 percent of risk assessors 
stated that more in-depth or relevant training would improve their 
ability to prepare risk assessments. More specifically, some risk 
assessors expressed the need for training on analytic tools, such as 
modeling the dose-response relationship and statistical analysis and 
software. Several risk assessors also expressed a strong interest in 
training on scientific topics, including toxicology and such emerging 
issues as genomics, as well as nonscientific topics, such as public 
communications and public relations. Some risk assessors suggested EPA 
provide formal, comprehensive training for new risk assessors to 
educate them on how to use the guidance documents and more advanced 
courses for experienced risk assessors. Furthermore, many risk 
assessors reported that more time (69 percent) and more funds (65 
percent) for training would moderately or greatly improve their ability 
to prepare risk assessments. (See fig. 6.) 

Figure 7: Modifications in Training That Would Moderately or Greatly 
Improve the Preparation of Risk Assessments: 

[See PDF for image] 

Source: GAO analysis of survey data. 

[End of figure] 

In addition to our survey respondents, some program officials we 
interviewed pointed out the need for additional training for both risk 
assessors and risk managers. For example, risk assessors need training 
in emerging issues, such as genomics, so that they can use these types 
of data in risk assessments, according to one agency official. In 
contrast, risk managers need training on the elements of risk 
assessment, so that they will be better able to interpret the 
information provided and apply it to risk management decisions. 
According to one program manager, risk managers often want the "bottom 
line" (numerical) results of a risk assessment without understanding 
the nature of the uncertainties in the assessment, or the potential 
value of obtaining more information to help clarify those 
uncertainties. 

Finally, the changing nature of the workforce may have implications for 
training at EPA. According to an agency official, many experienced risk 
assessors who possess years of institutional knowledge are retiring or 
nearing retirement age. Consequently, the agency needs to educate newer 
risk assessment staff as quickly and thoroughly as possible to help 
ensure that the agency's ability to accurately and effectively produce 
risk assessments does not decline. While recently hired risk assessors 
come to EPA with specialized knowledge in fields related to risk 
assessment, they may not understand the broader context of risk 
assessment. For example, a new employee with degrees in biology and 
toxicology may not know how to integrate that knowledge with other 
scientific information to prepare a risk assessment. Furthermore, one 
EPA official noted that the agency has no formal training course, or 
set of courses, to help develop staff's ability to prepare risk 
assessments. 

Enhanced Risk Assessment Planning, Improved Analysis and Review, and 
Added Training Could Further Improve EPA's Process, but Barriers Could 
Limit Progress: 

The experts we spoke with, including representatives of federal and 
state agencies, regulated industry, environmental advocacy groups, and 
outside researchers and consultants, said the modifications EPA has 
made over the past 10 years have been beneficial overall. However, they 
identified additional actions EPA could take to improve its risk 
assessment process, recognizing that EPA would face barriers to doing 
so. Specifically, EPA could improve its planning process of what will 
be required to complete a risk assessment by better identifying the 
scientific data it has and data it needs on the potential adverse 
effects from exposure to a contaminant and by seeking stakeholder input 
early in this planning process. In addition, EPA could more thoroughly 
evaluate methods and models, transparently document its analytic 
choices, and enhance internal review. Finally, experts believe EPA 
could provide additional training for risk assessors, managers, and 
stakeholders. While these efforts would further improve the risk 
assessment process, EPA could face barriers in carrying them out, such 
as the scientific complexity of risk assessment, the difficulty of 
obtaining and applying data, and a cultural resistance to deviating 
from established methods. 

Enhance Planning by Increasing Focus on Data Needs and Involving a 
Broad Range of Stakeholders: 

In order to ensure that EPA has the data needed for risk assessment, it 
needs to better identify data that are available, prioritize its data 
needs, and collaborate with the external research community during the 
planning phase. For example, several experts said EPA should generate a 
searchable database of studies conducted by different agencies and 
organizations related to the chemicals being evaluated, so that 
researchers and risk assessors could more easily identify what studies 
are available and what additional research is needed. Experts also 
suggested several ways for EPA to prioritize its data needs. For 
example, sensitivity analysis can be used within individual risk 
assessments to determine which data gaps are the most critical to the 
risk assessment result. Some experts also said EPA could better 
prioritize its data needs by increasing its use of data on the amounts 
of contaminants in people's bodies to help concentrate its research on 
the chemicals to which humans are actually exposed. Finally, several 
experts suggested that EPA increase its collaboration with external 
researchers, in part because the agency lacks the resources to 
independently generate all of the data that are needed. If EPA more 
effectively collaborated with other federal research organizations, 
such as the National Toxicology Program and the National Institute of 
Environmental Health Sciences, federal research dollars could be better 
harnessed to help EPA protect the public from exposure to contaminants. 
For example, the National Toxicology Program has the technology to 
assist the Office of Pesticide Programs with its screening of inert 
ingredients, which are all of the "other" chemicals in a pesticide 
product. In addition, experts said EPA should use all relevant data, 
including data from industry research laboratories, provided EPA takes 
steps to ensure the data are generated in an unbiased and 
scientifically defensible way. For example, experts suggested that EPA 
could subject studies to independent peer review and evaluate the 
sufficiency of data produced by these organizations to increase 
confidence about using these data in EPA risk assessments. 

Experts also said that EPA could improve the quality of risk 
assessments if the agency enhanced its planning by more consistently 
involving stakeholders, especially early in the process. Several 
experts said that increased involvement with a broad range of 
stakeholders early in the planning process would help identify 
alternate methods and models to use and obtain stakeholder concurrence 
with the agency's approach. Although all stakeholders might not agree 
on the methods chosen, some experts believe that by seeking stakeholder 
input on these issues early in the process, EPA may minimize arguments 
later. In addition to stakeholder involvement with early planning, 
several experts recommended that EPA increase coordination with 
stakeholders throughout the process. For example, one expert said EPA 
could more transparently acknowledge and address comments from the 
public and other stakeholders, regardless of whether the agency planned 
to implement their suggestions. 

More Thoroughly Evaluate and Transparently Document Analytic Choices, 
and Increase Internal Review: 

EPA could more thoroughly evaluate its analytic choices and incorporate 
or develop a wider variety of analytic tools. Some experts said that, 
with regard to EPA's use of default options, risk assessors should more 
thoroughly document why available data are insufficient to allow EPA to 
use another analytic approach and commented that the revised cancer 
guidelines may provide a useful framework for making this decision. 
Furthermore, several experts said EPA could use a tool like sensitivity 
analysis to assess and clearly communicate the extent to which the 
choice of a method or default assumption affects the risk assessment 
outcome. For example, if sensitivity analysis demonstrated that the 
impacts were significant, EPA could use the analysis to identify the 
critical areas where additional studies might reduce the need to rely 
on a default assumption for that assessment. One expert pointed out 
that the Office of Pesticide Programs uses sensitivity analysis in its 
exposure assessments to estimate which uses of a pesticide present the 
greatest risk to workers and how to mitigate those risks. In addition, 
several experts recommended that EPA make better use of existing 
analytic tools and develop new ones, where needed. For example, some 
experts said EPA should more frequently employ probabilistic analyses 
in risk assessments and incorporate the latest scientific tools, such 
as genomics and computational toxicology, to better assess uncertainty 
and variability. In addition, several experts noted that EPA needs to 
develop tools and methodologies to better analyze certain aspects of 
risk assessment, such as the combined effects of exposure to multiple 
chemicals through multiple pathways. 

Experts also said EPA risk assessments should clearly describe the 
sufficiency of the data and the scientific basis for its choice of a 
default assumption, method, or model. Some experts pointed out that 
risk assessments should identify and clearly discuss any data that are 
not available for the analysis, including the form the data need to be 
in and the most appropriate study design or methodology to obtain the 
needed data. In addition, several experts said EPA needs to more 
explicitly communicate which default assumptions were used in a risk 
assessment and why the defaults were chosen. For example, one expert 
said that even though a risk assessment may be perfect, if the public 
does not understand the rationale behind the agency's choices, the risk 
assessment might be seen as flawed. Furthermore, in individual risk 
assessments, the agency could more transparently identify which 
critical studies would help the agency avoid relying on default 
assumptions. Some experts also suggested that EPA use as case studies 
completed assessments for which the agency had sufficient data to use 
models and other analytic tools rather than default assumptions to more 
accurately assess risks. Finally, some experts said that EPA should 
more transparently consider alternate methods and models in each risk 
assessment. For example, EPA should be more transparent about the 
judgments it makes when it employs certain methods, such as the 
benchmark dose method, which identifies the dose that produces a small 
increase in the risk of an adverse effect. 

Finally, experts suggested that EPA increase internal reviews of risk 
assessments by staff members with extensive risk assessment experience. 
Internal reviews could improve the risk assessment process in two ways: 
first, to assure the quality of risk assessments and second, to ensure 
that the design of its risk assessments match the needs of risk 
managers. For example, one expert suggested EPA reinstate a senior peer 
review group, composed of experienced risk assessors from throughout 
the agency. Others suggested that EPA could also internally peer review 
risk assessments prepared by less experienced staff to ensure that 
default assumptions are applied appropriately and transparently 
explained. In addition to increased review of individual risk 
assessments, some experts also felt the risk assessment process could 
benefit from additional examination of agencywide cross-cutting issues 
applicable to all program offices. For example, one expert said that 
some analytic tools, such as Monte Carlo analysis, were not developed 
specifically for use in risk assessments and suggested that EPA work 
with ORD's National Center for Computational Toxicology to define how 
these tools could be used in risk assessments across EPA. Moreover, 
according to several experts, agencywide discussions and activities 
promote consistency in risk assessment practices. For example, some 
experts thought EPA could benefit from a systematic agencywide 
discussion of the sources of uncertainty in risk assessment. 

More Training Could Improve Risk Assessor, Risk Manager, and 
Stakeholder Understanding of the Process: 

Experts emphasized the importance of training for risk assessors, risk 
managers, and the stakeholder community on all elements of the risk 
assessment process. Several experts said risk assessors are not 
adequately trained in basic risk assessment principles, such as 
available default assumptions and when they should be used or replaced. 
Some experts also suggested risk assessors receive training in using 
and applying models and in how to interpret data from emerging 
scientific fields to improve their ability to use these data, as 
appropriate, in risk assessments. Several experts also believe that 
training for risk managers would help improve risk assessments because 
risk managers need to better understand the role risk assessment plays 
in risk management. According to some experts, risk managers who are 
more familiar with the process are better equipped to support risk 
assessors and ensure that the risk assessment considers all appropriate 
factors. Finally, a few experts also suggested that EPA hold training 
for stakeholders in the risk assessment process. For example, one 
expert suggested that EPA develop Web-based training for both the 
regulated community and regulators themselves to help ensure 
consistency in how they understand the process. As part of this 
training, EPA could explain how risk assessment fits into the overall 
risk management process. 

EPA Faces Barriers to Improving Its Risk Assessment Process, Such as 
the Complexity of Risk Assessment, Difficulty of Acquiring and Applying 
Data, and a Culture Resistant to Change: 

While experts identified a number of actions EPA could take to improve 
the risk assessment process, they said EPA may face barriers such as 
the highly complex, technical, and time-intensive nature of preparing 
risk assessments, challenges in acquiring and applying data from all 
available sources, and a general reluctance to deviate from its 
established methods and assumptions. Several experts pointed out that 
EPA's risk assessments have grown more technically challenging and 
require risk assessors and managers to possess different skills than in 
the past. For example, some experts told us risk managers have 
different levels of expertise and background in risk assessment, and 
may not fully understand how risk assessment helps inform regulatory 
decisions. Moreover, some experts said that because risk assessment is 
just one piece of information used to make a regulatory decision, it is 
difficult to explain to stakeholders and the public the impact of risk 
assessments on risk management decisions. Experts also pointed out that 
scientific knowledge on subjects, such as uncertainty and variability, 
is limited and analytic tools are still being developed. For example, 
several experts said that while it would be useful for EPA to more 
fully consider the risks of exposure to a single chemical from all 
exposure pathways, at present it is an emerging science with few well 
developed analytic tools to use in risk assessments. In addition, using 
tools, such as probabilistic analysis, to assess variability requires 
large amounts of data that are seldom available. Finally, several 
experts said that improving the process by such steps as incorporating 
new analytic techniques and conducting thorough internal review 
requires more time and coordination. For example, one expert pointed 
out that EPA does not always have the staff and time to analyze all 
sets of data or to examine alternative methods or models that might 
provide a more robust risk estimate. 

In addition to barriers attributable to the complexity of preparing 
risk assessments, experts also said EPA may face barriers in acquiring 
and applying data from all available sources. Many experts commented 
that data are expensive to obtain, and EPA has limited financial 
resources to devote to such activities. For example, some experts 
pointed out that some of the more direct studies of human exposure, 
such as epidemiological or biomonitoring studies, are quite expensive 
to conduct. Furthermore, EPA may be reluctant to use available data 
from all sources. As several experts pointed out, data from industry- 
sponsored researchers might be perceived as biased, potentially 
subjecting EPA to criticism. Despite potential perceptions of bias, 
some experts thought EPA should have the ability to use all available 
data, regardless of its source, as long as the data in question have 
been appropriately peer reviewed. In addition, some experts said 
statutory requirements may limit EPA's ability to use certain data. For 
example, the Toxic Substances Control Act limits EPA's authority to 
require extensive data from industry before deciding whether to approve 
a new chemical. Some experts also pointed out that research does not 
always produce clear-cut results. For example, one expert commented 
that epidemiological studies of the general population may not account 
for confounding factors, such as exposure to other chemicals, which 
complicate efforts to draw conclusions about the effects of a single 
chemical. In addition, some experts said that variability, an important 
but scientifically complicated issue, often creates inconsistencies 
across studies because many factors such as geography, lifestyle, and 
food intake affect an individual's response. 

Finally, experts said that EPA has a general reluctance to deviate from 
using methods and assumptions it has used in the past. As a result, 
experts said EPA prefers to use techniques that have been generally 
accepted in the scientific community than to use methods that rely on 
recent scientific advances. For example, some experts told us EPA is 
often reluctant to deviate from its established default assumptions. 
Furthermore, some experts also commented that risk assessors may not 
have an incentive to deviate from methods and assumptions they have 
used in the past because it may make the risk assessment more easily 
challenged by those who disagree with it. In addition, some experts 
said the level of comfort in using new methods varies throughout the 
agency. For example, one expert believes that probabilistic models have 
been applied inconsistently because some risk assessors have been 
unwilling to deviate from the standard models. 

Conclusions: 

While technical and difficult to understand by nature, risk assessment 
is a key element in EPA's efforts to protect human health from the 
potentially harmful effects of chemicals, pollutants, and toxic 
substances that people encounter in their everyday lives. Since 1994, 
EPA has taken a number of steps, including greater involvement by the 
public and other stakeholders, to strengthen and improve its process 
for preparing assessments of the risks posed by contaminants in the 
environment. Independent reviewers as well as the experts and EPA risk 
assessors we contacted said overall EPA's efforts have improved the 
agency's risk assessments. However, the agency itself and the 
individuals we contacted acknowledge that EPA needs to do more. While 
some barriers to further improvement depend on scientific advances that 
are largely beyond EPA's control, other actions to improve its risk 
assessment process are within its control. Specifically, when EPA 
engages the stakeholder and research communities after the risk 
assessment has largely been completed, it misses opportunities to 
benefit from their expertise. By working with stakeholders early and 
periodically throughout the process to identify key issues, studies, 
methods, and default assumptions that need to be considered in the 
analysis, EPA would help ensure consistent, transparent, and high- 
quality risk assessments. On the other hand, failure to take full 
advantage of stakeholders' knowledge and points of view is likely to 
contribute to the perception among stakeholders that their concerns are 
not adequately represented in the risk assessments and that EPA's 
decisions lack transparency. While EPA has issued a number of guidance 
and policy documents advocating the benefits of early planning, it 
acknowledges it could do more to ensure that such planning and 
consultation take place and involve relevant stakeholders. Furthermore, 
EPA does not always systematically communicate its data needs to the 
research community. While EPA has begun to better identify and 
prioritize its specific data needs, it has not been able to 
consistently develop data it needs in a timely manner. A more proactive 
approach to communicating its research needs to outside public and 
private researchers would help EPA more efficiently use the limited 
resources it has to obtain the data it needs. Furthermore, this 
approach would increase the likelihood that EPA would have data it 
needs to complete risk assessments now and into the future and that 
appropriately designed research projects would be conducted. 
Transparently communicating its research needs would also enhance EPA's 
ability to produce high-quality, scientifically defensible risk 
assessments and reduce the uncertainty associated with the effects of 
many contaminants on human health. Although experts we interviewed said 
EPA may hesitate to seek and use data from a wide range of sources 
because it could be seen as biased, EPA could take steps to ensure the 
quality of data generated by others. By doing so, EPA would expand its 
cache of available data and, potentially, reduce its reliance on 
default assumptions. Finally, current workforce models of high- 
performing organizations stress the need to formally and 
comprehensively assess the skill and competency requirements for staff 
and to identify related training and developmental needs to ensure that 
the workforce retains a high level of needed skills. In recent years, 
EPA's emphasis on training for its risk assessors and managers has 
declined in the areas risk assessors and experts say are needed to 
improve the quality of risk assessments and take advantage of recent 
scientific and analytic advances. Without an agencywide training 
program for its risk assessment and risk management workforce, the 
quality, consistency, and transparency of risk assessments and risk 
management decisions will likely continue to be challenged by 
stakeholders and the public. 

Recommendations for Executive Action: 

To improve the overall quality, consistency, and transparency of its 
risk assessments, we recommend that the Administrator of EPA direct the 
appropriate agency entities to take the following three actions: 

* Develop a strategy to ensure that offices engage in early planning to 
identify and seek the expertise needed, both within the EPA workforce 
and from external subject matter experts. The strategy should delineate 
such things as how EPA could use the available expertise to determine 
the needed data, the relevant default assumptions, the extent of 
internal and external review that needs to be included in the 
assessment, and the approach used to consistently involve a broad range 
of stakeholders--including the public, regulated industry, federal 
agencies, and advocacy groups--as appropriate to the risk assessment. 

* More proactively identify the data most relevant to the current risk 
assessment needs, including the specific studies required and how those 
studies should be designed, and communicate those needs to the research 
community. Increased collaboration among program offices in identifying 
needed data would help ensure that the resulting data will meet the 
needs of multiple offices. In addition, EPA should better communicate 
these data needs and better coordinate research planning with the 
external public and private research community to help focus EPA's 
limited resources. 

* Ensure that risk assessors and risk managers have the skills needed 
to produce quality risk assessments by developing and implementing in- 
depth training. This training should address the needs of risk 
assessors and managers with varying levels of expertise by including 
basic courses, such as an overview of risk assessment, as well as more 
advanced courses on topics such as modeling, toxicology, and other 
advanced scientific techniques. 

Agency Comments: 

We provided EPA with a draft of this report for review and comment. EPA 
neither agreed nor disagreed with our findings and recommendations. 
However, the agency provided specific comments to improve the report's 
technical accuracy, which we incorporated as appropriate. 

As agreed with your office, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies to the 
Administrator, EPA, as well as to appropriate congressional committees, 
and other interested Members of Congress. We also will make copies 
available to others upon request. In addition, the report will be 
available at no charge on the GAO Web site at [Hyperlink, 
http://www.gao/gov]. 

If you or your staff have questions about this report, please contact 
me at (202) 512-6225 or stephensonj@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made key contributions to 
this report are listed in appendix II. 

Sincerely yours, 

Signed by: 

John B. Stephenson: 
Director, Natural Resources and Environment: 

[End of section] 

Appendix I: Objectives, Scope, and Methods: 

Our objectives for this review were to (1) identify the significant 
recommendations to improve human health risk assessment that have been 
made since 1994; (2) describe what the Environmental Protection Agency 
(EPA) has done to modify its human health risk assessment process over 
the same period; (3) determine the effects these past modifications 
have had on the preparation of risk assessments; and (4) identify any 
additional actions experts believe EPA could take to improve its risk 
assessment process in the future, and the barriers EPA would face in 
doing so. 

To identify significant recommendations to improve human health risk 
assessment since 1994, we reviewed EPA documents, including those 
produced by EPA's Risk Assessment Forum (RAF), Science Policy Council 
(SPC), Council for Regulatory Environmental Modeling (CREM), and 
Science Advisory Board (SAB) as well as each of EPA's program offices. 
We also reviewed our own reports and documents produced by the National 
Academy of Sciences and the Presidential/Congressional Commission on 
Risk Assessment and Risk Management. To gain an external perspective, 
we spoke with experts in the risk assessment field, who identified many 
of these documents in the course of our discussions and provided 
insight into some of the documents' recommendations. 

To describe what EPA has done to modify its human health risk 
assessment process, we interviewed program office managers from the 
Office of Air and Radiation (OAR), Office of Pesticide Programs (OPP), 
Office of Pollution Prevention and Toxics (OPPT), Office of Water (OW), 
and Office of Research and Development (ORD). We did not include the 
site-specific risk assessment activities of the Office of Solid Waste 
and Emergency Response in our review. Within ORD, we interviewed 
managers in two of EPA's laboratories (the National Health and 
Environmental Effects Laboratory and the National Exposure Research 
Laboratory) and three of EPA's research centers (the National Center 
for Environmental Assessment, National Center for Environmental 
Research, and National Center for Computational Toxicology). We also 
interviewed officials from RAF, SPC, CREM, SAB, and the Office of the 
Science Advisor. Furthermore, we attended various EPA and stakeholder 
group training sessions and meetings. Since we limited our review to 
the human health aspects of risk assessment since 1994, our analysis 
does not highlight EPA's modifications prior to 1994, including 
publication of guidance documents that are highly relevant to risk 
assessment practices, and does not address issues specifically related 
to ecological risk assessment. 

To assess the effects these modifications have had on the preparation 
of risk assessments, we conducted a Web-based nonprobability survey of 
all human health risk assessors from ORD and four EPA program offices 
that conduct human health risk assessment (OAR, OPP, OPPT, and OW). We 
used the survey to obtain an internal perspective on the usefulness of 
many of the modifications EPA made since 1994, as well as on aspects of 
EPA's risk assessment process, including guidance documents, training, 
organizational structure, and collaboration. In developing the Web- 
based questionnaire, we met with EPA officials from the five offices 
surveyed to gain a thorough understanding of the risk assessment issues 
specific to each office and identify the sampling frame. In order to 
identify human health risk assessors--a label that is not an EPA job 
series--we obtained from EPA officials in each program office being 
reviewed the names of agency staff who worked on any part of the human 
health risk assessment process since January 2001. Our sampling frame 
consisted of 270 staff that met this criterion. This report does not 
contain all the results from the survey. The survey and results can be 
viewed at [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-637SP]. 

The practical difficulties of conducting any survey may introduce 
nonsampling error. For example, differences in how a particular 
question is interpreted, the sources of information available to 
respondents, or the types of people who do not respond can introduce 
unwanted variability into the survey results. In order to reduce 
nonsampling error, we pretested the questionnaire with five risk 
assessors, one from each of the offices surveyed. During these 
pretests, we asked agency officials to complete the survey as we 
observed the process. We then interviewed the respondents to ensure 
that (1) the questions were clear and unambiguous, (2) the terms used 
were precise, (3) the questionnaire did not place an undue burden on 
the agency officials completing it, and (4) the questionnaire was 
independent and unbiased. On the basis of the feedback from the 
pretests, we modified the questions, as appropriate. Information about 
accessing the questionnaire was provided via e-mail for all survey 
participants. The survey was activated, and staff informed of its 
availability on October 17, 2005; it was available until January 13, 
2006. To ensure security and data integrity, we provided all 
participants with a personal password that allowed them to access and 
complete a questionnaire. No one else could access that questionnaire 
or edit its data. We included steps in both the data collection and 
data analysis stages for the purpose of minimizing such nonsampling 
errors. To reduce survey nonresponse, we sent e-mail reminders and 
conducted follow-up telephone calls with nonrespondents. Overall, 82 
percent of the 270 risk assessors in our sampling frame responded to 
our survey, and all offices had a response rate of at least 80 percent. 

We used general modifiers (i.e., many, several, some, a few, and a 
couple) to characterize written responses to some open ended survey 
questions. We used the following method to assign these modifiers to 
our statements about risk assessor's survey responses: "many" 
represents 22 to 44 respondents (roughly 10 to 20 percent), "several" 
represents 12 to 21 respondents (5 to 10 percent), "some" represents 4 
to 11 respondents, "a few" represents 3 respondents, and "a couple" 
represents 2 respondents. These divisions do not represent technically 
established categories; rather, we chose these divisions because they 
aligned with natural breaks in response "themes" highlighted in the 
report. 

To assess further actions EPA could take to improve its risk assessment 
process and to identify barriers it may face in doing so, we 
interviewed experts representing a range of stakeholders in the 
process. Specifically, we contacted risk assessment scientists; 
toxicologists; scientific advisers to EPA; state officials; and 
representatives from regulated industries, government agencies, and 
environmental advocacy groups who have an expertise in risk assessment. 
We used an iterative process (often referred to as the "snowball 
sampling" technique) to identify these knowledgeable experts and 
selected for interviews those who would provide us with a broad and 
balanced range of perspectives on EPA risk assessment practices. 

We first contacted the National Academy of Sciences' Board of 
Environmental Studies and Toxicology, which is the academy's principal 
study unit for environmental pollution problems affecting human health 
and the assessment and management of related risks to human health and 
the environment. We presented our engagement to the board and sought 
its input on the areas in which EPA has made the most progress 
improving its risk assessment practices and areas EPA will need to 
focus on in the future. We also asked members if they would be willing 
to participate in a future interview and solicited the names of other 
experts who would be appropriate for us to contact about this 
engagement. We selected for interviews experts who would provide us 
with a broad and balanced range of perspectives on EPA risk assessment 
practices. We continued interviewing and soliciting names until we 
determined we had appropriate coverage from all the relevant 
stakeholder groups. Our sampling identified 22 experts, listed 
alphabetically, as follows: Elizabeth L. Anderson, Ph.D; Gail Charnley, 
Ph.D; Harvey J. Clewell, M.A; Shannon Cunniff; Kerry Dearfield, Ph.D; 
Michael L. Dourson, Ph.D; Elaine M. Faustman, Ph.D; Paul Gilman, Ph.D; 
Gary Ginsberg, Ph.D; Sherri Goodman, Esq; Judith A. Graham, Ph.D; Dan 
Greenbaum; Leslie J. Hushka, Ph.D; Annie M. Jarabek, B.S; James H. 
Johnson, Ph.D; Elizabeth Julien, Ph.D; Dorothy Patton, Ph.D; Jonathan 
M. Samet, Ph.D; Jennifer Sass, Ph.D; Chris Whipple, Ph.D; Richard 
Wiles, M.A; and Lauren Zeise, Ph.D. 

We used a standard set of questions to interview each of these experts 
to ensure we consistently discussed each aspect of EPA risk assessment 
policies and practices. To develop the questions, we reviewed 
documentation on EPA's risk assessment process and reports prepared by 
the National Academy of Sciences. We pretested our questions with two 
of the experts and refined the questions accordingly. We used content 
analysis to identify the main themes among their responses. In 
addition, we asked the experts for their opinions about the many risk 
assessment modifications EPA has made since 1994, and used content 
analysis to synthesize their comments. 

We conducted our work from February 2005 through March 2006 in 
accordance with generally accepted government auditing standards. 

[End of section] 

Appendix II: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

John B. Stephenson, (202) 512-6225 or stephensonj@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Cheryl Williams (Assistant 
Director), Jennifer Cook, Michelle Cooper, Elizabeth Erdmann, and 
Rebecca Shea made key contributions to this report. Also contributing 
to this report were Nancy Crothers, Richard Frankel, and Roderick 
Moore. 

FOOTNOTES 

[1] Although other regulatory agencies, such as the Food and Drug 
Administration, also use risk assessments as part of regulatory 
decision making, this report discusses only the risk assessment process 
used by EPA. 

[2] EPA's mission is to protect human health and to safeguard the 
natural environment--air, water, and land--upon which life depends. 
This report focuses only on EPA's efforts to protect human health. 

[3] Pollutants are generally categorized as hazardous air pollutants 
under the Clean Air Act if they cause or may cause cancer or other 
serious health effects, such as reproductive effects or birth defects, 
or adverse environmental and ecological effects. Currently, the Clean 
Air Act regulates 187 chemicals and chemical categories as hazardous 
air pollutants. 

[4] National Academy of Sciences, Risk Assessment in the Federal 
Government: Managing the Process (Washington, D.C., 1983). 

[5] The Presidential/Congressional Commission was created by Pub. L. 
No. 101-549 (1990). 

[6] Peer review is a documented critical review of a specific 
scientific or technical work product, conducted by qualified 
individuals who are independent of those who performed the work, but 
who are collectively equivalent in technical expertise--for example, 
EPA's Science Advisory Board or the National Academy of Sciences. 
According to EPA's 2006 Peer Review Policy, peer review can be 
internal, in which the reviewers are independent experts from inside 
EPA, or external, in which the reviewers are independent experts from 
outside EPA. 

[7] National Academy of Sciences, Science and Judgment in Risk 
Assessment (Washington, D.C., 1994). 

[8] National Academy of Sciences, Understanding Risk: Informing 
Decision in a Democratic Society (Washington, D.C., 1996). 

[9] National Academy of Sciences, Strengthening Science at the U.S. 
Environmental Protection Agency (Washington, D.C., 2000). 

[10] EPA's Science Advisory Board provides independent advice and peer 
review on scientific and technical aspects of environmental problems 
and issues. Experts, including scientists, engineers, and economists, 
provide independent, balanced, and scientifically sound advice to EPA. 

[11] IRIS contains information on the human health effects that may 
result from exposure to various chemicals in the environment and helps 
provide consistent information on chemical substances for use in risk 
assessments. 

[12] The Agency for Toxic Substances and Disease Registry is the 
primary public health agency involved with hazardous waste issues and 
works to prevent or reduce the harmful effects of exposure to hazardous 
substances on human health by, for example, supporting research that 
assists with risk assessment. 

[13] Endocrine disrupting chemicals are thought to mimic natural human 
hormones that influence important regulatory and development mechanisms 
such as blood pressure, metabolism, and reproduction. Important 
endocrine glands include the thyroid, pancreas, and male and female 
gonads (testes and ovaries). 

[14] The STAR program funds peer reviewed competitive grants on 
research topics selected by ORD and postgraduate training for 
scientists in environmental fields. 

[15] EPA's defaults represent a choice that, although scientifically 
plausible given the existing uncertainty, is more likely to result in 
overestimating than underestimating human health risk. 

[16] Contaminants can progressively accumulate in the tissues of an 
organism, such as a human or a fish, as a result of uptake by the body 
from all routes of exposure. This process, called bioaccumulation, 
occurs because the rate of intake exceeds the organism's ability to 
eliminate the substance from the body. 

[17] Neurotoxicity is an adverse change in the structure or function of 
the central or peripheral nervous system following exposure to a 
chemical, physical, or biological agent. 

[18] U.S. Environmental Protection Agency, Peer Review Handbook 
(Washington, D.C., 2000). 

[19] ORD's laboratories and centers are the National Center for 
Environmental Assessment (NCEA), National Health and Environmental 
Effects Research Laboratory (NHEERL), National Exposure Research 
Laboratory (NERL), National Center for Environmental Research (NCER), 
National Risk Management Research Laboratory (NRMRL), and National 
Homeland Security Research Center. 

[20] U.S. Environmental Protection Agency, Risk Assessment Principles 
and Practice (Washington, D.C., 2004). 

[21] U.S. Environmental Protection Agency, Guidance on Cumulative Risk 
Assessment--Part 1: Planning and Scoping (Washington, D.C., 1997). 

[22] U.S. Environmental Protection Agency, Lessons Learned on Planning 
and Scoping for Environmental Risk Assessments (Washington, D.C., 
2002). 

[23] These six principal pollutants, known as "criteria pollutants," 
are carbon monoxide, lead, nitrogen dioxide, ozone, particulate matter, 
and sulfur dioxide. 

[24] EPA is reviewing older pesticides (those initially registered 
prior to November 1984) under the Federal Insecticide, Fungicide, and 
Rodenticide Act to ensure they meet current scientific and regulatory 
standards. EPA is also reassessing tolerances (pesticide residue limits 
in food) to ensure they meet safety standards established under the 
Food Quality Protection Act (FQPA) of 1996. Under FQPA, EPA must 
reassess all tolerances established before August 3, 1996, within 10 
years. The Office of Pesticide Programs reviews the registrations of 
all pesticide products every 15 years to determine whether they still 
meet safety standards as part of its registration review process. 

[25] Office of Management and Budget, Guidelines for Ensuring and 
Maximizing the Quality, Objectivity, Utility, and Integrity of 
Information Disseminated by Federal Agencies (Washington, D.C., 2002). 

[26] U.S. Environmental Protection Agency, A Summary of General 
Assessment Factors for Evaluating the Quality of Scientific and 
Technical Information (Washington, D.C., 2003). 

[27] In September 2002, EPA formed the National Homeland Security 
Research Center to, among other things, provide appropriate, 
affordable, effective, and validated technologies and methods for 
assessing risks posed by chemical, biological, and radiological terror 
attacks. 

[28] To comply with the Government Performance and Results Act (GPRA), 
EPA every 3 years generates an agencywide 5-year strategic plan that 
highlights high-level environmental issues. The most recent plan for 
fiscal years 2003-2008 identified key research needs related to EPA's 
mission. 

[29] The four strategic research directions are harmonizing cancer and 
noncancer risk assessments, assessing aggregate and cumulative risk, 
determining risk to susceptible human subpopulations, and conducting 
research to enable evaluation of public health outcomes from risk 
management decisions. 

[30] The Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA), 7 
U.S.C. §§ 136-136y. 

[31] Applicants must submit data on the acute and chronic toxicity of 
the pesticide product under different conditions, such as how and at 
what rate it can be applied. See 40 C.F.R. Part 158 (2005). 

[32] ILSI-RSI is primarily funded through cooperative agreements with 
regulatory agencies such as EPA and Health Canada, but also receives 
financial support from the European Commission, the National Institute 
of Environmental Health Sciences, and industry trade groups. 

[33] EPA uses four criteria to prioritize chemicals for IRIS reviews: 
(1) EPA statutory, regulatory, or program needs; (2) availability of 
new scientific information that might significantly change the current 
information; (3) interest from state and local government or the 
public; and (4) the amount of additional information that would be 
needed to complete the review. Ultimately, the decision to assess any 
given substance hinges on available agency resources. 

[34] The Air Toxics Risk Assessment Reference Library is a multivolume 
technical resource manual that provides information on the fundamental 
principles of risk-based assessment for hazardous air pollutants and 
how to apply those principles in different settings. 

[35] U.S. Environmental Protection Agency, Report of the Agency Task 
Force on Environmental Regulatory Modeling--Guidance, Support Needs, 
Draft Criteria and Charter (Washington, D.C., 1994). 

[36] U.S. Environmental Protection Agency, Model Validation for 
Predictive Exposure Assessments (Washington, D.C., 1994). 

[37] U.S. Environmental Protection Agency, Draft Guidance on the 
Development, Evaluation, and Application of Regulatory Environmental 
Models (Washington, D.C., 2003). 

[38] U.S. Environmental Protection Agency, Policy for Use of 
Probabilistic Analysis in Risk Assessment (Washington, D.C., 2003). 

[39] U.S. Environmental Protection Agency, Children's Total Exposure to 
Persistent Pesticides and Other Persistent Organic Pollutants 
(Washington, D.C., 1999). 

[40] Exec. Order No. 13045, 62 Fed. Reg. 19885 (Apr. 21, 1997). 

[41] U.S. Environmental Protection Agency, Review of the Reference Dose 
and Reference Concentration Processes (Washington, D.C., 2002). 

[42] U.S. Environmental Protection Agency, Supplementary Guidance for 
Conducting Health Risk Assessment of Chemical Mixtures (Washington, 
D.C., 2000). 

[43] U.S. Environmental Protection Agency, Framework for Cumulative 
Risk Assessment (Washington, D.C., 2003). 

[44] See § 405 of the Food Quality Protection Act of 1996, amending 21 
U.S.C. § 346a(b)(2)(D)(vi). 

[45] U.S. Environmental Protection Agency, General Principles for 
Performing Aggregate Exposure and Risk Assessments (Washington, D.C., 
2001). 

[46] U.S. Environmental Protection Agency, Guidance on Cumulative Risk 
Assessment of Pesticide Chemicals That Have a Common Mechanism of 
Toxicity (Washington, D.C., 2002). 

[47] For example, metals released into the air may be deposited on the 
ground, where they remain in surface soils for long periods of time. 
The chemicals in the soil may be taken up into plants through the roots 
and accumulate in the tissues of foraging animals. 

[48] By statute, hazardous air pollutants are regulated not as 
individual pollutants but by emission sources that consist of a group 
of similar industrial processes or industries that release multiple 
pollutants. 

[49] Guidelines refer to the Guidelines for Carcinogen Risk Assessment, 
Guidelines for Neurotoxicity Risk Assessment, Guidelines for 
Reproductive Toxicity Risk Assessment, and Supplemental Guidelines for 
Chemical Mixtures. Reference documents refer to the Assessment Factors 
Handbook, Exposure Factors Handbook, Framework for Cumulative Risk 
Assessment, Guiding Principles for Monte Carlo Analysis, Peer Review 
Handbook, Review of Reference Dose and Reference Concentration 
Processes, Risk Assessment Principles and Practices, and Risk 
Characterization Handbook. Policy documents refer to the Policy on 
Evaluating Health Risks to Children, Policy for Use of Probabilistic 
Analysis in Risk Assessment, Interim Genomics Policy, and Interim 
Position on Environmental Endocrine Disruption. 

[50] The National Toxicology Program is an interagency program 
established by the Department of Health and Human Services that 
provides information about potentially toxic chemicals to health, 
regulatory, and research agencies, scientific and medical communities, 
and the public. 

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, 

NelliganJ@gao.gov 

(202) 512-4800 

U.S. Government Accountability Office, 

441 G Street NW, Room 7149 

Washington, D.C. 20548: