This is the accessible text file for GAO report number GAO-03-846 
entitled 'Grants Management: EPA Needs to Strengthen Efforts to Address 
Persistent Challenges' which was released on September 25, 2003.

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

Report to Congressional Requesters:

August 2003:

Grants Management:

EPA Needs to Strengthen Efforts to Address Persistent Challenges:

GAO-03-846:

GAO Highlights:

Highlights of GAO-03-846, a report to congressional requesters

Why GAO Did This Study:

The Environmental Protection Agency (EPA) has long faced problems 
managing its grants, which constitute over one-half of the agency’s 
annual budget, or about $4 billion. EPA uses grants to implement its 
programs to protect human health and the environment and awards grants 
to thousands of recipients, including state and local governments, 
tribes, universities, and nonprofit organizations. EPA’s ability to 
efficiently and effectively accomplish its mission largely depends on 
how well it manages its grant resources. As requested, GAO determined 
(1) major challenges EPA faces in managing its grants and how it has 
addressed these challenges in the past, (2) extent to which EPA’s 
recently issued policies and grants management plan address these 
challenges, and (3) promising practices, if any, that could assist EPA 
in addressing these challenges.  

What GAO Found:

EPA continues to face four key management challenges, despite past 
efforts to address them. These challenges are (1) selecting the most 
qualified applicants, (2) effectively overseeing grantees, (3) 
measuring the results of grants, and (4) effectively managing grants 
staff and resources. In recent years, EPA has taken a series of 
actions to address these challenges by, among other things, issuing 
policies, conducting training, and developing a new data system for 
grants management. However, these past actions had mixed results 
because of the complexity of the problems, weaknesses in design and 
implementation, and insufficient management emphasis.

EPA’s 2002 competition and oversight policies and 2003 grants 
management plan focus on the major challenges GAO identified, but will 
require strengthening, enhanced accountability, and a sustained 
commitment to succeed. For example, EPA’s new oversight policy 
mandates more in-depth monitoring of grantees but it does not build in 
a process for analyzing the results of the in-depth monitoring to 
address systemic grantee problems. Such analysis could better target 
EPA’s oversight efforts. In addition, its 5-year grants management 
plan does offer, for the first time, a comprehensive road map with 
objectives, goals, and milestones for addressing grants management 
challenges. However, the plan does not completely address how EPA will 
hold all managers and staff accountable for successfully fulfilling 
their management responsibilities. Therefore, EPA cannot ensure the 
sustained commitment required for the plan’s success. 

Although information on promising grants management practices is 
limited, the federal agencies and other organizations GAO contacted 
identified some practices for each of EPA’s four major challenges that 
may further assist EPA in improving its grants management. For 
example, one federal agency takes into account applicants’ potential 
to achieve results when selecting grantees. A private foundation 
conducts preaward reviews of some applicants to learn about the 
organization and assess its fiscal health. In addition, GAO has 
developed a guide for federal agencies to use to hold managers and 
staff accountable for achieving desired agency results. This guide 
could be useful in helping EPA ensure accountability for grants 
management performance.

What GAO Recommends:

The Administrator of EPA should ensure that the agency (1) meets 
goals, targets, and time frames in its grants management plan, and (2) 
takes specific actions to strengthen its efforts and report annually 
to Congress on its grant management accomplishments.

In commenting on a draft of this report, EPA stated that it agreed 
with GAO’s recommendations and will implement them as part of its 5-
year grants management plan.

www.gao.gov/cgi-bin/getrpt?GAO-03-846.

To view the full product, including the scope and methodology, click 
on the link above. For more information contact John B. Stephenson at 
(202) 512-3841 or stephensonj@gao.gov.

[End of section]

Contents:

Letter: 

Results in Brief: 

Background: 

Four Key Grants Management Challenges Persist despite EPA's Past 
Efforts to Address Them: 

New Policies and Plan Show Promise but Require Strengthening, Enhanced 
Accountability, and Sustained Commitment to Succeed: 

Promising Practices Could Assist EPA in Addressing Its Grants 
Management Challenges: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendixes:

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: EPA'S 10 Regions and Regional Office Locations: 

Appendix III: Results of Our Analysis of EPA'S 2002 In-Depth Reviews: 

Analysis of Number and Types of EPA's In-depth Reviews: 

Analysis of Problems Identified in EPA's In-depth Reviews: 

Analysis of Corrective Actions Identified in EPA's In-depth Reviews: 

Analysis of Protocols Used for EPA's In-depth Reviews: 

Analysis of Significant Actions Taken Against Grantees as a Result of 
In-depth Reviews: 

Appendix IV: Summary of EPA's Grants Management Plan, 2003-2008: 

Goal 1: Enhance the Skills of EPA Personnel Involved in Grants 
Management: 

Goal 2: Promote Competition in the Award of Grants: 

Goal 3: Leverage Technology to Improve Program Performance: 

Goal 4: Strengthen EPA Oversight of Grants: 

Goal 5: Support Identifying and Achieving Environmental Outcomes: 

Appendix V: Comments from the Environmental Protection Agency: 

Appendix VI: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

Staff Acknowledgments: 

Related GAO Products:

Tables:

Table 1: Number of In-depth Reviews by Type of Grantee and Review, 
2002: 

Table 2: Most Frequently Identified Problems, by Problem Area for 
Administrative Reviews, 2002: 

Table 3:  Most Frequently Identified Problems, by Problem Area for 
Programmatic Reviews, 2002: 

Table 4: Description of Most Frequently Identified Problems in EPA's In-
depth Reviews: 

Table 5: Number of Reviews by Type of Reviews: 

Table 6: Number of Off-site and On-site Reviews and Number of Reviews 
EPA Indicated It Performed, by Program and Regional Office: 

Table 7: Number of Problems Identified by Type of Review: 

Table 8: Number of Problems by Type of Grantee: 

Table 9: Number of Problems by Dollar Amount of Grants: 

Table 10: Number of Problems by Type of Review: 

Table 11: Number of Problems Per In-depth Review: 

Table 12: Description of Problems Identified in EPA's Reviews: 

Table 13: Number of Problems and Corrective Actions: 

Table 14: Number of Mandated and Nonmandated Reviews: 

Table 15: Number of Nonmandated Reviews With or Without a Review 
Protocol, by Program and Regional Office, and by Type of Review: 

Table 16: Number of Significant Actions Taken Against Grantees by Type 
of Grantee: 

Table 17: Number of Significant Actions Per In-depth Review: 

Table 18: Goal 1 Performance Measures: 

Table 19: Goal 2 Performance Measures: 

Table 20: Goal 3 Performance Measures: 

Table 21: Goal 4 Performance Measures: 

Table 22: Goal 5 Performance Measures: 

Figures:

Figure 1: EPA's Key Offices Involved in Grant Activities: 

Figure 2: Percentage of EPA Grant Dollars Awarded by Recipient Type, 
Fiscal Year 2002: 

Abbreviations: 

CFDA: Catalog of Federal Domestic Assistance:

EPA: Environmental Protection Agency:

HHS: Department of Health and Human Services:

IFMS: Integrated Financial Management System:

IGMS: Integrated Grants Management System:

OMB: Office of Management and Budget:

PART: Program Assessment Rating Tool:

SAMHSA: Substance Abuse and Mental Health Services Administration:

STAR: Science To Achieve Results:

Letter August 29, 2003:

The Honorable Don Young 
Chairman, 
Committee on Transportation and Infrastructure 
House of Representatives:

The Honorable Anne M. Northup 
House of Representatives:

The Environmental Protection Agency (EPA) has faced persistent 
challenges in managing its grants, which constitute over one-half of 
the agency's budget, or about $4 billion annually. To support its 
mission of protecting human health and the environment, in fiscal year 
2002, EPA awarded grants to a variety of recipients, including state 
and local governments, tribes, universities, and nonprofit 
organizations. As of September 30, 2002, there were 4,100 grant 
recipients. Given the size and diversity of EPA's programs, its ability 
to efficiently and effectively accomplish its mission largely depends 
on how well it manages its grant resources and builds accountability 
into its efforts.

Congressional hearings in 1996, 1999, and 2003 have focused on EPA's 
problems in effectively managing its grants. We and EPA's Inspector 
General have reported on a number of weaknesses throughout the grants 
management process--from awarding grants to measuring grant 
results.[Footnote 1] EPA's efforts to address its grants management 
problems have not fully resolved them. Because EPA has faced persistent 
problems in managing grants, we designated its grants management as a 
major management challenge in January 2003.[Footnote 2]

Late in 2002, EPA launched new efforts to address some of its grants 
management problems. It issued two new policies--one to promote 
competition in awarding grants and one to improve its oversight of 
grants. Furthermore, in April 2003, EPA issued a 5-year grants 
management plan to address its long-standing grants management 
problems.

You asked us to determine the (1) major challenges EPA faces in 
managing its grants and how it has addressed these challenges in the 
past, (2) extent to which EPA's recently issued policies and grants 
management plan address these challenges, and (3) promising practices, 
if any, that could assist EPA in addressing these challenges.

To identify the challenges EPA faces in managing its grants and to 
examine how it has addressed these challenges in the past, we (1) 
analyzed 93 reports on EPA's grants management, including our reports, 
EPA's Inspector General reports, and EPA's internal management reviews 
conducted from 1996 through 2003, (2) systematically reviewed and 
recorded information from the 1,232 records of calendar year 2002 in-
depth reviews of grantee performance--from financial management to 
progress in achieving grant objectives, and (3) interviewed EPA 
officials and reviewed documents obtained from them.[Footnote 3] To 
determine the extent to which EPA's recently issued policies and grants 
management plan address these challenges, we (1) reviewed the new 
policies and plan and interviewed EPA officials responsible for key 
aspects of the plan, (2) attended EPA's grants management training 
courses, and (3) observed five EPA in-depth reviews of grantees. To 
identify what promising practices, if any, could assist EPA in 
addressing these challenges, we interviewed officials in six federal 
agencies, including the five largest grants-awarding agencies, and in 
private sector organizations and reviewed documentation from them. We 
also reviewed two GAO reports that describe effective training and 
performance management systems for federal agencies. Our detailed scope 
and methodology are discussed in appendix I.

Results in Brief:

EPA continues to face four key grants management challenges, despite 
past efforts to address them. These challenges are (1) selecting the 
most qualified grant applicants, (2) effectively overseeing grantees, 
(3) measuring the results of grants, and (4) effectively managing grant 
staff and resources. In recent years, EPA took a series of actions to 
address these challenges by, among other things, issuing policies, 
conducting training, and developing a new data system for grants 
management. However, these actions had mixed results because of the 
complexity of the problems, weaknesses in design and implementation, 
and insufficient management attention. For example, EPA's efforts to 
improve oversight included in-depth monitoring of grantees but did not 
include a statistical approach to identifying grantees for reviews, 
collecting standard information from the reviews, and analyzing the 
results to identify systemic grants management problems. EPA, 
therefore, could not be assured that it was identifying and resolving 
grantee problems and using its resources to more effectively target its 
oversight efforts.

EPA's 2002 competition and oversight policies and 2003 grants 
management plan focus on the major grants management challenges we 
identified but will require strengthening, enhanced accountability, and 
a sustained commitment to succeed. Specifically, the new competition 
policy should encourage EPA to select the most qualified applicants by 
requiring competition for more grants. However, effective 
implementation of the policy will require a major cultural shift for 
EPA managers and staff because the competitive process will require 
significant planning and take more time than awarding grants 
noncompetitively. Furthermore, the new oversight policy, while now 
requiring more in-depth monitoring to identify problems, still does not 
address the problem we noted in EPA's earlier polices--the need for EPA 
to develop information on systemic problems with grantees. Finally, 
EPA's new grants management plan does offer, for the first time, a 
comprehensive road map with goals, objectives, and milestones for 
addressing grants management challenges. However, while the plan 
recognizes the importance of accountability, it does not completely 
address how EPA will hold all managers and staff accountable for 
successfully fulfilling their grants responsibilities. For example, the 
plan does not call for including grants management performance 
standards in managers' and supervisors' performance agreements. Without 
holding all managers and staff accountable for grants management, EPA 
cannot ensure the sustained commitment required for the plan's success.

Although information on promising grants management practices is 
limited, the federal agencies and other organizations we contacted 
identified some practices for each of EPA's four major challenges that 
may further assist EPA in improving its grants management. For example, 
one federal agency takes into account applicants' potential to achieve 
results when selecting grantees. A private foundation conducts preaward 
reviews of some applicants to learn about the organization and assess 
its fiscal health. In addition, we have provided a guide for federal 
agencies to use in developing a comprehensive performance management 
system to hold managers and staff accountable for achieving desired 
agency results. This guide could be useful in helping EPA ensure 
accountability for grants management performance.

We are making recommendations to the EPA Administrator to strengthen 
grants management, specifically in overseeing grantees, measuring 
environmental outcomes, incorporating accountability for grants 
management responsibilities, considering promising practices, and 
reporting on the progress of its efforts in its annual report to 
Congress.

We provided a draft of this report to EPA for its review and comment. 
EPA stated that it agreed with our recommendations and will implement 
them as part of its 5-year grants management plan. EPA's comments are 
presented in appendix V.

Background:

EPA administers and oversees grants primarily through the Office of 
Grants and Debarment, 10 program offices in headquarters,[Footnote 4] 
and program offices and grants management offices in EPA's 10 regional 
offices. Figure 1 shows EPA's key offices involved in grants activities 
for headquarters and the regions. (See app. II for a map of EPA's 
regional office locations.):

Figure 1: EPA's Key Offices Involved in Grant Activities:

[See PDF for image]

[End of figure]

The management of EPA's grants program is a cooperative effort 
involving the Office of Administration and Resources Management's 
Office of Grants and Debarment, program offices in headquarters, and 
grants management and program offices in the regions. The Office of 
Grants and Debarment develops grant policy and guidance. It also 
carries out certain types of administrative and financial functions for 
the grants approved by the headquarters program offices, such as 
awarding grants and overseeing the financial management of these 
grants. On the programmatic side, headquarters program offices 
establish and implement national policies for their grant programs, and 
set funding priorities. They are also responsible for the technical and 
programmatic oversight of their grants. In the regions, grants 
management offices carry out certain administrative and financial 
functions for the grants, such as awarding grants approved by the 
regional program offices,[Footnote 5] while the regional program staff 
provide technical and programmatic oversight of their grantees.

As of June 2003, 109 grants specialists in the Office of Grants and 
Debarment and the regional grants management offices were largely 
responsible for administrative and financial grant functions. 
Furthermore, 1,835 project officers were actively managing grants in 
headquarters and regional program offices. These project officers are 
responsible for the technical and programmatic management of grants. 
Unlike grant specialists, however, project officers generally have 
other primary responsibilities, such as using the scientific and 
technical expertise for which they were hired.

In fiscal year 2002, EPA took 8,070 grant actions[Footnote 6] totaling 
about $4.2 billion.[Footnote 7] These awards were made to six main 
categories of recipients as shown in figure 2.

Figure 2: Percentage of EPA Grant Dollars Awarded by Recipient Type, 
Fiscal Year 2002:

[See PDF for image]

[End of figure]

EPA offers two types of grants--nondiscretionary and discretionary:

* Nondiscretionary grants support water infrastructure projects, such 
as the drinking water and clean water state revolving fund programs, 
and continuing environmental programs, such as the Clean Air Program 
for monitoring and enforcing Clean Air Act regulations. For these 
grants, Congress directs awards to one or more classes of prospective 
recipients who meet specific eligibility criteria; the grants are often 
awarded on the basis of formulas prescribed by law or agency 
regulation. In fiscal year 2002, EPA awarded about $3.5 billion in 
nondiscretionary grants. EPA awarded these grants primarily to states 
or other governmental entities.

* Discretionary grants fund a variety of activities, such as 
environmental research and training. EPA has the discretion to 
independently determine the recipients and funding levels for these 
grants. In fiscal year 2002, EPA awarded about $719 million in 
discretionary grants. EPA awarded discretionary grants primarily to 
nonprofit organizations, universities, and government entities.

The grant process has the following four phases:

* Preaward. EPA reviews the application paperwork and makes an award 
decision.

* Award. EPA prepares the grant documents and instructs the grantee on 
technical requirements, and the grantee signs an agreement to comply 
with all requirements.

* Postaward. After awarding the grant, EPA provides technical 
assistance, oversees the work, and provides payments to the grantee; 
the grantee completes the work, and the project ends.

* Closeout of the award. EPA ensures that all technical work and 
administrative requirements have been completed; EPA prepares closeout 
documents and notifies the grantee that the grant is completed.

EPA's grantees are subject to the same type of financial management 
oversight as the recipients of other federal assistance. Specifically, 
the Single Audit Act[Footnote 8] requires grantees to have an audit of 
their financial statements and federal awards or a program specific 
audit if they spend $300,000 or more in federal awards in a fiscal 
year.[Footnote 9] Grantees submit these audits to a central 
clearinghouse operated by the Bureau of the Census, which then forwards 
the audit findings to the appropriate agency for any necessary action. 
However, the act does not cover all grants and all aspects of grants 
management and, therefore, agencies must take additional steps to 
ensure that federal funds are spent appropriately.

In addition, EPA conducts in-depth reviews to analyze grantees' 
compliance with grant regulations and specific grant 
requirements.[Footnote 10] Furthermore, to determine how well offices 
and regions oversee grantees, EPA conducts three types of internal 
management reviews that address grants management: (1) Management 
Oversight Reviews, which assess grants management activities in 
regional offices; (2) Management Effectiveness Reviews, which are self-
assessments of grants management activities performed by EPA's 
headquarters and regional offices; and (3) Post-Award Validation 
Reviews, which assess the effectiveness of initiatives EPA has taken to 
improve oversight efforts.

EPA has faced persistent problems in managing its grants. EPA's 
Inspector General testified before Congress in 1996 and again in 1999 
that EPA did not fulfill its obligation to properly monitor grants. 
Acknowledging these problems, EPA identified oversight, including grant 
closeouts, as a material weakness--an accounting and internal control 
system weakness that the EPA Administrator must report to the President 
and Congress.[Footnote 11] EPA's fiscal year 1999 Federal Managers' 
Financial Integrity Act report indicated that this oversight material 
weakness had been corrected, but the Inspector General testified that 
the weakness continued. In 2002, the Inspector General again 
recommended that EPA designate grants management as a material 
weakness. The Office of Management and Budget (OMB) also recommended in 
2002 that EPA designate grants management as a material weakness. In 
its fiscal year 2002 Annual Report,[Footnote 12] EPA ultimately decided 
to maintain this issue as an agency-level weakness, which is a lower 
level of risk than a material weakness. EPA reached this decision 
because it believes its ongoing corrective action efforts will help to 
resolve outstanding grants management challenges. However, in adding 
EPA's grants management to our list of EPA's major management 
challenges in January 2003, we signaled our concern that EPA has not 
yet taken sufficient action to ensure that it can manage its grants 
effectively.

Four Key Grants Management Challenges Persist despite EPA's Past 
Efforts to Address Them:

We identified four key challenges that EPA continues to face in 
managing its grants despite efforts to address them. These challenges 
are (1) selecting the most qualified grant applicants, (2) effectively 
overseeing grantees, (3) measuring the results of grants, and (4) 
effectively managing grant staff and resources. In recent 
years,[Footnote 13] EPA took a series of actions to address these 
challenges by, among other things, issuing policies on competition and 
oversight, conducting training for project officers and nonprofit 
organizations, and developing a new data system for grants management. 
However, these actions had mixed results because of the complexity of 
the problems, weaknesses in design and implementation, and insufficient 
management attention.

EPA Has Not Always Selected the Most Qualified Applicants despite 
Issuing a Competition Policy:

EPA has not always ensured that the most qualified applicants receive 
grant awards. For discretionary grants, the agency requires project 
officers to evaluate grant applications and recommend funding those 
that will most effectively contribute to EPA's program objectives and 
priorities. But project officers have not always performed all of the 
steps required to review grant applications. According to a 2003 
Inspector General report,[Footnote 14] project officers did not always, 
among other things, (1) ensure a link between the project funded and 
EPA's mission, (2) assess the probability of success prior to the 
award, and (3) determine the reasonableness of proposed project costs. 
As a result, EPA may have lost the opportunity to fund other projects 
that could have better achieved its mission.

The Federal Grant and Cooperative Agreement Act of 1977[Footnote 15] 
encourages agencies to use competition in awarding grants. To encourage 
competition, EPA issued a grants competition policy in 1995. However, 
EPA's policy did not result in meaningful competition throughout the 
agency, according to EPA officials.[Footnote 16] Furthermore, EPA's own 
internal management reviews and a 2001 Inspector General report found 
that EPA has not always encouraged competition.[Footnote 17]

Finally, EPA has not always engaged in widespread solicitation when it 
could be beneficial to do so. Widespread solicitation would provide 
greater assurance that EPA receives proposals from a variety of 
eligible and highly qualified applicants who otherwise may not have 
known about grant opportunities. According to a 2001 EPA Inspector 
General report,[Footnote 18] program officials indicated that 
widespread solicitation was not necessary because "word gets out" to 
eligible applicants. Applicants often sent their proposals directly to 
these program officials, who funded them using "uniquely qualified" as 
the justification for a noncompetitive award. This procedure created 
the appearance of preferential treatment by not offering the same 
opportunities to all potential applicants. In addition, the agency 
provided incomplete or inconsistent public information on its grant 
programs in the Catalog of Federal Domestic Assistance. Therefore, 
potential applicants may not have been adequately informed of funding 
opportunities.

EPA Has Not Always Effectively Overseen Grant Recipients despite Past 
Actions to Improve Oversight:

EPA has not always effectively overseen grant recipients despite 
efforts to improve monitoring. To address oversight problems, EPA 
issued a series of policies starting in 1998. However, these oversight 
policies have had mixed results in addressing this challenge.

EPA Has Not Always Effectively Overseen Grant Recipients:

EPA's oversight efforts have not always been proactive or well 
documented, or ensured compliance with grant requirements and 
objectives. According to EPA's 2001 internal management reviews, 
project officers had minimal contact with grantees, limiting their 
oversight to reviewing grantees' progress reports. However, project 
officers did not proactively ensure the recipient submitted the reports 
on time or that the reports included the required financial 
information. According to our analysis of EPA's 2002 in-depth reviews, 
15 percent of the reviews identified a problem with grantees' 
submitting complete progress reports on time.

Furthermore, EPA's guidance emphasizes the importance of documenting 
oversight efforts, in part, because a lack of documentation raises 
questions about the extent of the monitoring conducted. However, 
project officers and grants specialists have not always documented 
their monitoring activities. According to a 2002 EPA internal 
management review, for example, one grants management office developed 
a form to make it easier to ensure monitoring activities were 
completed, but the form was missing from 50 percent of the grant files 
reviewed. Even when staff used the monitoring form, they did not always 
complete it. Furthermore, project officers did not always document that 
they had monitored the required five key areas under the policy, such 
as ensuring compliance with the terms and conditions of the grant 
award.[Footnote 19]

Finally, EPA has not effectively ensured that grantees comply with the 
following grant requirements and objectives:

* Adequate financial and internal controls. In 2001,[Footnote 20] we 
reported that EPA's oversight of nonprofit grantees' costs did not 
ensure that grantees used funds for costs allowed under OMB's 
guidance.[Footnote 21] For example, EPA did not include transaction 
testing, which can identify unallowable expenditures, such as lobbying.

* Compliance with grant regulations. In 2002, according to the 
Inspector General, EPA did not monitor grantees' procurements to 
determine if the grantees were using a competitive process to obtain 
the best products, at the best price, from the most qualified 
firms.[Footnote 22] In addition, in 1999 and again in 2002, the 
Inspector General reported conflict-of-interest problems because grant 
recipients had awarded contracts to parties who had helped them prepare 
their grants and, therefore, had advance knowledge about grantees' 
plans to award contracts.[Footnote 23] Our review of EPA's 2002 in-
depth reviews also found that a number of compliance problems persist. 
(See table 7 in app. III.) For example, EPA found accounting problems 
in 37 of 189 administrative reviews and procurement problems in 70 of 
189 of these reviews.

* Meeting grant objectives. EPA has not fully ensured that recipients 
are meeting grant objectives. For example, in 2000, we reported that 
EPA had not adequately tracked its Science To Achieve Results (STAR) 
research grants to ensure that the grantees had submitted the results 
of their scientific investigations--the objective of the grant--in a 
timely manner.[Footnote 24] We found that 144 of the nearly 200 grants 
we reviewed had missed their deadline for submitting final reports, 
even after some extensions had been given.

EPA's Past Actions to Improve Oversight Have Had Mixed Results:

To address oversight problems, EPA issued three oversight policies--in 
1998, 1999, and February 2002--designed to promote proactive and well-
documented oversight of grantees to ensure compliance with grant 
regulations and objectives. For the first time, these policies required 
the following:

* Baseline monitoring of all grantees at least once during the life of 
the grant. EPA staff are responsible for contacting the grantee, 
reviewing the grantee's reports, assessing compliance with terms and 
conditions, and monitoring payments to the grantee.

* In-depth reviews for at least 5 to 10 percent of grant recipients. 
Both grants management offices and program offices conduct these 
reviews either at the grantee's location (on-site) or at EPA's office 
or another location (off-site). The grant management offices conduct 
administrative reviews, which are designed to evaluate grantees' 
financial and administrative capacity. The program offices conduct 
programmatic reviews, which are designed to assess the grantees' 
activities in five key areas: assessing progress of work, reviewing 
financial expenditures, meeting the grant's terms and conditions, 
meeting all programmatic and statutory and regulatory requirements, and 
verifying that equipment purchased under the award is managed and 
accounted for. The policy required that programmatic staff use a 
suggested protocol in conducting their in-depth reviews. Furthermore, 
the policy included suggested criteria for selecting grantees to be 
reviewed.

* Annual monitoring plans submitted by headquarters and regional 
offices that outline the offices' proposed oversight activities. The 
policy also suggested activities to be included in these plans, such as 
monitoring grantees' progress of work, documenting their efforts, and 
closing out grants in a timely manner.

* Grantee compliance database--developed by the Office of Grants and 
Debarment--that the grants management offices would use to store the 
in-depth reviews.

We found that EPA's design and implementation of its actions to improve 
grantee oversight have had mixed results. In particular, we found both 
improvements and weaknesses in EPA's in-depth reviews and its ability 
to track grantee compliance.

On the positive side, EPA's policies required headquarters and regional 
offices to conduct a minimum number of in-depth reviews. According to 
our analysis of EPA's in-depth reviews, the agency conducted 1,232 
reviews in 2002. Table 1 shows that these reviews were conducted almost 
equally on-site and off-site, and for a variety of different types of 
grantees.

Table 1: Number of In-depth Reviews by Type of Grantee and Review, 
2002:

Type of grantee: Nonprofits; Type of review: Off-site: 110; 
Type of review: On-site: 134; Total reviews: 244; Percentage 
of total reviews: 20.

Type of grantee: State; Type of review: Off-site: 140; Type of 
review: On-site: 198; Total reviews: 338; Percentage of total 
reviews: 27.

Type of grantee: Native American tribes; Type of review: Off-
site: 190; Type of review: On-site: 137; Total reviews: 327; 
Percentage of total reviews: 27.

Type of grantee: Local government; Type of review: Off-site: 
82; Type of review: On-site: 44; Total reviews: 126; 
Percentage of total reviews: 10.

Type of grantee: University; Type of review: Off-site: 105; 
Type of review: On-site: 55; Total reviews: 160; Percentage of 
total reviews: 13.

Type of grantee: Other; Type of review: Off-site: 17; Type of 
review: On-site: 20; Total reviews: 37; Percentage of total 
reviews: 3.

Type of grantee: Total; Type of review: Off-site: 644; Type of 
review: On-site: 588; Total reviews: 1,232; Percentage of 
total reviews: 100.

Source: GAO analysis of EPA's in-depth reviews.

[End of table]

EPA also more evenly distributed, and increased the number of, on-site 
reviews among its program and regional offices between 2001 and 2002. 
According to the Inspector General, EPA conducted 466 on-site, in-depth 
reviews in 2001, but 265 of these, or 57 percent, were conducted by 
only two of EPA's offices.[Footnote 25] However, according to our 
analysis of the 2002 in-depth reviews, on-site reviews were more evenly 
distributed: 588 on-site reviews, with the two EPA offices that 
conducted the most reviews representing approximately 28 percent of the 
total. (See table 6 in app. III.):

Furthermore, about half of EPA's in-depth reviews found problems with 
grantees. (See table 11 in app. III.) According to our analysis, in 
2002, EPA grants specialists and project officers identified 1,250 
problems in 21 areas. (See table 7 in app. III.) Tables 2 and 3 show 
the most frequently identified problems for the 189 administrative and 
1,017 programmatic reviews. For example, 73 of 189 administrative 
reviews found problems with grantees' written procedures, while 308 of 
the 1,017 programmatic reviews identified technical issues.

Table 2: Most Frequently Identified Problems, by Problem Area for 
Administrative Reviews, 2002:

Type of problem: Written procedures; Number of reviews with reported 
problem: 73.

Type of problem: Procurement; Number of reviews with reported problem: 
70.

Type of problem: Personnel/payroll; Number of reviews with reported 
problem: 51.

Type of problem: Accounting; Number of reviews with reported problem: 
37.

Source: GAO analysis of EPA's in-depth reviews.

[End of table]

Table 3: Most Frequently Identified Problems, by Problem Area for 
Programmatic Reviews, 2002:

Type of problem: Technical issues; Number of reviews with reported 
problem: 308.

Type of problem: Progress reports; Number of reviews with reported 
problem: 167.

Type of problem: Personnel/payroll; Number of reviews with reported 
problem: 92.

Type of problem: Quality assurance; Number of reviews with reported 
problem: 71.

Source: GAO analysis of EPA's in-depth reviews.

[End of table]


The differences in types of problems frequently identified, as shown in 
tables 2 and 3, reflect differences in the focus of administrative and 
programmatic reviews. Table 4 describes the nature of these problems. 
(See table 12 in app. III for a description of all the problems.):

Table 4: Description of Most Frequently Identified Problems in EPA's 
In-depth Reviews:

Problem: Accounting; Types of problems included in EPA's in-depth 
reviews: Any failure of a grantee's financial management system or 
shortcomings in the procedures it used to ensure the proper accounting 
of federal funds. For example, EPA found cases in which a grantee: * 
could not compare budgeted amounts to actual expenditures, * did not 
properly reconcile report balances to the general ledger, or; * did not 
separately track funds for different grants.

Problem: Personnel/; payroll; Types of problems included in EPA's in-
depth reviews: Problems varied depending on the type of review 
conducted. Administrative reviews included cases in which a grantee did 
not track the amount of time its employees spent on specific grant 
activities. Programmatic reviews included cases in which grantees did 
not have sufficient staff resources to perform the grant activities.

Problem: Procurement; Types of problems included in EPA's in-depth 
reviews: Grantees lacked documentation to support sole-source 
contracts, and grantees did not report their efforts to encourage 
procurement from minority-and woman-owned businesses.

Problem: Progress reports; Types of problems included in EPA's in-depth 
reviews: A grantee's progress report was missing, late, or did not 
include all the necessary information.

Problem: Quality assurance; Types of problems included in EPA's in-
depth reviews: A grantee needed to revise its quality assurance plan, 
which is required to ensure the quality of data collected during the 
grant work.

Problem: Technical issues; Types of problems included in EPA's in-depth 
reviews: A grantee was behind in the progress of his or her work.

Problem: Written procedures; Types of problems included in EPA's in-
depth reviews: A grantee's written policies or procedures were either 
missing or inadequate.

Source: GAO analysis of EPA's in-depth reviews.

[End of table]

In response to these problems, EPA required or recommended that 
grantees take corrective actions for about half, or 640, of the 1,250 
problems identified. (See table 13 in app. III.) For example, EPA found 
that one grantee had not adequately separated duties to provide checks 
and balances with the grantee's organization. That is, the executive 
director was also the project manager, payment authorizer, contract 
executor, and payroll officer. EPA recommended that the organization 
separate these responsibilities to ensure proper handling of federal 
funds.

EPA also took more significant actions against some grantees. These 
significant actions include the following:

* Suspension order--EPA requires a grantee to stop work and take action 
to minimize its grant-related expenditures until it resolves EPA's 
areas of concern.

* Payment holds--EPA decides it will not make any more payments to a 
grantee until the grantee has resolved an area of concern.

* Disallowance of costs--EPA disallows a grantee's expenditures when it 
determines that the grantee did not spend its grant funds in accordance 
with federal cost principles or its particular grant agreement.

* Debarment--EPA declares that a particular organization or individual 
is not eligible to receive grants for a specific period of time.

* Termination--EPA cancels the grant.

In our analysis of EPA's 2002 in-depth reviews, we identified 47 
significant actions against its grantees. The most frequent action 
taken was withholding payment: 15 of the 47 significant actions. In no 
cases did EPA debar a grantee. Nonprofit organizations and tribes each 
received 21 of these actions, states received 5, and local governments 
and universities received none. (See table 16 in app. III.) EPA's 
significant actions included the following, for example:

* Region 8 issued a payment hold against one of its tribal grantees 
after questioning costs because the recipient appeared to be performing 
work outside the scope of the grant agreement. The region determined 
that the remaining $27,459 in the grantee's account should be withheld 
until the tribe submitted documentation to demonstrate that the 
questioned costs were allowable.

* Region 3 disallowed costs for one of its nonprofit grantees. The 
region disallowed $51,085 based on a conflict-of-interest, lack of 
competition in the subagreement, and lack of supporting documentation 
in accounting records.

* The Grants Administration Division terminated a grant for a nonprofit 
grantee. EPA took this action because the grantee actually had no paid 
employees of its own but instead used the employees of a for-profit 
company to conduct the work. This arrangement created an apparent 
conflict of interest.

* Region 4 withheld $40,000 of funding from a state because it found 
that the state had not agreed to fully implement its responsibilities 
as defined by the grant.

EPA data also showed that the agency took other significant actions in 
2002 that did not result from in-depth reviews. According to our 
analysis, EPA took 24 other significant actions because of problems 
identified during an investigation by EPA's Inspector General or during 
oversight of grantees in actions other than in-depth reviews.

On another positive note, EPA improved its closeout of grants after it 
developed specific procedures for closing out nonconstruction grants 
and identified a strategy for closing out construction grants.[Footnote 
26] This strategy assessed impediments to closing out grants and 
actions to address these impediments. EPA had successfully resolved its 
backlog by 2002. As a result, EPA has eliminated this backlog as a 
material weakness, and thus it better ensures that grant commitments 
have been met.

Despite these positive steps, EPA's actions did not address the need to 
develop information on systemic problems with grantees to better target 
its oversight efforts--a weakness in policy design. EPA could not 
develop systemic information for three reasons. First, EPA did not use 
a statistical approach to selecting grantees for in-depth reviews. In 
particular, EPA allowed each of its offices to determine how many in-
depth reviews the office would conduct--from at least 5 to 10 percent 
of grantees--and what criteria the office would use to select grantees 
for in-depth reviews. The policy's flexibility in selecting grantees 
was premised on the belief that those closest to grantees, such as a 
program or regional office, could best decide on which grantees to 
select for oversight. However, if EPA used a statistical approach to 
grantee selection, it could then project the results of the reviews to 
all EPA grantees.[Footnote 27] Since EPA did not use such a sampling 
plan in conducting its 2002 reviews, neither EPA nor we could use data 
from these reviews to determine the overall compliance of grantees. 
Furthermore, without a statistical approach, EPA cannot determine 
whether 5, 10, or any other percentage, adequately provides it with a 
comprehensive assessment of its grantees.

Second, EPA does not have a standard reporting format to ensure 
consistency and clarity in reporting review results. Review results 
include such items as problems identified and corrective actions 
required to address these problems. As we found in our analysis of the 
2002 in-depth reviews, EPA officials across the agency report in 
various formats that do not always clearly present the results of the 
review. For example, some EPA officials provided a narrative report on 
the results of their reviews, while others completed a protocol that 
they used in conducting their review. We found 349 instances where the 
project officer or grants management specialist did not clearly explain 
whether he or she had discovered a problem or not.

Finally, EPA did not develop data elements for its grantee compliance 
database or a plan for using that data to identify and act on systemic 
problems. We found that valuable information could be collected from 
the 2002 in-depth reviews for assessing such issues as the (1) types of 
grantees having problems, (2) types of problem areas needing further 
attention, (3) types of reviews--on-site or off-site--that provide the 
best insights into certain problems areas, and (4) corrective actions 
required or recommended to resolve problems.

With such information, EPA could, over time, identify problem areas and 
develop trends to assess the effectiveness of corrective actions in 
order to better target its oversight efforts. In particular, according 
to our analysis of EPA's 2002 in-depth reviews, administrative reviews 
identify more problems when conducted on site, while the number of 
problems identified by programmatic reviews does not differ by on-site 
or off-site reviews. (See table 10 in app. III.) However, nearly half 
of the programmatic reviews, which constituted more than 80 percent of 
the 2002 reviews, were conducted on site. (See table 5 in app. III.) 
Since on-site reviews are resource intensive because of travel costs 
and staff used, a systematic analysis could enable EPA to better target 
its resources. Similarly, EPA could incorporate other information into 
its grantee compliance database, such as Inspector General reports, to 
identify problem areas, and target oversight resources. In addition, 
EPA could use the database to track the resolution of problems.

The weaknesses in the policies are compounded by the fact that EPA did 
not always effectively implement its oversight policies. Specifically, 
EPA used its database to collect and store in-depth reviews, but not 
all EPA offices were required under the policies to enter the results 
of their reviews into the database, and not all of the offices required 
to enter information into the database did so. Moreover, EPA officials 
acknowledged the agency did not have an effective method to verify the 
number of in-depth reviews it reported to Congress. We found that our 
total number of in-depth reviews--1,232--was about the same as EPA's 
total--1,208--but we found discrepancies, on an office-by-office basis, 
in the number of reviews conducted. (See table 6 in app. III.) For 
example, in one office we identified 176 more reviews than EPA did, and 
in another office we identified 76 fewer reviews than EPA did. Finally, 
we found that EPA's in-depth reviews did not always indicate whether 
these reviews had used a protocol. For example, in 294 of the 893 
nonmandated reviews, it was unclear whether the grants specialist or 
project officer used a protocol.[Footnote 28] (See tables 14 and 15 in 
app. III.):

The Inspector General has also reported on problems with EPA's 
implementation of its actions to address oversight problems. In 2002, 
the Inspector General found, among other things, inconsistent 
performance of monitoring responsibilities, inadequate preparation of 
monitoring plans, and inadequate internal management reviews.[Footnote 
29] For example, the Inspector General found that EPA did not use the 
internal management reviews to draw overall conclusions on the 
effectiveness of oversight and did not quantify its review results. 
Furthermore, some senior resource officials did:

not emphasize the importance of grants oversight.[Footnote 30] 
According to these officials, the minimal control they have over the 
regional program offices' priorities contributes to problems in 
effective grants management oversight. The Inspector General found that 
EPA lacked an official who is clearly responsible for monitoring the 
regional program offices' priorities for overseeing grants.

EPA's Efforts to Address the Complex Problem of Measuring Environmental 
Results Have Not Consistently Ensured That Grantees Achieve Them:

Planning for grants to achieve environmental results--and measuring 
results--is a difficult, complex challenge. As pointed out in an 
earlier report,[Footnote 31] while it is important to measure outcomes 
of environmental activities rather than just the activities (outputs) 
themselves, agencies face challenges in doing this. Output measures are 
inherently easier to develop and report on than environmental results 
measures. In contrast, defensible outcome measures are substantially 
more difficult to develop because of the (1) absence of baseline data 
against which environmental improvements can be measured, (2) inherent 
problems in quantifying certain results, (3) difficulty of linking 
program activities with environmental results because multiple 
conditions influence environmental results, and (4) resources needed 
for measurement.

Nonetheless, EPA has focused on grants' environmental results in its 
partnership with the states under the National Environmental 
Performance Partnership System. Under this system, states may enter 
into "Performance Partnership Agreements" with their EPA regional 
offices. The partners use a common set of national environmental 
indicators ("core performance measures"). In developing these 
performance measures, EPA and state officials have sought to move 
beyond counting the number of actions conducted and toward 
environmental outcomes.

However, EPA has awarded some discretionary grants before considering 
how the results of the grantees' work would contribute to achieving 
environmental results. In 2001, we reported that EPA program officials 
treated EPA's strategic goals and objectives as a clerical tool for 
categorizing grants after the funds were already awarded rather than as 
a tool to guide grant selection, although the grants generally were 
aligned with EPA's strategic goals and objectives.[Footnote 32] By 
assessing the relevance of these grants to EPA's strategic plan after 
awarding the grant, EPA cannot ensure that it is awarding grants that 
will best help it accomplish its mission.

EPA has also not developed environmental measures and outcomes for all 
of its grant programs. In 2000, we reported that EPA did not have 
program criteria to measure the effectiveness of its STAR 
program.[Footnote 33] Instead, EPA focused on the procedures and 
processes for awarding grants. As a result, EPA was uncertain about 
what the program was achieving. Similarly, OMB recently evaluated four 
EPA grant programs to assess their effectiveness in achieving and 
measuring results.[Footnote 34] According to this evaluation, these 
four programs lacked outcome-based measures--measures that 
demonstrated the impact of the programs on improving human health and 
the environment. It concluded that one of EPA's major challenges was 
demonstrating program effectiveness in achieving public health and 
environmental results. Furthermore, as we reported in 1999, the 
National Environmental Performance Partnership System was hampered by 
the technical problems we noted earlier, such as the absence of 
baseline data against which environmental improvements could be 
measured.[Footnote 35]

Finally, EPA often does not require grantees to submit work plans that 
explain how a project will achieve measurable environmental results. 
The grantee work plan should describe the project, its objectives, the 
method the grantee will use to accomplish the objectives, and expected 
outcomes. The project officer uses the work plan to evaluate 
performance under the grant agreement. In 2002, EPA's Inspector General 
reported that EPA approved some grantees' work plans without 
determining the projects' long-term human health and environmental 
outcomes.[Footnote 36] In fact, for almost half of the 42 discretionary 
grants the Inspector General reviewed, EPA did not even attempt to 
measure the projects' outcomes. Instead, EPA funded grants on the basis 
of work plans that focused on short-term procedural results, such as 
meetings or conferences. In some cases, it was unclear what the grant 
had accomplished. In 2003, the Inspector General again found the 
project officers had not negotiated environmental outcomes in work 
plans. The Inspector General found that 42 percent of the grant work 
plans reviewed--both discretionary and nondiscretionary grants--lacked 
negotiated environmental outcomes.[Footnote 37]

EPA Has Not Always Effectively Managed Its Grants Staff and Resources 
despite Some Past Efforts:

EPA has not always appropriately allocated the workload for staff 
managing grants, provided them with adequate training, or held them 
accountable. Additionally, EPA has not always provided staff with the 
resources, support, and information necessary to manage the agency's 
grants. To address these problems, EPA has taken a number of actions, 
such as conducting additional training and developing a new electronic 
grants management system. However, implementation weaknesses have meant 
that EPA has not completely resolved its resource management problems.

EPA Has Not Always Effectively Managed Its Grant Resources:

In terms of staff, EPA faces workload problems, according to its 
internal management reviews. EPA has not had enough staff in some 
locations to adequately manage the number of grants it awards. 
Furthermore, workload was sometimes unevenly distributed. For example, 
in one program office, 29 project officers had more than 4 grants to 
manage, while 11 had more than 20. Additionally, in one regional 
office, project officers and grant specialists did not promptly provide 
required approvals for certain grants management tasks, in part because 
their competing workloads prevented them from processing approvals on 
time. Finally, in 2002, the Inspector General reported that there was 
considerable uncertainty over the size of the workload that grants 
specialists and project officers could manage effectively.[Footnote 38] 
These workload problems could, in some instances, result in poor grants 
management.

EPA requires all project officers to certify their grants management 
skills by attending a 3-day training course; this course is based on a 
training manual that EPA periodically updates. Even though EPA has this 
requirement, it has not always provided grants staff with the training 
necessary to properly manage all aspects of grants. According to EPA's 
internal management reviews from 1997 through 2002, some staff managing 
grants had not completed the basic training required to manage grants. 
Other staff may have completed the basic training but need additional 
training to refresh their skills or to become familiar with all of 
their grants management responsibilities and requirements. For example, 
in some instances, project officers were not familiar with the five key 
areas they were to review when monitoring grantees, such as the 
financial aspects of a grantee's performance. According to a senior EPA 
official, with such a small percentage of some project officers 
spending time on grants management duties, it is difficult for EPA to 
ensure that its 1,835 project officers who are actively managing grants 
have the level of training they need.

Furthermore, the agency has not always held its staff--such as project 
officers--accountable for fulfilling grants management 
responsibilities. According to the Inspector General and internal 
management reviews, EPA has not clearly defined project officers' 
grants management responsibilities in their position descriptions and 
performance agreements. Without specific standards for grants 
management in performance agreements, it is difficult for EPA to hold 
staff accountable. It is, therefore, not surprising that, according to 
the Inspector General, project officers faced no consequences for 
failing to effectively perform grants management duties. Compounding 
the problem, agency leadership has not always emphasized the importance 
of project officers' grants management duties.[Footnote 39]

It may be difficult to hold project officers accountable because many 
project officers spend only a small proportion of their time managing 
grants. For example, a recent EPA workload assessment noted that 
approximately 35 percent of the agency's project officers manage only 
one grant, instead devoting the bulk of their time to their scientific 
and technical responsibilities.

EPA has also not adequately managed its resources to support grant 
staff in performing their duties. Some EPA internal management reviews 
noted a lack of resource commitment--time and money--to conduct grant 
management activities and develop staff. For example, in some cases, 
grantee oversight, particularly the on-site reviews, was limited by the 
scarcity of such resources as travel funds.

Finally, staff did not always have the information they needed to 
effectively manage grants. According to several EPA internal management 
reviews, staff lacked accessible or useable reference material--such as 
policy and guidance documents, and other information resources, such as 
reports of grantee expenditures. Additionally, we and others have 
reported that EPA did not use information from performance evaluations 
or information systems to better manage its grants. For example, one 
EPA region did not analyze the results of its own internal surveys, 
which were designed to assess the effectiveness of its grants 
management operations.

Efforts to Improve the Management of Grant Resources Have Had Mixed 
Results:

EPA has taken actions to address concerns that all of its grants 
management staff receive training. EPA developed a database to document 
which project officers had received the required 3-day basic training 
and began to require that all project officers take an on-line 
refresher course every 3 years. While these steps were primarily 
designed to improve the training of project officers, grants 
specialists also have received some training, and EPA has recently 
pilot-tested a standard training course specifically designed for them. 
EPA managers and staff have also received other types of training to 
enhance grants management skills. For example, EPA has piloted a 1-day 
course on grants management for managers and supervisors. The Office of 
Grants and Debarment also provides training on specific aspects of 
grants management--such as postaward monitoring--as requested by 
program or regional offices. Finally, in response to concerns that some 
grantees did not have the skills required to manage grants, EPA 
developed a 1-day grants management training course for nonprofit 
grantees.

However, these actions have not been fully successful. In 2002, EPA's 
Inspector General reported that the agency did not have adequate 
internal controls in place to ensure that project officers complied 
with the training 
requirements.[Footnote 40] Specifically, one region did not track the 
names and dates of project officers who received training, the 
agencywide database on training for project officers was inaccurate and 
had limited functionality, and the on-line refresher course did not 
have the controls necessary to prevent staff from obtaining false 
certifications.

EPA's training efforts have also needed strengthening to better ensure 
that staff receive and retain the skills necessary to manage grants. In 
particular, while offices can request specific training from the Office 
of Grants and Debarment, EPA does not have a process to ensure that all 
staff receive the training needed to implement new policies and 
regulatory requirements in a timely manner. For instance, when EPA 
issued new post-award monitoring policies to improve its grants 
oversight, project officers did not receive training on these new 
requirements unless their program or regional office specifically 
requested that training or it was time for the officers to take the 
refresher course. EPA also does not measure the impact of its training 
programs by determining the baseline level of skills that employees 
possess before taking training and determining what, if any, skills 
have improved as a result of the training. Nor does EPA link its 
training activities with improved performance on the job, especially in 
targeted areas such as monitoring contacts with grantees and 
documentation of files. As a result, EPA cannot hold either the 
trainers or the trainees accountable for improving grants management 
skills and performance.

EPA has also taken steps to improve another critical resource--its 
primary data system for managing grants. In 1997, it began developing 
the Integrated Grants Management System (IGMS), which, according to an 
EPA official, will facilitate oversight by allowing electronic 
management throughout the life of a grant. EPA is developing the system 
through a series of modules (databases). EPA estimates that once IGMS 
becomes fully operational, it will save roughly $1.6 million annually. 
EPA believes IGMS could help resolve some of the long-standing problems 
in grants management by implementing controls to prevent the submission 
of certain documents without required elements and to provide 
electronic reminders of when certain activities or documents are due. 
Additionally, EPA designed the system to reduce the potential for data 
entry errors. In 2001, EPA began implementing the module for the 
application and award phases of a grant. Using IGMS, EPA expects to 
electronically review the grantee's application, prepare and review 
EPA's documents, and approve the award. In April 2003, EPA began using 
IGMS' postaward module. This module allows project officers to 
electronically enter project milestones into the system, communicate 
with other staff involved in overseeing grants, receive reports from 
grantees, and initiate closeout activities. EPA expects that grants 
specialists and regional program staff will use IGMS to electronically 
manage grants by September 2004 and headquarters program staff by the 
end of 2006.

Despite this promising development, the implementation period for IGMS 
has been lengthy. This delay was due in part to Y2K concerns, the 
complexity of the system for grants management, and the addition of 
other major business processes into the system. Consequently, EPA has 
had to plan a lengthy implementation schedule in order to ensure that 
it can properly roll out the system to all of the programs and regions 
with its available training and user-support resources. Therefore, 
while those offices that have received IGMS are using it to manage 
grants, EPA cannot yet determine whether IGMS has solved some of the 
agency's resource management problems.

New Policies and Plan Show Promise but Require Strengthening, Enhanced 
Accountability, and Sustained Commitment to Succeed:

EPA's recently issued policies on competition and oversight and a 5-
year plan to address its long-standing grants management problems are 
promising and focus on the major management challenges, but these 
policies and plan require strengthening, enhanced accountability, and 
sustained commitment to succeed. In particular, EPA's policy generally 
requiring competition for discretionary grants should encourage 
selection of the most qualified applicants. However, successful 
implementation of this policy requires a more planned, rigorous 
approach to awarding some grants than in the past, which is a major 
cultural shift for EPA managers and staff. Similarly, EPA's new 
oversight policy should increase the agency's monitoring of individual 
grantees, but it does not enable the agency to identify and address 
systemic problems. Finally, EPA's new grants management plan takes 
significant steps toward improving grant management, but the plan will 
be difficult to implement if EPA does not demonstrate that it is 
committed to carrying out the plan over the 5-year period and if it 
does not hold managers and staff accountable for their grants 
management performance.

Competition Policy Should Encourage the Selection of the Most Qualified 
Applicants if Successfully Implemented:

In September 2002, EPA issued a policy to promote competition in grant 
awards by requiring that most discretionary grants be competed. 
Discretionary grants totaled about $719 million of the $4.2 billion in 
grants awarded in fiscal year 2002. The policy applies to most 
discretionary grant programs or individual grants of more than 
$75,000.[Footnote 41] It exempts individual grants only if they meet 
certain criteria, such as national security interests. Exemptions 
require detailed, written justification and approval. The policy also 
promotes widespread solicitation for competed grants by establishing 
specific requirements for announcing funding opportunities in, for 
example, the Federal Register and on Web sites.

Because the competition policy represents a major change in the way EPA 
awards discretionary grants, the agency created the position of a 
senior-level competition advocate for grants. This advocate oversees 
the policy's implementation and compliance and evaluates its 
effectiveness. Among other things, the competition advocate is 
responsible for developing implementation guidance and coordinating 
training to ensure successful implementation. The competition advocate 
ensures compliance, in part, by reviewing any requests for programs to 
be exempted from competition. The advocate will also have to work with 
headquarters and regional offices as well as staff and management 
officials at all levels throughout EPA to successfully implement the 
policy.

The competition policy faces implementation barriers because it 
represents a major cultural shift for EPA staff and managers, who have 
had limited experience with competition, according to the Office of 
Grants and Debarment. The policy requires EPA officials to take a more 
planned, rigorous approach to awarding grants. That is, EPA staff must 
determine the evaluation criteria and ranking of these criteria for a 
grant, develop the grant announcement, and generally publish it at 
least 60 days before the application deadline. Staff must also evaluate 
applications--potentially from a larger number of applicants than in 
the past--and notify applicants of their decisions. These activities 
will require significant planning and take more time than awarding 
grants noncompetitively. Office of Grants and Debarment officials 
anticipate a learning curve as staff implement the policy and will 
evaluate the policy's effectiveness, including the threshold dollar 
level in 2005. While the policy has been in effect for a number of 
months, it is too early to tell if the policy has resulted in increased 
competition over the entire fiscal year. EPA officials believe that 
preliminary results indicate that the policy is increasing the use of 
competition.

Oversight Policy Makes Important Improvements but Requires 
Strengthening to Identify Systemic Problems:

EPA's December 2002 oversight policy makes important improvements, but 
it does not fully address the weaknesses in previous oversight policies 
that we noted earlier. The new policy, among other things, requires the 
following:

* Offices to increase baseline monitoring from at least once during the 
life of the grant to annually throughout the life of the grant;

* Offices to increase the percentage of in-depth monitoring to at least 
10 percent of grant recipients;

* Grant management specialists conducting on-site, in-depth reviews to 
include "transaction testing" for unallowable costs, such as lobbying 
and entertainment expenditures. Transaction testing consists of 
reviewing grantees' accounting ledgers and underlying documentation;

* Offices to identify their criteria for selecting recipients for in-
depth reviews and submit these criteria as part of their annual plans;

* Offices to enter in-depth reviews into the grantee compliance 
database; and:

* The Office of Grants and Debarment's Grants Administration Division 
to have a work group recommend database enhancements to make the 
database more useful for program offices. This work group plans to 
report its findings in August 2003.

While the new policy makes these refinements in oversight, it still 
does not enable EPA to identify systemic problems in grants management 
that we noted earlier. Specifically, the policy does not do the 
following:

* Incorporate a statistical approach to selecting grantees for review. 
EPA's new policy annually selects 10 percent of grant recipients for 
in-depth review. EPA officials believe that a single, higher 
percentage, rather than a range, will eliminate variation among its 
offices and provide better coverage. However, without a statistical 
approach, EPA cannot evaluate whether this 10 percent is appropriate. 
With such an approach, EPA could adjust the number and allocation of 
in-depth reviews to match the level of risk associated with each type 
of grant recipient and project the results of the reviews to all EPA 
grantees.

* Require a standard reporting format for in-depth reviews. EPA needs 
the clear and precise information from its in-depth reviews that a 
standard reporting format would provide--and which was absent from the 
2002 in-depth reviews we analyzed. Standard information is important so 
that EPA can use the information to guide its grants oversight efforts 
agencywide.

* Maximize use of information in the grantee compliance database. While 
EPA has recognized the importance of the information in the database by 
forming a database work group and collecting a limited amount of data 
from in-depth reviews, it has not yet developed a plan for using data 
from all its oversight efforts--in-depth reviews, corrective actions, 
and other compliance efforts--to fully identify systemic problems and 
then inform grants management officials about oversight areas that need 
to be addressed.

New Plan Focuses on the Four Major Management Challenges but Will 
Require Strengthening, Sustained Commitment, and Enhanced 
Accountability:

According to EPA's Assistant Administrator for Administration and 
Resources Management, the agency's April 2003 5-year grants management 
plan is the most critical component of EPA's efforts to improve its 
grants management.[Footnote 42] The plan has five goals and 
accompanying objectives, as follows:

* Promote competition in awarding grants. Identify annual funding 
priorities, encourage a large and diverse group of applicants, promote 
the importance of competition within the agency, and provide adequate 
support for the grant competition advocate.

* Strengthen EPA's grants oversight. Improve internal management 
reviews of EPA's offices, improve and expand reviews of grant 
recipients, develop approaches to prevent or limit grants management 
weaknesses, establish clear lines of accountability for grants 
oversight, and provide high-level coordination, planning, and priority 
setting.

* Support the identification and achievement of environmental outcomes. 
Ensure that grantees include expected environmental outcomes and 
performance measures in grant work plans and improve reporting on 
progress made in achieving environmental outcomes.

* Enhance the skills of EPA personnel involved in grants management. 
Update training materials and courses and improve delivery of training 
to project officers and grants specialists.

* Leverage technology to improve program performance. Enhance and 
expand information systems that support grants management and 
oversight, among other things.

In addition to the goals and objectives, the plan establishes 
performance measures, targets, and action steps with completion dates 
for 2003 through 2006. For example, to strengthen EPA's grants 
oversight, the plan calls for conducting consolidated management 
reviews[Footnote 43] of headquarters and regional offices to review the 
operations of grants management activities and for completing reviews 
for three regional offices and four headquarters offices in 2003. These 
reviews will examine EPA offices' oversight of grantees throughout the 
life of the grant. (See app. IV.):

We believe the grants management plan is comprehensive in that it 
focuses on the four major management challenges--grantee selection, 
oversight, environmental results, and resources--that we identified in 
our work. For the first time, EPA plans a coordinated, integrated 
approach to improving grants management. The plan is also a positive 
step because it (1) identifies objectives, milestones, and resources to 
achieve the plan's goals; (2) provides an accompanying annual tactical 
plan that outlines specific tasks for each goal and objective, 
identifies the person accountable for completing the task, and sets an 
expected completion date; (3) attempts to build accountability into 
grants management by establishing performance measures for each of the 
plan's five goals; (4) recognizes the need for greater involvement of 
senior officials in coordinating grants management throughout the 
agency by establishing a senior-level grants management council to 
coordinate, plan, and set priorities for grants management; and (5) 
establishes best practices for grants management offices.

EPA has already begun implementing several of the actions in the plan 
or meant to support the plan; these actions address previously 
identified problems. For example, EPA now ensures that grants are 
adequately solicited by posting its available grants on the federal 
grants Web site [Hyperlink, http://www.fedgrants.gov] http://
www.fedgrants.gov. To enhance oversight, it initiated the consolidated 
grants management reviews of headquarters and regional offices. To 
better manage for results, EPA formed an Environmental Outcomes Work 
Group to address concerns about focusing on environmental outcomes. 
Finally, for training, EPA (1) instituted verification procedures to 
ensure that only certified project officers manage grants; (2) drafted 
its long-term plan for grants management training; (3) updated the 
project officer training manual; and (4) launched a new training course 
for grants managers. In addition, EPA is implementing an office award 
program to promote effective grants management.

Although EPA's plan lays out a road map for improving grants 
management, it has two weaknesses. First, in terms of measuring 
results, while the plan supports identifying and achieving 
environmental outcomes, the suggested protocol for in-depth reviews 
does not call for project officers to ask grantees about their progress 
in using measures to achieve environmental outcomes. Second, EPA's plan 
may not fully address long-standing training problems. Specifically, it 
is too early to tell whether EPA's effort to develop a long-term 
training plan will provide (1) a process to ensure that all staff 
members receive the training necessary to implement new policies and 
regulations in a timely manner or (2) a mechanism to evaluate the 
impact of its training efforts on improving staff's grants management 
knowledge and skills, the extent to which staff's performance improves 
over time, and demonstrated results in improving grants management 
performance. The impact of training could be measured by indicators 
such as the proportion of financial or other problems identified early 
on by ongoing monitoring, thereby avoiding potentially greater losses 
in the long run. Furthermore, the plan's performance measures for 
evaluating training are output-oriented--how many staff are trained--
rather than outcome-oriented--how much skills have improved.

Successful implementation of the new plan requires more than addressing 
these two weaknesses, however. It requires all staff--senior 
management, project officers, and grants specialists--to be fully 
committed to, and accountable for, grants management. Recognizing the 
importance of commitment and accountability, EPA's 5-year grants 
management plan has as one of its objectives the establishment of clear 
lines of accountability for grants oversight. The plan, among other 
things, calls for (1) ensuring that performance standards established 
for grants specialists and project officers adequately address grants 
management responsibilities in 2004; (2) clarifying and defining the 
roles and responsibilities of senior resource officials, grant 
specialists, project officers, and others in 2003; and (3) analyzing 
project officers' and grants specialists' workload in 2004.

In implementing this plan, however, EPA faces challenges to enhancing 
accountability. Although the plan calls for ensuring that project 
officers' performance standards adequately address their grants 
management responsibilities, agencywide implementation may be 
difficult. Currently, project officers do not have uniform performance 
standards, according to officials in EPA's Office of Human Resources 
and Organizational Services. Instead, each supervisor sets standards 
for each project officer, and these standards may not include grants 
management responsibilities. It could take up to a year to establish 
and implement a uniform performance standard, according to these 
officials. Instead, the Assistant Administrator for the Office of 
Administration and Resources Management is planning to issue guidance 
in October 2003 on including grants management responsibilities in 
individual performance agreements for the next performance cycle 
beginning in January 2004. Once individual project officers' 
performance standards are established for the approximately 1,800 
project officers, strong support by managers at all levels, as well as 
regular communication on performance expectations and feedback, will be 
key to ensuring that staff with grants management duties successfully 
meet their responsibilities.

Although EPA's current performance management system can accommodate 
development of performance standards tailored to each project officer's 
specific grants management responsibilities, the current system 
provides only two choices for measuring performance--satisfactory or 
unsatisfactory, which may make it difficult to make meaningful 
distinctions in performance. Such an approach may not provide enough 
meaningful information and dispersion in ratings to recognize and 
reward top performers, help everyone attain their maximum potential, 
and deal with poor performers.

Furthermore, it is difficult to implement performance standards that 
will hold project officers accountable for grants management because 
these officers have a variety of responsibilities and some project 
officers manage few grants, and because grants management 
responsibilities often fall into the category of "other duties as 
assigned." To address this issue, EPA officials are considering, among 
other options, whether the agency needs to develop a smaller cadre of 
well-trained project officers to oversee grantees, rather than rely on 
approximately 1,800 project officers with different levels of grants 
management responsibilities and skills. Some EPA officials believe that 
having a cadre may help the agency more effectively implement revised 
grants management performance standards because fewer officers with 
greater expertise would oversee a larger percentage of the grants.

EPA will also have difficulty achieving the plan's goals if all 
managers and staff are not held accountable for grants management. The 
plan does not call for including grants management standards in 
managers' and supervisors' agreements. In contrast, senior grants 
managers in the Office of Grants and Debarment as well as other Senior 
Executive Service managers have performance standards that address 
grants management responsibilities.[Footnote 44] However, middle-level 
managers and supervisors also need to be held accountable for grants 
management because they oversee many of the staff that have important 
grants management responsibilities. According to Office of Grants and 
Debarment officials, they are working on developing performance 
standards for all managers and supervisors with grants 
responsibilities.

Further complicating the establishment of clear lines of 
accountability, the Office of Grants and Debarment does not have direct 
control over many of the managers and staff who perform grants 
management duties--particularly the approximately 1,800 project 
officers in headquarters and regional program offices. The division of 
responsibilities between the Office of Grants and Debarment and program 
and regional offices will continue to present a challenge to holding 
staff accountable and improving grants management, and will require the 
sustained commitment of EPA's senior managers.

Promising Practices Could Assist EPA in Addressing Its Grants 
Management Challenges:

Although information on successful grants management practices is 
limited, we identified promising practices in the five federal agencies 
that award the largest amount of grant dollars, as well as in other 
organizations, that may help EPA address its grants management 
challenges. These practices include (1) screening grantees and 
assessing some grantees' financial and internal controls before 
awarding a grant; (2) using an information system to track and manage 
in-depth grantee reviews, identify problems, and analyze trends; and 
(3) incorporating a results orientation throughout the grants process, 
including the preaward phase and the final reporting of results 
achieved. In addition, we have developed a guide to assist federal 
agencies in developing effective training programs and performance 
management systems.[Footnote 45]

Screen and Select the Best Applicants:

According to some grants management officials we spoke with, screening 
and reviewing the adequacy of grantees' financial and internal controls 
before awarding a grant helps to avoid problem grantees. For example,

* The Robert Wood Johnson Foundation conducts preaward desk reviews of 
some applicants to learn about the organization and assess its fiscal 
health. Specifically, for grants under $200,000, the foundation may 
check a Web site that contains financial information on nonprofit 
organizations. For grants larger than $200,000, especially when the 
grantee is newly formed or has never received money from the 
foundation, staff may request more detailed information--such as an 
organizational chart, copies of audit reports, accounting policies, 
investment policies, and a conflict-of-interest statement. In rare 
cases, the foundation's financial staff may visit the applicant to 
assess its systems.

* The Department of Commerce (Commerce) Office of Inspector General 
conducts a three-step screening process to identify deficiencies that 
could hinder the ability of nonprofit applicants to properly control 
federal funds: (1) a name check, through the Federal Bureau of 
Investigation, to determine whether the individuals connected with a 
proposed award have been convicted of embezzlement or other crimes; (2) 
a review of the single audit clearinghouse database to determine if the 
applicant had a single audit performed and subsequent follow-up to 
determine if any findings were resolved; and (3) a review of a credit 
report to determine whether the applicant is financially viable. For 
the last two reporting periods--April 2002 to March 2003--Commerce's 
Inspector General screened 1,657 proposed awards; delayed 67 until 
concerns were satisfactorily resolved; and established special 
conditions for 42 awards to adequately safeguard federal funds.

* The Department of Health and Human Services (HHS) uses an internal 
"alert list," accessible to grants management staff throughout the 
department, to help identify potential grantees who (1) previously had 
not complied with grant regulations or the terms and conditions of a 
grant, (2) failed to have had a single audit performed, or (3) have 
significant single audit findings that need correction. For these 
potential grantees, HHS may incorporate special terms and conditions 
into their agreements and more closely monitor them throughout the 
grant. Applicants may be put on the alert list by agencies within the 
department or at the recommendation of the Office of Inspector General. 
According to HHS officials, the alert list should be used for problem 
grantees so that the department has the necessary controls in place to 
safeguard federal funds rather than be used to prohibit grantees from 
receiving grant funding.

* HHS's Substance Abuse and Mental Health Services Administration 
(SAMHSA) uses the alert list and also conducts pre-award reviews to 
assess applicants' ability to manage grant funds. Like the Department 
of Commerce, SAMHSA screens some nonprofit grantees to ensure they have 
the financial systems needed to manage grant funds. SAMHSA examines the 
database for the single audit clearinghouse and its own audit database 
to determine if the grant applicant has received a single audit and 
whether the audit had significant findings. If the applicant has no 
significant findings, SAMHSA assumes that the applicant has adequate 
financial systems in place. However, if an audit found significant 
problems that the applicant did not adequately address, SAMHSA may 
designate the applicant as high risk and add it to the HHS alert list. 
SAMHSA may still award grant funds but set restrictions to safeguard 
the funds. For example, the grantee may be required to draw funds on a 
reimbursement basis, that is, after it submits documentation on its 
expenses. Grantees who have not received a single audit must provide 
copies of their policies and procedures. SAMHSA assesses this 
information to determine if it is sufficient for managing federal 
funds.

Use an Information System to Manage In-Depth Reviews, Identify 
Problems, and Analyze Trends:

The Federal Transit Administration, within the Department of 
Transportation, uses an information system to track and manage its 
reviews of grantees. This system contains information on problems 
identified in the reviews and a schedule of corrective actions that the 
grantee needs to take. Using this system, officials can track 
correspondence between the agency and grantee and document the 
resolution of problems. In addition, officials can use this system to 
analyze trends, such as the compliance of individual grantees over 
time, the compliance of all grantees or a group of grantees (such as 
those in a region), compliance issues found in different types of 
reviews, and areas where grantees may require additional technical 
assistance. This information system could serve as a model for 
enhancing EPA's grantee compliance database.

Incorporate a Results Orientation throughout the Grants Process:

Several agencies consider grant results in awarding and reviewing 
grantees, including the following:

* Rural Development, within the U.S. Department of Agriculture, takes 
outcomes into account by requiring applicants to provide information on 
the results they expect to achieve and by selecting grantees, in part, 
on the basis of this information. For instance, when job creation is a 
goal of the grant, the Rural Business Enterprise Grant Program gives 
applicants points for jobs they expect to create or save with a grant 
and requires them to provide written commitments from businesses that 
plan on creating jobs as a result of the grant. Rural Development then 
gives points to applicants based on the grant dollars per job they 
expect to create.

* HHS's Health Resources and Services Administration incorporates 
information on the results its grantees have achieved into its annual 
performance report. The administration collects data directly from its 
grantees and surveys a sample of the people its grantees serve to 
determine the results the grantees have achieved. Because of the 
agency's success in managing its health center grants to achieve 
results,[Footnote 46] OMB rated the program "effective" in its 
governmentwide performance and assessment-rating tool. Out of 79 grant 
programs evaluated, this was one of only two rated "effective," OMB's 
highest rating category.[Footnote 47] According to OMB, the health 
center program uses performance information to improve annual 
administrative and clinical outcomes. The agency also conducts its own 
evaluations and has a contractor regularly evaluate the program; these 
evaluations indicate the program is effective at achieving its goal of 
extending high-quality health care to underserved populations.

Train and Hold Staff Accountable for Grants Management:

We have developed a guide that can assist federal agencies in planning, 
designing, implementing, and evaluating effective training and 
development programs.[Footnote 48] This guide may help EPA address a 
training weakness in the plan that we identified--EPA's lack of a 
mechanism to determine the extent to which training improves staff's 
grant management skills. For example, the guide suggests evaluating the 
impact of training based on such measures as changes in employee 
skills, knowledge, or abilities; changes in on-the-job behaviors; the 
impact of the training on program or organizational results, including, 
in some cases, assessing the return on investment by comparing training 
costs with benefits.

In terms of accountability, we developed key practices that federal 
agencies can use to establish effective performance management 
systems.[Footnote 49] These systems can be used to drive internal 
change and achieve desired results. The key practices have helped 
public sector organizations in the United States and abroad create a 
clear linkage between individual performance and organizational 
success. They are also critical for ensuring individual accountability 
for results throughout the organization. Key practices of an effective 
performance management system include, for example:

* Align individual performance expectations with organizational goals. 
An explicit alignment helps individuals see the connection between 
their daily activities and organizational goals.

* Use competencies to provide a fuller assessment of performance. 
Competencies define the skills and supporting behaviors staff need to 
effectively contribute to organizational results.

* Make meaningful distinctions in performance. Effective performance 
management systems strive to provide candid and constructive feedback 
and the necessary objective information and documentation to award top 
performers and deal with poor performers.

Conclusions:

If EPA is to better achieve its environmental mission, it must more 
effectively manage its grants programs--which account for more than 
half of its annual budget. EPA's new policies and 5-year grants 
management plan show promise, but they are missing several critical 
elements necessary for the agency to address past grants management 
weaknesses. Specifically, EPA's new oversight policy lacks (1) 
statistical methods to identify grantees for review, (2) a standard 
reporting format to ensure consistency and clarity in report results, 
and (3) a plan for integrating and analyzing compliance information 
from multiple sources. These actions would help EPA identify systemic 
problems with its grantees and better target its oversight resources. 
EPA's plan does not adequately address the need to assess grantees' 
progress towards achieving environmental outcomes. If project officers 
tracked grantees' progress during in-depth reviews, they would better 
help the agency manage grants for results. Finally, in our view, EPA's 
plan does not go far enough to ensure that all managers and staff with 
grant management responsibilities are held accountable for fulfilling 
their grants management responsibilities. Until EPA holds all managers 
and staff accountable for these responsibilities, it cannot be assured 
that it will achieve the objectives of its grants management plan.

EPA recognizes the importance of continuously improving its grants 
management practices. To this end, we believe EPA could benefit from 
considering the practical approaches successfully implemented by other 
federal agencies and organizations. It could also benefit from using 
our guide to establish effective training programs to develop relevant 
staff skills and a performance management system to hold staff 
accountable for using these skills to carry out their grants management 
responsibilities.

Given EPA's uneven performance in improving its grants management 
problems, it is too early to tell if its new policies and comprehensive 
5-year grants management plan will succeed more than past initiatives. 
While the plan shows promise, we believe that congressional oversight 
is important to ensure that EPA's Administrator, managers, and staff 
implement the plan in a sustained, coordinated fashion to meet the 
plan's ambitious targets and time frames. To help facilitate this 
oversight, EPA's annual report to Congress could serve as a vehicle for 
EPA to report on its achievements in improving grants management.

Recommendations for Executive Action:

To ensure that EPA's recent efforts to address its grants management 
challenges are successful, we recommend that the Administrator of EPA 
provide sufficient resources and commitment to meeting the agency's 
grants management plan's goals, objectives, and performance targets 
within the specified time frames. Furthermore, to strengthen EPA's 
efforts we recommend:

* incorporating appropriate statistical techniques in selecting 
grantees for in-depth reviews;

* requiring EPA staff to use a standard reporting format for in-depth 
reviews so that the results can be entered into the grant databases and 
analyzed agencywide;

* developing a plan, including modifications to the grantee compliance 
database, to use data from its various oversight efforts--in-depth 
reviews, significant actions, corrective actions taken, and other 
compliance information--to fully identify systemic problems, inform 
grants management officials of areas that need to be addressed, and 
take corrective action as needed;

* modifying its in-depth review protocols to include questions on the 
status of grantees' progress in measuring and achieving environmental 
outcomes;

* incorporating accountability for grants management responsibilities 
through performance standards that address grants management for all 
managers and staff in headquarters and the regions responsible for 
grants management and holding managers and staff accountable for 
meeting these standards; and:

* evaluating the promising practices identified in this report and 
implementing those that could potentially improve EPA grants 
management.

To better inform Congress about EPA's achievements in improving grants 
management, we recommend that the Administrator of EPA report on the 
agency's accomplishments in meeting the goals and objectives developed 
in the grants management plan and other actions to improve grants 
management, beginning with its 2003 annual report to Congress.

Agency Comments and Our Evaluation:

We provided a draft of this report to EPA for its review and comment. 
In response, EPA stated that it agreed with our recommendations and 
that it will implement them as part of its 5-year grants management 
plan. EPA further pointed out that it has demonstrated commitment to 
grants management as evidenced by its (1) issuance of directives from 
the Deputy Administrator/Acting Administrator requiring senior 
managers to hold employees accountable and to include compliance with 
grants management policies as part of midyear performance discussions; 
(2) ongoing review of performance standards, which will ensure that 
performance agreements of all employees involved in grants management 
adequately reflect their grants management responsibilities; (3) 
requirement for assistant administrators and regional administrators to 
address in the assurance letters under the Federal Managers' Financial 
Integrity Act the steps they are taking to address grants management 
weaknesses; and (4) development of a tactical action plan, which 
outlines commitments and milestone dates under the grants management 
plan and identifies who is responsible for completing those 
commitments. EPA also provided the Office of Grants and Debarment's 
Fiscal Year 2003 Federal Managers' Financial Integrity Act assurance 
letter, which describes in detail the actions the office has taken.

In technical comments, EPA added that it had redeployed resources to 
the Office of Grants and Debarment and anticipates receiving new 
resources in fiscal year 2004 to conduct grants management oversight. 
Other technical comments have been incorporated into the report, as 
appropriate.

EPA's written comments are presented in appendix V.

:

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies of this report 
to the congressional committees with jurisdiction over EPA and its 
activities; the Honorable Marianne Lamont Horinko, Acting EPA 
Administrator; and the Honorable Joshua B. Bolten Director, Office of 
Management and Budget. We will also make copies available to others 
upon request. In addition, the report will be available at no charge on 
the GAO Web site at htpp://www.gao.gov.

If you have any questions about this report, please contact me at (202) 
512-3841. Key contributors to this report are listed in appendix VI.

John B. Stephenson 
Director, Natural Resources and Environment:

Signed by John B. Stephenson: 

[End of section]

Appendixes:

Appendix I: Objectives, Scope, and Methodology:

This appendix details the methods we used to determine the (1) major 
challenges the Environmental Protection Agency (EPA) faces in managing 
its grants and how it has addressed these challenges in the past, (2) 
extent to which EPA's recently issued policies and grants management 
plan address these challenges, and (3) promising practices, if any, 
that could assist EPA in addressing these challenges.

To identify the challenges EPA faces in managing its grants and how it 
has addressed these challenges in the past, we identified and analyzed 
93 reports written on EPA's grants, including our reports, EPA's 
Inspector General reports, and EPA's internal management reviews 
conducted from 1996 through 2003. To analyze the findings from these 
reviews, we summarized the findings from each report and then grouped 
the findings using the long-table method of analysis. From those 
groups, we identified four broad grants management challenges. We next 
interviewed and obtained documents from EPA officials to determine the 
actions they have taken in the past to address these challenges. We 
also obtained the records of 1,232 in-depth grantee reviews conducted 
in 2002 and analyzed each one using a data collection instrument. We 
conducted inter-rater reliability tests to ensure that information from 
in-depth reviews was entered consistently by each reviewer. In the 
event a document item was unclear, reviewers flagged the case and 
consulted to determine the appropriate response. We requested that EPA 
send documents for all evaluative reviews conducted in calendar year 
2002. Since EPA does not have information on the exact number of 
reviews it conducts each year, we were not able to verify whether we 
received information on the total population of evaluative reviews 
conducted in 2002. Because we could not verify the total number of 
reviews conducted and because some regions and program offices conduct 
a small number of reviews, we requested documents for all in-depth 
evaluative reviews in 2002 (rather than a sample) to ensure coverage of 
all 10 regions and 10 program offices.

To determine the extent to which EPA's recently issued policies and 
grants management plan address these challenges, we reviewed the new 
policies and plan and interviewed EPA officials responsible for 
implementing key aspects of the plan. We also attended EPA's grants 
management training course for project officers both before and after 
they revised their project officer training manual and EPA's training 
course for nonprofit organizations. We also observed several of EPA's 
in-depth reviews of grantees: one administrative on-site review 
conducted by EPA headquarters, two on-site joint reviews conducted by 
EPA's Region 1 office, and two joint desk reviews conducted by EPA's 
Region 1 office. In addition, we interviewed officials in EPA's Office 
of Human Resources and Organizational Services to discuss the agency's 
performance management system.

To identify what promising practices, if any, could assist EPA in 
addressing its grants management challenges, we interviewed and 
obtained documentation from grants management officials, and other 
officials, in the federal government and from private sector 
organizations. For the federal government, we met with officials from 
the five largest granting agencies: the Departments of Agriculture, 
Education, Health and Human Services, Housing, and Transportation. At 
the recommendation of these and other officials, we interviewed other 
agency officials, including those from Agriculture's Rural Development, 
the Department of Commerce, Health and Human Service's Health Resources 
and Services Administration and Substance Abuse and Mental Health 
Services Administration, and Transportation's Federal Transit 
Administration. We also interviewed officials from the Council on 
Foundations, a membership organization, and two foundations: the Robert 
Wood Johnson Foundation and the Ford Foundation. We asked these 
officials for examples of their grant practices that might address the 
types of challenges EPA faces in managing grants. We also reviewed two 
GAO reports that describe effective training and performance management 
systems for federal agencies.

We conducted our work from June 2002 through June 2003 in accordance 
with generally accepted government auditing standards.

[End of section]

Appendix II: EPA'S 10 Regions and Regional Office Locations:

[See PDF for image]

Note: Region 2 also includes Puerto Rico and the Virgin Islands. Region 
9 also includes Guam, American Samoa, Trust Territories, and the 
Commonwealth of the Northern Mariana Islands.

[End of figure]

[End of section]

Appendix III: Results of Our Analysis of EPA'S 2002 In-Depth Reviews:

This appendix presents tables that provide detail on the in-depth 
reviews that the Environmental Protection Agency (EPA) conducted in 
2002. In-depth reviews include both on-site reviews conducted at the 
grantee's location and off-site reviews conducted at an EPA office or 
another location. Tables 5 and 6 show the number of different types of 
reviews and how many reviews each EPA office conducted. Tables 7 to 12 
show the number and types of problems EPA found and how many problems 
were found for each type of grantee and size of grant. Table 13 shows 
the number of times EPA required or recommended action to correct a 
problem. Tables 14 and 15 show how many reviews EPA conducted with or 
without a protocol. Tables 16 and 17 show how many and what types of 
significant actions EPA took against grantees.

Analysis of Number and Types of EPA's In-depth Reviews:

Table 5: Number of Reviews by Type of Reviews:

Type of review: Administrative; Off-site review: 100; On-site review: 
89; Total: 189; Percentage of total: 15.

Type of review: Programmatic; Off-site review: 526; On-site review: 
491; Total: 1,017; Percentage of total: 83.

Type of review: Joint; Off-site review: 18; On-site review: 8; Total: 
26; Percentage of total: 2.

Type of review: Total; Off-site review: 644; On-site review: 588; 
Total: 1,232; Percentage of total: 100.

Source: GAO's analysis of EPA's in-depth reviews.

[End of table]

Table 6: Number of Off-site and On-site Reviews and Number of Reviews 
EPA Indicated It Performed, by Program and Regional Office:

Program/regional office: 

Office of Grants and Debarment; Off-site review: GAO: 42; Off-site 
review: EPA: 45; On-site review: GAO: 30; On-site review: EPA: 
43.

Office of the Administrator; Off-site review: GAO: 43; Off-site review: 
EPA: 66; On-site review: GAO: 9; On-site review: EPA: 10.

Office of Air and Radiation; Off-site review: GAO: 6; Off-site review: 
EPA: 6; On-site review: GAO: 17; On-site review: EPA: 23.

Office of Enforcement and Compliance Assurance; Off-site review: GAO: 
1; Off-site review: EPA: 0; On-site review: GAO: 2; On-site 
review: EPA: 6.

Office of Environmental Information; Off-site review: GAO: 2; Off-site 
review: EPA: 2; On-site review: GAO: 0; On-site review: EPA: 
1.

Office of International Affairs; Off-site review: GAO: 0; Off-site 
review: EPA: 0; On-site review: GAO: 3; On-site review: EPA: 
3.

Office of Prevention, Pesticides and Toxic Substances; Off-site review: 
GAO: 5; Off-site review: EPA: 6; On-site review: GAO: 3; On-
site review: EPA: 2.

Office of Research and Development; Off-site review: GAO: 72; Off-site 
review: EPA: 137; On-site review: GAO: 43; On-site review: 
EPA: 54.

Office of Solid Waste and Emergency Response; Off-site review: GAO: 5; 
Off-site review: EPA: 0; On-site review: GAO: 2; On-site 
review: EPA: 7.

Office of Water; Off-site review: GAO: 22; Off-site review: EPA: 17; 
On-site review: GAO: 24; On-site review: EPA: 20.

Region 1; Off-site review: GAO: 11; Off-site review: EPA: 13; 
On-site review: GAO: 16; On-site review: EPA: 18.

Region 2; Off-site review: GAO: 3; Off-site review: EPA: 6; 
On-site review: GAO: 12; On-site review: EPA: 13.

Region 3; Off-site review: GAO: 34; Off-site review: EPA: 62; 
On-site review: GAO: 55; On-site review: EPA: 35.

Region 4; Off-site review: GAO: 107; Off-site review: EPA: 137; 
On-site review: GAO: 104; On-site review: EPA: 107.

Region 5; Off-site review: GAO: 12; Off-site review: EPA: 0; 
On-site review: GAO: 28; On-site review: EPA: 62.

Region 6; Off-site review: GAO: 63; Off-site review: EPA: 64; 
On-site review: GAO: 59; On-site review: EPA: 60.

Region 7; Off-site review: GAO: 26; Off-site review: EPA: 32; 
On-site review: GAO: 36; On-site review: EPA: 33.

Region 8; Off-site review: GAO: 21; Off-site review: EPA: 20; 
On-site review: GAO: 54; On-site review: EPA: 25.

Region 9; Off-site review: GAO: 145; Off-site review: EPA: 3; 
On-site review: GAO: 57; On-site review: EPA: 23.

Region 10; Off-site review: GAO: 24; Off-site review: EPA: 13; 
On-site review: GAO: 34; On-site review: EPA: 34.

Total; Off-site review: GAO: 644; Off-site review: EPA: 629; 
On-site review: GAO: 588; On-site review: EPA: 579.

Source: GAO's analysis of EPA's in-depth reviews.

Note: The columns with a GAO heading represent the review totals we 
found in the documents EPA provided. The columns with an EPA heading 
represent the review totals EPA officials reported to Congress.

[End of table]

Analysis of Problems Identified in EPA's In-depth Reviews:

Table 7: Number of Problems Identified by Type of Review:

Problem: Accounting; Administrative (N=189): 37; Programmatic 
(N=1,017): 26; Joint: (N=26): 1; Total: 64.

Problem: Administrative; Administrative (N=189): 16; Programmatic 
(N=1,017): 23; Joint: (N=26): 1; Total: 40.

Problem: Approval of modifications; Administrative (N=189): 2; 
Programmatic (N=1,017): 4; Joint: (N=26): 0; Total: 6.

Problem: Conflict of interest; Administrative (N=189): 5; Programmatic 
(N=1,017): 0; Joint: (N=26): 0; Total: 5.

Problem: Cost sharing; Administrative (N=189): 10; Programmatic 
(N=1,017): 4; Joint: (N=26): 0; Total: 14.

Problem: Financial expenditures; Administrative (N=189): 20; 
Programmatic (N=1,017): 35; Joint: (N=26): 0; Total: 55.

Problem: Indirect costs; Administrative (N=189): 12; Programmatic 
(N=1,017): 2; Joint: (N=26): 5; Total: 19.

Problem: Internal controls; Administrative (N=189): 13; Programmatic 
(N=1,017): 1; Joint: (N=26): 0; Total: 14.

Problem: Lobbying; Administrative (N=189): 0; Programmatic (N=1,017): 
0; Joint: (N=26): 0; Total: 0.

Problem: Missing required audit; Administrative (N=189): 4; 
Programmatic (N=1,017): 2; Joint: (N=26): 2; Total: 8.

Problem: Personnel/payroll; Administrative (N=189): 51; Programmatic 
(N=1,017): 92; Joint: (N=26): 5; Total: 148.

Problem: Procurement; Administrative (N=189): 70; Programmatic 
(N=1,017): 41; Joint: (N=26): 12; Total: 123.

Problem: Program income; Administrative (N=189): 1; Programmatic 
(N=1,017): 1; Joint: (N=26): 0; Total: 2.

Problem: Progress reports; Administrative (N=189): 13; Programmatic 
(N=1,017): 167; Joint: (N=26): 4; Total: 184.

Problem: Property management; Administrative (N=189): 12; Programmatic 
(N=1,017): 0; Joint: (N=26): 0; Total: 12.

Problem: Quality assurance; Administrative (N=189): 0; Programmatic 
(N=1,017): 71; Joint: (N=26): 1; Total: 72.

Problem: Subagreements; Administrative (N=189): 1; Programmatic 
(N=1,017): 9; Joint: (N=26): 0; Total: 10.

Problem: Technical issues; Administrative (N=189): 20; Programmatic 
(N=1,017): 308; Joint: (N=26): 6; Total: 334.

Problem: Terms and conditions of work; Administrative (N=189): 3; 
Programmatic (N=1,017): 30; Joint: (N=26): 2; Total: 35.

Problem: Travel; Administrative (N=189): 16; Programmatic (N=1,017): 3; 
Joint: (N=26): 0; Total: 19.

Problem: Written procedures; Administrative (N=189): 73; Programmatic 
(N=1,017): 11; Joint: (N=26): 2; Total: 86.

Problem: Total; Administrative (N=189): 379; Programmatic (N=1,017): 
830; Joint: (N=26): 41; Total: 1,250.

Source: GAO's analysis of EPA's in-depth reviews.

[End of table]

Table 8: Number of Problems by Type of Grantee:

Problem: Accounting; Nonprofits: (N=244): 17; States: (N=338): 17; 
Native American tribes (N=327): 27; Local: government: (N=126): 2; 
Universities: (N=160): 1; Other: (N=37): 0; Total: 64.

Problem: Administrative; Nonprofits: (N=244): 12; States: (N=338): 13; 
Native American tribes (N=327): 14; Local: government: (N=126): 0; 
Universities: (N=160): 0; Other: (N=37): 1; Total: 40.

Problem: Approval of modifications; Nonprofits: (N=244): 1; States: 
(N=338): 3; Native American tribes (N=327): 0; Local: government: 
(N=126): 1; Universities: (N=160): 1; Other: (N=37): 0; Total: 6.

Problem: Conflict of interest; Nonprofits: (N=244): 4; States: (N=338): 
0; Native American tribes (N=327): 1; Local: government: (N=126): 0; 
Universities: (N=160): 0; Other: (N=37): 0; Total: 5.

Problem: Cost sharing; Nonprofits: (N=244): 1; States: (N=338): 3; 
Native American tribes (N=327): 7; Local: government: (N=126): 2; 
Universities: (N=160): 1; Other: (N=37): 0; Total: 14.

Problem: Financial expenditures; Nonprofits: (N=244): 13; States: 
(N=338): 18; Native American tribes (N=327): 20; Local: government: 
(N=126): 2; Universities: (N=160): 2; Other: (N=37): 0; Total: 55.

Problem: Indirect costs; Nonprofits: (N=244): 7; States: (N=338): 0; 
Native American tribes (N=327): 12; Local: government: (N=126): 0; 
Universities: (N=160): 0; Other: (N=37): 0; Total: 19.

Problem: Internal controls; Nonprofits: (N=244): 10; States: (N=338): 
1; Native American tribes (N=327): 3; Local: government: (N=126): 0; 
Universities: (N=160): 0; Other: (N=37): 0; Total: 14.

Problem: Lobbying; Nonprofits: (N=244): 0; States: (N=338): 0; Native 
American tribes (N=327): 0; Local: government: (N=126): 0; 
Universities: (N=160): 0; Other: (N=37): 0; Total: 0.

Problem: Missing required audit; Nonprofits: (N=244): 2; States: 
(N=338): 2; Native American tribes (N=327): 4; Local: government: 
(N=126): 0; Universities: (N=160): 0; Other: (N=37): 0; Total: 8.

Problem: Personnel/payroll; Nonprofits: (N=244): 27; States: (N=338): 
43; Native American tribes (N=327): 58; Local: government: (N=126): 11; 
Universities: (N=160): 7; Other: (N=37): 2; Total: 148.

Problem: Procurement; Nonprofits: (N=244): 37; States: (N=338): 18; 
Native American tribes (N=327): 39; Local: government: (N=126): 19; 
Universities: (N=160): 7; Other: (N=37): 3; Total: 123.

Problem: Program income; Nonprofits: (N=244): 1; States: (N=338): 1; 
Native American tribes (N=327): 0; Local: government: (N=126): 0; 
Universities: (N=160): 0; Other: (N=37): 0; Total: 2.

Problem: Progress reports; Nonprofits: (N=244): 28; States: (N=338): 
26; Native American tribes (N=327): 88; Local: government: (N=126): 24; 
Universities: (N=160): 14; Other: (N=37): 4; Total: 184.

Problem: Property management; Nonprofits: (N=244): 2; States: (N=338): 
0; Native American tribes (N=327): 9; Local: government: (N=126): 0; 
Universities: (N=160): 1; Other: (N=37): 0; Total: 12.

Problem: Quality assurance; Nonprofits: (N=244): 5; States: (N=338): 
16; Native American tribes (N=327): 18; Local: government: (N=126): 26; 
Universities: (N=160): 6; Other: (N=37): 1; Total: 72.

Problem: Subagreements; Nonprofits: (N=244): 1; States: (N=338): 8; 
Native American tribes (N=327): 0; Local: government: (N=126): 1; 
Universities: (N=160): 0; Other: (N=37): 0; Total: 10.

Problem: Technical issues; Nonprofits: (N=244): 48; States: (N=338): 
117; Native American tribes (N=327): 119; Local: government: (N=126): 
33; Universities: (N=160): 12; Other: (N=37): 5; Total: 334.

Problem: Terms and conditions of work; Nonprofits: (N=244): 8; States: 
(N=338): 16; Native American tribes (N=327): 8; Local: government: 
(N=126): 1; Universities: (N=160): 2; Other: (N=37): 0; Total: 35.

Problem: Travel; Nonprofits: (N=244): 4; States: (N=338): 1; Native 
American tribes (N=327): 14; Local: government: (N=126): 0; 
Universities: (N=160): 0; Other: (N=37): 0; Total: 19.

Problem: Written procedures; Nonprofits: (N=244): 40; States: (N=338): 
8; Native American tribes (N=327): 27; Local: government: (N=126): 6; 
Universities: (N=160): 5; Other: (N=37): 0; Total: 86.

Problem: Total; Nonprofits: (N=244): 268; States: (N=338): 311; Native 
American tribes (N=327): 468; Local: government: (N=126): 128; 
Universities: (N=160): 59; Other: (N=37): 16; Total: 1,250.

Source: GAO's analysis of EPA's in-depth reviews.

[End of table]

Table 9: Number of Problems by Dollar Amount of Grants:

Dollars in thousands.

Problem: 

Accounting; Dollar amount not identified[A]: (N=537): $30; 1-99 
(N=222): $7; 100-299 (N=200): $5; 300-999 (N=128): $4; 1,000 + (N=145): 
$18; Total: $64.

Administrative; Dollar amount not identified[A]: (N=537): 19; 1-99 
(N=222): 5; 100-299 (N=200): 3; 300-999 (N=128): 5; 1,000 + (N=145): 8; 
Total: 40.

Approval of modifications; Dollar amount not identified[A]: (N=537): 
3; 1-99 (N=222): 0; 100-299 (N=200): 1; 300-999 (N=128): 0; 1,000 + 
(N=145): 2; Total: 6.

Conflict of interest; Dollar amount not identified[A]: (N=537): 1; 1-
99 (N=222): 0; 100-299 (N=200): 1; 300-999 (N=128): 1; 1,000 + (N=145): 
2; Total: 5.

Cost sharing; Dollar amount not identified[A]: (N=537): 10; 1-99 
(N=222): 1; 100-299 (N=200): 1; 300-999 (N=128): 1; 1,000 + (N=145): 1; 
Total: 14.

Financial expenditures; Dollar amount not identified[A]: (N=537): 28; 
1-99 (N=222): 8; 100-299 (N=200): 4; 300-999 (N=128): 1; 1,000 + 
(N=145): 14; Total: 55.

Indirect costs; Dollar amount not identified[A]: (N=537): 10; 1-99 
(N=222): 0; 100-299 (N=200): 1; 300-999 (N=128): 3; 1,000 + (N=145): 5; 
Total: 19.

Internal controls; Dollar amount not identified[A]: (N=537): 4; 1-99 
(N=222): 0; 100-299 (N=200): 3; 300-999 (N=128): 2; 1,000 + (N=145): 5; 
Total: 14.

Lobbying; Dollar amount not identified[A]: (N=537): 0; 1-99 (N=222): 
0; 100-299 (N=200): 0; 300-999 (N=128): 0; 1,000 + (N=145): 0; Total: 
0.

Missing required audit; Dollar amount not identified[A]: (N=537): 4; 
1-99 (N=222): 0; 100-299 (N=200): 1; 300-999 (N=128): 0; 1,000 + 
(N=145): 3; Total: 8.

Personnel/payroll; Dollar amount not identified[A]: (N=537): 68; 1-99 
(N=222): 13; 100-299 (N=200): 25; 300-999 (N=128): 18; 1,000 + (N=145): 
24; Total: 148.

Procurement; Dollar amount not identified[A]: (N=537): 53; 1-99 
(N=222): 12; 100-299 (N=200): 25; 300-999 (N=128): 16; 1,000 + (N=145): 
17; Total: 123.

Program income; Dollar amount not identified[A]: (N=537): 0; 1-99 
(N=222): 0; 100-299 (N=200): 1; 300-999 (N=128): 1; 1,000 + (N=145): 0; 
Total: 2.

Progress reports; Dollar amount not identified[A]: (N=537): 80; 1-99 
(N=222): 39; 100-299 (N=200): 34; 300-999 (N=128): 16; 1,000 + (N=145): 
15; Total: 184.

Property management; Dollar amount not identified[A]: (N=537): 3; 1-
99 (N=222): 0; 100-299 (N=200): 2; 300-999 (N=128): 3; 1,000 + (N=145): 
4; Total: 12.

Quality assurance; Dollar amount not identified[A]: (N=537): 39; 1-99 
(N=222): 8; 100-299 (N=200): 17; 300-999 (N=128): 4; 1,000 + (N=145): 
4; Total: 72.

Subagreements; Dollar amount not identified[A]: (N=537): 4; 1-99 
(N=222): 0; 100-299 (N=200): 3; 300-999 (N=128): 0; 1,000 + (N=145): 3; 
Total: 10.

Technical issues; Dollar amount not identified[A]: (N=537): 139; 1-99 
(N=222): 60; 100-299 (N=200): 68; 300-999 (N=128): 23; 1,000 + (N=145): 
44; Total: 334.

Terms and conditions of work; Dollar amount not identified[A]: 
(N=537): 21; 1-99 (N=222): 4; 100-299 (N=200): 1; 300-999 (N=128): 2; 
1,000 + (N=145): 7; Total: 35.

Travel; Dollar amount not identified[A]: (N=537): 9; 1-99 (N=222): 3; 
100-299 (N=200): 2; 300-999 (N=128): 2; 1,000 + (N=145): 3; Total: 19.

Written procedures; Dollar amount not identified[A]: (N=537): 41; 1-
99 (N=222): 7; 100-299 (N=200): 16; 300-999 (N=128): 10; 1,000 + 
(N=145): 12; Total: 86.

Total; Dollar amount not identified[A]: (N=537): $566; 1-99 (N=222): 
$167; 100-299 (N=200): $214; 300-999 (N=128): $112; 1,000 + (N=145): 
$191; Total: $1,250.

Source: GAO's analysis of EPA's in-depth reviews.

[A] We could not identify the value of the grant award for these 
reviews.

[End of table]

Table 10: Number of Problems by Type of Review:

Problem: 

Accounting; Off-site: Administrative: (N=100): 2; Off-site: 
Programmatic: (N=526): 9; Off-site: Joint: (N=18): 1; On-site: 
Administrative: (N=89): 35; On-site: Programmatic: (N=491): 17; On-
site: Joint: (N=8): 0; Total: 64.

Administrative; Off-site: Administrative: (N=100): 6; Off-
site: Programmatic: (N=526): 9; Off-site: Joint: (N=18): 0; 
On-site: Administrative: (N=89): 10; On-site: Programmatic: (N=491): 
14; On-site: Joint: (N=8): 1; Total: 40.

Approval of modifications; Off-site: Administrative: (N=100): 
1; Off-site: Programmatic: (N=526): 1; Off-site: Joint: (N=18): 0; 
[Empty]; On-site: Administrative: (N=89): 1; On-site: Programmatic: 
(N=491): 3; On-site: Joint: (N=8): 0; Total: 6.

Conflict of interest; Off-site: Administrative: (N=100): 0; 
Off-site: Programmatic: (N=526): 0; Off-site: Joint: (N=18): 0; 
[Empty]; On-site: Administrative: (N=89): 5; On-site: Programmatic: 
(N=491): 0; On-site: Joint: (N=8): 0; Total: 5.

Cost sharing; Off-site: Administrative: (N=100): 1; Off-site: 
Programmatic: (N=526): 3; Off-site: Joint: (N=18): 0; On-site: 
Administrative: (N=89): 9; On-site: Programmatic: (N=491): 1; On-site: 
Joint: (N=8): 0; Total: 14.

Financial expenditures; Off-site: Administrative: (N=100): 2; 
Off-site: Programmatic: (N=526): 17; Off-site: Joint: (N=18): 0; 
[Empty]; On-site: Administrative: (N=89): 18; On-site: Programmatic: 
(N=491): 18; On-site: Joint: (N=8): 0; Total: 55.

Indirect costs; Off-site: Administrative: (N=100): 2; Off-
site: Programmatic: (N=526): 0; Off-site: Joint: (N=18): 3; 
On-site: Administrative: (N=89): 10; On-site: Programmatic: (N=491): 2; 
On-site: Joint: (N=8): 2; Total: 19.

Internal controls; Off-site: Administrative: (N=100): 3; Off-
site: Programmatic: (N=526): 0; Off-site: Joint: (N=18): 0; 
On-site: Administrative: (N=89): 10; On-site: Programmatic: (N=491): 1; 
On-site: Joint: (N=8): 0; Total: 14.

Lobbying; Off-site: Administrative: (N=100): 0; Off-site: 
Programmatic: (N=526): 0; Off-site: Joint: (N=18): 0; On-site: 
Administrative: (N=89): 0; On-site: Programmatic: (N=491): 0; On-site: 
Joint: (N=8): 0; Total: 0.

Missing required audit; Off-site: Administrative: (N=100): 1; 
Off-site: Programmatic: (N=526): 0; Off-site: Joint: (N=18): 2; 
[Empty]; On-site: Administrative: (N=89): 3; On-site: Programmatic: 
(N=491): 2; On-site: Joint: (N=8): 0; Total: 8.

Personnel/payroll; Off-site: Administrative: (N=100): 11; 
Off-site: Programmatic: (N=526): 39; Off-site: Joint: (N=18): 3; 
[Empty]; On-site: Administrative: (N=89): 40; On-site: Programmatic: 
(N=491): 53; On-site: Joint: (N=8): 2; Total: 148.

Procurement; Off-site: Administrative: (N=100): 26; Off-site: 
Programmatic: (N=526): 14; Off-site: Joint: (N=18): 8; On-
site: Administrative: (N=89): 44; On-site: Programmatic: (N=491): 27; 
On-site: Joint: (N=8): 4; Total: 123.

Program income; Off-site: Administrative: (N=100): 1; Off-
site: Programmatic: (N=526): 0; Off-site: Joint: (N=18): 0; 
On-site: Administrative: (N=89): 0; On-site: Programmatic: (N=491): 1; 
On-site: Joint: (N=8): 0; Total: 2.

Progress reports; Off-site: Administrative: (N=100): 3; Off-
site: Programmatic: (N=526): 88; Off-site: Joint: (N=18): 3; 
On-site: Administrative: (N=89): 10; On-site: Programmatic: (N=491): 
79; On-site: Joint: (N=8): 1; Total: 184.

Property management; Off-site: Administrative: (N=100): 2; 
Off-site: Programmatic: (N=526): 0; Off-site: Joint: (N=18): 0; 
[Empty]; On-site: Administrative: (N=89): 10; On-site: Programmatic: 
(N=491): 0; On-site: Joint: (N=8): 0; Total: 12.

Quality assurance; Off-site: Administrative: (N=100): 0; Off-
site: Programmatic: (N=526): 48; Off-site: Joint: (N=18): 1; 
On-site: Administrative: (N=89): 0; On-site: Programmatic: (N=491): 23; 
On-site: Joint: (N=8): 0; Total: 72.

Subagreements; Off-site: Administrative: (N=100): 0; Off-
site: Programmatic: (N=526): 4; Off-site: Joint: (N=18): 0; 
On-site: Administrative: (N=89): 1; On-site: Programmatic: (N=491): 5; 
On-site: Joint: (N=8): 0; Total: 10.

Technical issues; Off-site: Administrative: (N=100): 5; Off-
site: Programmatic: (N=526): 165; Off-site: Joint: (N=18): 5; 
On-site: Administrative: (N=89): 15; On-site: Programmatic: (N=491): 
143; On-site: Joint: (N=8): 1; Total: 334.

Terms and conditions of work; Off-site: Administrative: 
(N=100): 0; Off-site: Programmatic: (N=526): 19; Off-site: Joint: 
(N=18): 1; On-site: Administrative: (N=89): 3; On-site: 
Programmatic: (N=491): 11; On-site: Joint: (N=8): 1; Total: 
35.

Travel; Off-site: Administrative: (N=100): 0; Off-site: 
Programmatic: (N=526): 0; Off-site: Joint: (N=18): 0; On-site: 
Administrative: (N=89): 16; On-site: Programmatic: (N=491): 3; On-site: 
Joint: (N=8): 0; Total: 19.

Written procedures; Off-site: Administrative: (N=100): 27; 
Off-site: Programmatic: (N=526): 5; Off-site: Joint: (N=18): 2; 
[Empty]; On-site: Administrative: (N=89): 46; On-site: Programmatic: 
(N=491): 6; On-site: Joint: (N=8): 0; Total: 86.

Total; Off-site: Administrative: (N=100): 93; Off-site: 
Programmatic: (N=526): 421; Off-site: Joint: (N=18): 29; On-
site: Administrative: (N=89): 286; On-site: Programmatic: (N=491): 409; 
On-site: Joint: (N=8): 12; Total: 1,250.

Source: GAO's analysis of EPA's in-depth reviews.

[End of table]

Table 11: Number of Problems Per In-depth Review:

Number of problems Identified: 0; Number of reviews (N=1,232): 623; 
Percentage: of reviews: 51.

Number of problems Identified: 1; Number of reviews (N=1,232): 286; 
Percentage: of reviews: 23.

Number of problems Identified: 2; Number of reviews (N=1,232): 166; 
Percentage: of reviews: 13.

Number of problems Identified: 3; Number of reviews (N=1,232): 83; 
Percentage: of reviews: 7.

Number of problems Identified: 4; Number of reviews (N=1,232): 36; 
Percentage: of reviews: 3.

Number of problems Identified: 5; Number of reviews (N=1,232): 17; 
Percentage: of reviews: 1.

Number of problems Identified: 6; Number of reviews (N=1,232): 9; 
Percentage: of reviews: 1.

Number of problems Identified: 7; Number of reviews (N=1,232): 4; 
Percentage: of reviews: 0.

Number of problems Identified: 8; Number of reviews (N=1,232): 4; 
Percentage: of reviews: 0.

Number of problems Identified: 9; Number of reviews (N=1,232): 1; 
Percentage: of reviews: 0.

Number of problems Identified: 10; Number of reviews (N=1,232): 2; 
Percentage: of reviews: 0.

Number of problems Identified: 11; Number of reviews (N=1,232): 1; 
Percentage: of reviews: 0.

Number of problems Identified: Total; Number of reviews (N=1,232): 
1,232; Percentage: of reviews: 100.

Source: GAO's analysis of EPA's in-depth reviews.

Note: Percentage does not add up to 100 due to rounding.

[End of table]

Table 12: Description of Problems Identified in EPA's Reviews:

Problem: Accounting; Description of problems included in EPA's in-depth 
reviews: Any failure of a grantee's financial management system or 
shortcomings in the procedures it used to ensure the proper accounting 
of federal funds. For example, EPA found cases in which a grantee: * 
could not compare budgeted amounts with actual expenditures, * did not 
properly reconcile report balances to the general ledger, or; * did not 
separately track funds for different grants.

Problem: Administrative; Description of problems included in EPA's in-
depth reviews: Cases in which a grantee's record keeping system was 
inadequate.

Problem: Approval of modifications; Description of problems included in 
EPA's in-depth reviews: Any instance in which a grantee had begun 
performing tasks outside its original scope of work without seeking 
prior approval from EPA.

Problem: Conflict of interest; Description of problems included in 
EPA's in-depth reviews: Cases in which a grantee, using grant funds, 
entered into a contract with a closely affiliated organization.

Problem: Cost sharing; Description of problems included in EPA's in-
depth reviews: Cases in which a grantee failed to appropriately track 
and document its cost-sharing expenditures.

Problem: Financial expenditures; Description of problems included in 
EPA's in-depth reviews: Cases in which the grantee did not sufficiently 
document expenses to determine the eligibility of costs or charged 
ineligible costs to the grant.

Problem: Indirect costs; Description of problems included in EPA's in-
depth reviews: Cases in which a grantee did not have an approved 
indirect cost rate, or indirect cost allocation plan.

Problem: Internal controls; Description of problems included in EPA's 
in-depth reviews: Cases in which a grantee did not adequately segregate 
financial responsibilities.

Problem: Lobbying; Description of problems included in EPA's in-depth 
reviews: No instances of lobbying problems.

Problem: Missing required audit; Description of problems included in 
EPA's in-depth reviews: Cases in which a grantee did not have its 
required audit performed or had not submitted a copy of its audit to 
EPA.

Problem: Personnel/payroll; Description of problems included in EPA's 
in-depth reviews: Problems varied depending on the type of review 
conducted. Administrative reviews included cases in which a grantee did 
not track the amount of time its employees spent on specific grant 
activities. Programmatic reviews included cases in which grantees did 
not have sufficient staff resources to perform the grant activities.

Problem: Procurement; Description of problems included in EPA's in-
depth reviews: Cases in which grantees lacked documentation to support 
sole-source contracts and did not report their efforts to encourage 
procurement from minority-and woman-owned businesses.

Problem: Program income; Description of problems included in EPA's in-
depth reviews: In one case, a grantee generated income through the use 
of grant funds but did not manage the funds in accordance with the 
grant agreement, and in another the grant agreement did not allow the 
grantee to generate such income.

Problem: Progress reports; Description of problems included in EPA's 
in-depth reviews: Instances in which a grantee's progress report was 
missing or late, or did not include all the necessary information.

Problem: Property management; Description of problems included in EPA's 
in-depth reviews: Cases in which the grantee did not properly control 
property, such as equipment.

Problem: Quality assurance; Description of problems included in EPA's 
in-depth reviews: Instances in which a grantee needed to revise its 
quality assurance plan. Quality assurance plans are required to ensure 
the quality of data collected during the grant work.

Problem: Subagreements; Description of problems included in EPA's in-
depth reviews: Cases in which a grantee did not properly monitor 
subgrantees, or when a subgrantee's files were incomplete.

Problem: Technical issues; Description of problems included in EPA's 
in-depth reviews: Cases in which a grantee was behind in the progress 
of its work.

Problem: Terms and conditions; Description of problems included in 
EPA's in-depth reviews: Cases in which a grantee was not meeting the 
terms and conditions of a grant agreement. Terms and conditions vary 
depending on the grant agreement and in some cases overlap with the 
other problem categories.

Problem: Travel; Description of problems included in EPA's in-depth 
reviews: Cases in which grantee lacked documentation to support travel 
expenditures or did not obtain written approval from the appropriate 
official prior to incurring travel expenses.

Problem: Written procedures; Description of problems included in EPA's 
in-depth reviews: Cases in which a grantee's written policies or 
procedures were either missing or inadequate.

Source: GAO's analysis of EPA's in-depth reviews.

[End of table]

Analysis of Corrective Actions Identified in EPA's In-depth Reviews:

Table 13: Number of Problems and Corrective Actions:

Problem: Accounting; Number of problems: 64; Number of corrective 
actions: 51; Percentage of problems with corrective action: 80.

Problem: Administrative; Number of problems: 40; Number of corrective 
actions: 25; Percentage of problems with corrective action: 63.

Problem: Approval of modifications; Number of problems: 6; Number of 
corrective actions: 5; Percentage of problems with corrective action: 
83.

Problem: Conflict of interest; Number of problems: 5; Number of 
corrective actions: 2; Percentage of problems with corrective action: 
40.

Problem: Cost sharing; Number of problems: 14; Number of corrective 
actions: 10; Percentage of problems with corrective action: 71.

Problem: Financial expenditures; Number of problems: 55; Number of 
corrective actions: 35; Percentage of problems with corrective action: 
64.

Problem: Indirect costs; Number of problems: 19; Number of corrective 
actions: 13; Percentage of problems with corrective action: 68.

Problem: Internal controls; Number of problems: 14; Number of 
corrective actions: 13; Percentage of problems with corrective action: 
93.

Problem: Lobbying; Number of problems: 0; Number of corrective actions: 
0; Percentage of problems with corrective action: 0.

Problem: Missing required audit; Number of problems: 8; Number of 
corrective actions: 7; Percentage of problems with corrective action: 
88.

Problem: Personnel/payroll; Number of problems: 148; Number of 
corrective actions: 65; Percentage of problems with corrective action: 
44.

Problem: Procurement; Number of problems: 123; Number of corrective 
actions: 81; Percentage of problems with corrective action: 66.

Problem: Program income; Number of problems: 2; Number of corrective 
actions: 0; Percentage of problems with corrective action: 0.

Problem: Progress reports; Number of problems: 184; Number of 
corrective actions: 74; Percentage of problems with corrective action: 
40.

Problem: Property management; Number of problems: 12; Number of 
corrective actions: 10; Percentage of problems with corrective action: 
83.

Problem: Quality assurance; Number of problems: 72; Number of 
corrective actions: 18; Percentage of problems with corrective action: 
25.

Problem: Subagreements; Number of problems: 10; Number of corrective 
actions: 8; Percentage of problems with corrective action: 80.

Problem: Technical issues; Number of problems: 334; Number of 
corrective actions: 124; Percentage of problems with corrective action: 
37.

Problem: Terms and conditions of work; Number of problems: 35; Number 
of corrective actions: 14; Percentage of problems with corrective 
action: 40.

Problem: Travel; Number of problems: 19; Number of corrective actions: 
17; Percentage of problems with corrective action: 89.

Problem: Written procedures; Number of problems: 86; Number of 
corrective actions: 68; Percentage of problems with corrective action: 
79.

Problem: Total; Number of problems: 1,250; Number of corrective 
actions: 640; Percentage of problems with corrective action: 51 .

Source: GAO's analysis of EPA's in-depth reviews.

[End of table]

Analysis of Protocols Used for EPA's In-depth Reviews:

Table 14: Number of Mandated and Nonmandated Reviews:

Mandated; Number: 339; Percentage: 28.

Nonmandated; Number: 893; Percentage: 72.

Total; Number: 1,232; Percentage: 100.

Source: GAO's analysis of EPA's in-depth reviews.

Note: Mandated reviews were defined in the GAO analysis as evaluations 
conducted as a result of statutory or periodic programmatic 
requirements.

[End of table]

Table 15: Number of Nonmandated Reviews With or Without a Review 
Protocol, by Program and Regional Office, and by Type of Review:

Program/regional office: 

Office of Grants and Debarment; Administrative: Protocol used: 
35; Administrative: Unclear/no protocol: 37; Programmatic: 
Protocol used: 0; Programmatic: Unclear/no protocol: 0; Joint: 
Protocol used: 0; Joint: Unclear/no protocol: 0.

Office of the Administrator; Administrative: Protocol used: 0; 
Administrative: Unclear/no protocol: 0; Programmatic: 
Protocol used: 51; Programmatic: Unclear/no protocol: 1; 
Joint: Protocol used: 0; Joint: Unclear/no protocol: 0.

Office of Air and Radiation; Administrative: Protocol used: 0; 
Administrative: Unclear/no protocol: 0; Programmatic: 
Protocol used: 22; Programmatic: Unclear/no protocol: 1; 
Joint: Protocol used: 0; Joint: Unclear/no protocol: 0.

Office of Enforcement and Compliance Assurance; 
Administrative: Protocol used: 0; Administrative: Unclear/no protocol: 
0; Programmatic: Protocol used: 3; Programmatic: Unclear/no 
protocol: 0; Joint: Protocol used: 0; Joint: Unclear/no 
protocol: 0.

Office of Environmental Information; Administrative: Protocol 
used: 0; Administrative: Unclear/no protocol: 0; Programmatic: 
Protocol used: 2; Programmatic: Unclear/no protocol: 0; Joint: 
Protocol used: 0; Joint: Unclear/no protocol: 0.

Office of International Affairs; Administrative: Protocol 
used: 0; Administrative: Unclear/no protocol: 0; Programmatic: 
Protocol used: 0; Programmatic: Unclear/no protocol: 3; Joint: 
Protocol used: 0; Joint: Unclear/no protocol: 0.

Office of Prevention, Pesticides and Toxic Substances; 
Administrative: Protocol used: 0; Administrative: Unclear/no protocol: 
0; Programmatic: Protocol used: 8; Programmatic: Unclear/no 
protocol: 0; Joint: Protocol used: 0; Joint: Unclear/no 
protocol: 0.

Office of Research and Development; Administrative: Protocol 
used: 0; Administrative: Unclear/no protocol: 0; Programmatic: 
Protocol used: 68; Programmatic: Unclear/no protocol: 47; 
Joint: Protocol used: 0; Joint: Unclear/no protocol: 0.

Office of Solid Waste and Emergency Response; Administrative: 
Protocol used: 0; Administrative: Unclear/no protocol: 0; 
Programmatic: Protocol used: 7; Programmatic: Unclear/no protocol: 0; 
Joint: Protocol used: 0; Joint: Unclear/no protocol: 0.

Office of Water; Administrative: Protocol used: 0; 
Administrative: Unclear/no protocol: 0; Programmatic: 
Protocol used: 38; Programmatic: Unclear/no protocol: 8; 
Joint: Protocol used: 0; Joint: Unclear/no protocol: 0.

Region 1; Administrative: Protocol used: 8; Administrative: 
Unclear/no protocol: 3; Programmatic: Protocol used: 1; 
Programmatic: Unclear/no protocol: 1; Joint: Protocol used: 3; 
Joint: Unclear/no protocol: 3.

Region 2; Administrative: Protocol used: 1; Administrative: 
Unclear/no protocol: 4; Programmatic: Protocol used: 0; 
Programmatic: Unclear/no protocol: 7; Joint: Protocol used: 1; 
Joint: Unclear/no protocol: 0.

Region 3; Administrative: Protocol used: 16; Administrative: 
Unclear/no protocol: 0; Programmatic: Protocol used: 17; 
Programmatic: Unclear/no protocol: 11; Joint: Protocol used: 
0; Joint: Unclear/no protocol: 1.

Region 4; Administrative: Protocol used: 0; Administrative: 
Unclear/no protocol: 0; Programmatic: Protocol used: 112; 
Programmatic: Unclear/no protocol: 29; Joint: Protocol used: 
0; Joint: Unclear/no protocol: 0.

Region 5; Administrative: Protocol used: 8; Administrative: 
Unclear/no protocol: 4; Programmatic: Protocol used: 0; 
Programmatic: Unclear/no protocol: 11; Joint: Protocol used: 
0; Joint: Unclear/no protocol: 0.

Region 6; Administrative: Protocol used: 8; Administrative: 
Unclear/no protocol: 1; Programmatic: Protocol used: 28; 
Programmatic: Unclear/no protocol: 33; Joint: Protocol used: 
0; Joint: Unclear/no protocol: 0.

Region 7; Administrative: Protocol used: 12; Administrative: 
Unclear/no protocol: 1; Programmatic: Protocol used: 27; 
Programmatic: Unclear/no protocol: 7; Joint: Protocol used: 7; 
Joint: Unclear/no protocol: 0.

Region 8; Administrative: Protocol used: 8; Administrative: 
Unclear/no protocol: 4; Programmatic: Protocol used: 51; 
Programmatic: Unclear/no protocol: 2; Joint: Protocol used: 0; 
Joint: Unclear/no protocol: 0.

Region 9; Administrative: Protocol used: 2; Administrative: 
Unclear/no protocol: 32; Programmatic: Protocol used: 12; 
Programmatic: Unclear/no protocol: 27; Joint: Protocol used: 
1; Joint: Unclear/no protocol: 1.

Region 10; Administrative: Protocol used: 0; Administrative: 
Unclear/no protocol: 5; Programmatic: Protocol used: 40; 
Programmatic: Unclear/no protocol: 3; Joint: Protocol used: 2; 
Joint: Unclear/no protocol: 7.

Total; Administrative: Protocol used: 98; Administrative: 
Unclear/no protocol: 91; Programmatic: Protocol used: 487; 
Programmatic: Unclear/no protocol: 191; Joint: Protocol used: 
14; Joint: Unclear/no protocol: 12.

Source: GAO's analysis of EPA's in-depth reviews.

[End of table]

Analysis of Significant Actions Taken Against Grantees as a Result of 
In-depth Reviews:

Table 16: Number of Significant Actions Taken Against Grantees by Type 
of Grantee:

Action taken: Agreement termination; Nonprofits (N=244): 1; States 
(N=338): 0; Native American tribes: N=327): 0; Local government: 
(N=126): 0; Universities (N=160): 0; Other (N=37): 0; Total: 1.

Action taken: Annulment; Nonprofits (N=244): 0; States (N=338): 0; 
Native American tribes: N=327): 0; Local government: (N=126): 0; 
Universities (N=160): 0; Other (N=37): 0; Total: 0.

Action taken: Disallowance of costs; Nonprofits (N=244): 5; States 
(N=338): 1; Native American tribes: N=327): 1; Local government: 
(N=126): 0; Universities (N=160): 0; Other (N=37): 0; Total: 7.

Action taken: Grantee debarment; Nonprofits (N=244): 0; States (N=338): 
0; Native American tribes: N=327): 0; Local government: (N=126): 0; 
Universities (N=160): 0; Other (N=37): 0; Total: 0.

Action taken: High risk designation/ special condition; Nonprofits 
(N=244): 3; States (N=338): 0; Native American tribes: N=327): 1; Local 
government: (N=126): 0; Universities (N=160): 0; Other (N=37): 0; 
Total: 4.

Action taken: Payment hold; Nonprofits (N=244): 4; States (N=338): 4; 
Native American tribes: N=327): 7; Local government: (N=126): 0; 
Universities (N=160): 0; Other (N=37): 0; Total: 15.

Action taken: Referral for investigation; Nonprofits (N=244): 2; States 
(N=338): 0; Native American tribes: N=327): 2; Local government: 
(N=126): 0; Universities (N=160): 0; Other (N=37): 0; Total: 4.

Action taken: Stop work orders; Nonprofits (N=244): 3; States (N=338): 
0; Native American tribes: N=327): 2; Local government: (N=126): 0; 
Universities (N=160): 0; Other (N=37): 0; Total: 5.

Action taken: Threat to take action; Nonprofits (N=244): 3; States 
(N=338): 0; Native American tribes: N=327): 6; Local government: 
(N=126): 0; Universities (N=160): 0; Other (N=37): 0; Total: 9.

Action taken: Other; Nonprofits (N=244): 0; States (N=338): 0; Native 
American tribes: N=327): 2; Local government: (N=126): 0; Universities 
(N=160): 0; Other (N=37): 0; Total: 2.

Action taken: Total; Nonprofits (N=244): 21; States (N=338): 5; Native 
American tribes: N=327): 21; Local government: (N=126): 0; Universities 
(N=160): 0; Other (N=37): 0; Total: 47.

Source: GAO's analysis of EPA's in-depth reviews.

[End of table]

Table 17: Number of Significant Actions Per In-depth Review:

Number of significant actions: 0; Number of reviews: 1,202; Percentage 
of reviews: 98.

Number of significant actions: 1; Number of reviews: 20; Percentage of 
reviews: 2.

Number of significant actions: 2; Number of reviews: 4; Percentage of 
reviews: 0.

Number of significant actions: 3; Number of reviews: 5; Percentage of 
reviews: 0.

Number of significant actions: 4; Number of reviews: 1; Percentage of 
reviews: 0.

Number of significant actions: Total; Number of reviews: 1,232; 
Percentage of reviews: 100.

Source: GAO's analysis of EPA's in-depth reviews.

[End of table]

[End of section]

Appendix IV: Summary of EPA's Grants Management Plan, 2003-2008:

Goal 1: Enhance the Skills of EPA Personnel Involved in Grants 
Management:

Objectives: 

1.1 Update and enhance training materials and course curricula to  
emphasize high priority areas.

1.2 Improve delivery and availability of training programs.

1.3 Provide training to managers and supervisors.

Table 18 shows the performance measures for goal 1.

Table 18: Goal 1 Performance Measures:

Performance measure: Percentage of grants managed by certified project 
officers; Baseline: 85%; Target: 2003: 100%.

Performance measure: Percentage of grants managed by project officers 
who have taken enhanced project officer refresher course; Baseline: 0%; 
Target: 2004: 30%; 2005: 60%; 2006: 100%.

Source: EPA.

[End of table]

Action steps:

2003:

* Issue Project Officer Training Manual (fifth edition), which will 
focus on core competencies needed to manage grants.

* Conduct project officer training with special emphasis on the core 
competency areas that were enhanced in the fifth edition of the 
training manual.

* Issue first edition of the Grants Specialist Training Manual.

* Conduct grants specialist training focusing on core competency areas.

* Develop long-term Grants Management Training Plan.

* Issue guidance on the proper use of amendments.

* Ensure that project officers receive basic and refresher grants 
management training on a timely basis (ongoing).

* Develop a pilot grants management training program for managers and 
supervisors.

2004:

* Enhance and update on-line reference materials for grant specialists 
in core competencies.

* Implement on-line training for grants specialists in core competency 
areas.

* Enhance on-line training for refresher project officers course to 
include additional materials on preapplication review, competition, 
post-award monitoring, environmental outcomes, and other new areas 
covered in the Project Officer Training Manual, as well as verification 
of completion at each step.

* Conduct grants management classroom training for managers and 
supervisors.

2005:

* Implement on-line basic project officers course, including 
verification of completion at each step.

* Conduct grants management on-line training for managers and 
supervisors.

2006:

* Establish certification program for grants specialists based on an 
examination of skills and core competencies required to manage grants.

Goal 2: Promote Competition in the Award of Grants:

Objectives:

2.1 Improve the agency's process for identifying annual funding 
priorities  and planning for competition.

2.2 Encourage a large and diverse group of grant applicants.

2.3 Promote agencywide understanding of the importance of competition.

2.4 Provide adequate support to the Grants Competition Advocate.

Table 19 shows the performance measures for goal 2.

Table 19: Goal 2 Performance Measures:

Performance measure: Percentage of new grants subject to the 
competition order that are competed; Baseline: 27%; Target: 2003: 30%; 
2004: 60%; 2005: 85%.

Performance measure: Percentage of new grants to nonprofit recipients 
subject to the competition order that are competed; Baseline: 24%; 
Target: 2003: 30%; 2004: 55%; 2005: 75%.

Source: EPA.

[End of table]

Action steps:

2003:

* Provide guidance to national program managers on Catalog of Federal 
Domestic Assistance (CFDA) descriptions that publicize funding 
priorities and opportunities.

* Include national program managers funding priorities in the CFDA on 
an annual basis to ensure they are well advertised and linked to the 
agency's Government Performance and Results Act goals.

* Develop guidelines and make postcompetition award information 
available to the public through the Internet.

* Develop a central grants competition Web page linked to program 
office Web sites to make grant solicitations easily available to the 
public.

* Post all grant solicitations on the Fed Biz Opps Web site.

* Develop standard agencywide grants competition training materials and 
incorporate them into project officer and grant specialist training.

* Provide full staffing for the Grants Competition Advocate.

2004:

* Develop a structured annual planning process for grant competitions.

* Expand public awareness of funding opportunities by improving the 
accuracy and specificity of program descriptions in the CFDA.

* Research innovative grants competition strategies performed by 
federal, state, or local governments and determine suitability for 
adapting them for the agency's use.

2005:

* Implement a structured annual planning process for grant 
competitions.

Goal 3: Leverage Technology to Improve Program Performance:

Objectives:

3.1 Continue deployment of the Integrated Grant Management System 
(IGMS).

3.2 Integrate IGMS with federal-wide e-grant initiatives.

3.3 Enhance and expand information systems that support grants 
oversight.

Table 20 shows performance measures for goal 3.

Table 20: Goal 3 Performance Measures:

Performance measure: Percentage of grants funding packages that are 
submitted electronically; Baseline: Regions: 53%; HQ: 0%; Target: 
Regions: 2003: 65%; 2004: 85%; HQ: 2006: 25%.

Performance measure: Average number of days to process a grant; 
Baseline: 60 days; Target: Regions 2003: 57 days; 2004: 51 days; HQ  
To be established.

Performance measure: Percentage of award transactions transmitted 
electronically into the Integrated Financial Management System (IFMS); 
Baseline: Regions 0%; Target: Regions  2004: 100%; HQ    
100%.

Performance measure: Number of electronic applications received in IGMS 
from e-grants portal; Baseline: [Empty]; Target: Target to be 
established.

Source: EPA.

[End of table]

Action steps:

2003:

* Complete deployment of IGMS in Regions 2, 4, 5, and 8 and of pilot in 
headquarters component.

* Develop an interface between IGMS and the IFMS so that commitment 
notice information need be entered only once.

* Complete analysis of changes required to IGMS and Grants Information 
Control System to enable these systems to interface with federal e-
grants portal.

* Develop and implement an IGMS postaward module.

* Correct inconsistencies and inaccuracies in the naming of grantees, 
grant specialists, and project officers so that accurate reports of 
workload and recipients counts are available.

* Enhance the IGMS electronic grant file reporting views for the 
reports needed to monitor grant specialist and project officer 
workloads.

* Expand the project officer database to track certification and 
recertification activities.

* Establish a tracking system to identify competition status of grants.

2004:

* Enable electronic transmission of award data from IGMS to the IFMS.

* Develop an interface to federal e-grants portal to enable IGMS to 
accept applications.

* Develop a project officer interface to simplify use of IGMS.

* Ensure that agency information technology systems make grants 
information readily available to EPA personnel.

* Create an agency Web site to allow grant specialists and project 
officers to access best practices and other tools for postaward 
management.

2005:

* Expand e-grants portal to enable IGMS to accept reporting.

* Modify IGMS data elements to e-grants standards for reporting.

* Develop an administrative review checklist in IGMS to ensure that 
grant packages are complete, comprehensive and in compliance with EPA 
orders and policies.

2006:

* Complete deployment of IGMS in headquarters.

Goal 4: Strengthen EPA Oversight of Grants:

Objectives:

4.1 Improve grants management reviews of EPA offices.

4.2 Improve and expand external reviews of grant recipients.

4.3 Develop approaches to prevent or limit grants management 
 weaknesses.

4.4 Establish clear lines of accountability for grants oversight.

4.5 Provide high-level coordination, planning, and priority setting for 
grants  management.

Table 21 shows the performance measures for goal 4.

Table 21: Goal 4 Performance Measures:

Performance measure: Percentage of grants awarded in the fourth quarter 
of the agency's fiscal year; Baseline: 67%; Target: 2004: 57%.

Performance measure: Number of comprehensive internal reviews of EPA 
grants management operations; Baseline: 0; Target: Regions 2003: 3; 
HQ 2003: 4.

Performance measure: Percentage of active recipients who receive 
advanced monitoring; Baseline: 5%; Target: 2003: 10%.

Performance measure: Percentage of active recipients who have on-site 
reviews conducted by program office and/or grants management offices; 
Baseline: Establish in 2003; Target: 2004: 5% increase; over baseline.

Performance measure: Percentage of offices that submit post-award 
monitoring plans on time; Baseline: 60%; Target: 2003: 100%.

Performance measure: Percentage of eligible grants closed out; 
Baseline: FY 2001: 89%; FY 2002: 51%; Target: FY 2001: 99%; FY 2002: 
90%.

Source: EPA.

[End of table]

Action steps:

2003:

* Combine the management oversight and postaward validation grants 
management review protocols of offices.

* Conduct cradle-to-grave management reviews of headquarters and 
regional offices starting in 2003.

* Issue revised guidance on grants management self-assessments.

* Require all offices to conduct baseline monitoring of all active 
grantees (ongoing).

* Increase the level of advanced monitoring of grantees by offices, 
including desk reviews and on-site reviews, to a minimum of 10 percent 
annually of active recipients.

* Ensure that all offices record their activities in the agency's 
compliance database including documentation of the results of 
compliance reviews.

* Ensure that all offices submit timely annual postaward monitoring 
plans.

* Ensure that all offices, on an annual basis, define and identify 
categories of at-risk grantees requiring technical and/or compliance 
assistance.

* Complete development, with the EPA's Office of Inspector General, of 
an instructional video for nonprofit recipients on how to manage their 
grants.

* Provide training courses for nonprofit recipients.

* Develop grants management tribal training manual and provide training 
to tribes.

* Develop guidance on required procurement processes under grants, 
including competition, cost and price analyses, and avoidance of 
conflicts of interest, and make available to grantees.

* Clarify roles and responsibilities of senior resource officials.

* Issue grants policy document defining roles and responsibilities of 
grants management officers, program office officials, and project 
officers.

* Establish senior-level Grants Management Council.

* Update Grants Management Office business plans in 2003 and annually 
thereafter.

2004:

* Establish a preaward financial systems review program for at-risk 
grantees.

* Establish grants clearinghouse/hotline for grant recipients.

* Review and update EPA's indirect cost rate policies.

* Ensure that performance standards established for grant specialists 
and project officers adequately address grants management 
responsibilities.

* Develop workload analysis of project officers and grant specialists.

* Establish standard operation procedures and best practices for grants 
management offices.

2005:

* Provide guidance to grants management offices and program offices on 
how resources should be allocated to ensure effective and efficient 
grants management oversight.

2006:

* Develop grants management resource tool to replace the Assistance 
Administration Manual.

Goal 5: Support Identifying and Achieving Environmental Outcomes:

Objectives:

5.1 Include expected environmental outcomes and performance measures 
  in grant work plans.

5.2 Improve reporting on progress made in achieving environmental 
 outcomes.

Table 22 shows the performance measures for goal 5.

Table 22: Goal 5 Performance Measures:

Performance measure: Percentage of grant work plans, decision 
memoranda, and terms of condition that include a discussion of how 
grantees plan to measure and report on environmental progress; 
Baseline: Establish in FY 2003; Target: 2004: 70%; 2005: 80%; 2006: 
100%.

Source: EPA.

[End of table]

Action steps:

2003:

* Develop a tutorial for grantees on how to develop performance 
measures for work plans.

* Issue grants policy guidance to ensure that all grant work plans, 
decision memoranda, and/or terms of condition include environmental 
outcomes and how to measure them.

2004:

* Require a discussion of expected environmental outcomes and 
performance measures in grant solicitations.

2005:

* Establish reporting on environmental outcomes as a criterion for 
approval of interim and final reports.

* Incorporate success in reporting on outcomes into the criteria for 
awarding new grants.

[End of section]

Appendix V: Comments from the Environmental Protection Agency:

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY:

WASHINGTON, D.C. 20460:

AUG 19 2003:

OFFICE OF ADMINISTRATION AND RESOURCES MANAGEMENT:

Mr. John B. Stephenson:

Director, Natural Resources and the Environment U. S. General 
Accounting Office:

Washington, D.C. 20548:

Dear Mr. Stephenson:

Thank you for the opportunity to comment on the U. S. General 
Accounting Office (GAO) draft report entitled "Grants Management: EPA 
Needs to Strengthen Efforts to Address Persistent Challenges (GAO-03-
846).":

I am pleased to note that the draft report recognizes the steps EPA is 
taking to improve grants management, including the development of a 
five-year Grants Management Plan (Plan) and associated policies on 
grant competition and post-award monitoring. The Plan and these 
policies will help ensure that the Agency meets its fiduciary 
obligation to the taxpayers and awards grants that produce measurable 
environmental results.

The draft report contains a number of recommendations for strengthening 
grants management. We agree with those recommendations and will 
implement them as part of our five-year Plan. Under the Plan, we are a 
conducting a workload analysis of project officers and grants 
specialists. This analysis will identify, with greater specificity, 
potential resource needs that will then be addressed within the 
confines of the Agency's budget.

A major theme of the draft report is that EPA must exhibit a sustained 
commitment to accountable grants management. I believe this commitment 
is firmly in place, as evidenced by 
Directives from the Deputy Administrator/Acting Administrator 
requiring senior managers to hold employees accountable for effective 
grants management and to include compliance with grants management 
policies as part of mid-year performance discussions;

The Agency's ongoing review of performance standards, which will ensure 
that the performance agreements of all employees involved in grants 
management adequately reflect their grants management 
responsibilities;

The requirement for Assistant Administrators and Regional 
Administrators to address in their assurance letters under the Federal 
Managers' Financial Integrity Act (FMFIA) the steps they are taking to 
address the grants management weakness; and:

The development of a Tactical Action Plan, which outlines commitments 
and milestone dates under the Grants Management Plan and identifies who 
is responsible for completing those commitments.

Our technical comments on the report are enclosed. I am also enclosing 
for your information a copy of the Office of Grants and Debarment's 
(OGD) Fiscal Year 2003 FMFIA assurance letter. The letter describes in 
detail the actions OGD is taking to strengthen grants management this 
fiscal year, including activities that we have undertaken or completed 
since the development of the draft report.

I want to thank you for the courteous and professional manner in which 
GAO conducted its review, and would like to commend Andrea Brown, Bruce 
Skud, Chris Murray and Paul Schearf of your staff for their excellent 
work.	If you have any questions about these comments, please contact 
Howard Corcoran, Director, Office of Grants and Debarment, at (202) 
564-1903.

Sincerely,

Morris X. Winn 
Assistant Administrator:

Signed by Morris X. Winn: 

Enclosures:

cc: Dave O'Connor: 
Sherry Kaschak: 
Howard Corcoran: 
Richard Kuhlman: 
Marguerite Pridgen: 
David Osterman: 
William Hedling: 
John Nolan:
Sandy Womack-Butler: 
Bernie Davis:
Arlene Bragg 
Steve Tiber:

[End of section]

Appendix VI: GAO Contacts and Staff Acknowledgments:

GAO Contacts:

Andrea Wamstad Brown (202) 512-3319 Bruce Skud (617) 788-0588: 

Staff Acknowledgments:

In addition to those named above, Carl Barden, Christopher Murray, Paul 
Schearf, Rebecca Shea, Carol Herrnstadt Shulman, Kelli Ann Walther, and 
Amy Webbink made key contributions to this report.

[End of section]

Related GAO Products:

[End of section]

Environmental Protection Agency: Problems Persist in Effectively 
Managing Grants. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-
628T] GAO-03-628T. Washington, D.C. June 11, 2003.

Federal Assistance: Grant System Continues to Be Highly Fragmented. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-718T] GAO-03-
718T. Washington, D.C. April 29, 2003.

Results-Oriented Cultures: Creating a Clear Linkage between Individual 
Performance and Organizational Success. [Hyperlink, http://
www.gao.gov/cgi-bin/getrpt?GAO-03-488] GAO-03-488. Washington, D.C. 
March 14, 2003.

Major Management Challenges and Risks: Environmental Protection Agency. 
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-112] GAO-03-112. 
Washington D. C. January 2003.

Results-Oriented Cultures: Using Balanced Expectations to Manage Senior 
Executive Performance. [Hyperlink, http://www.gao.gov/cgi-bin/
getrpt?GAO-02-966] GAO-02-966. Washington, D.C. September 27, 2002.

Environmental Protection: Grants Awarded for Continuing Environmental 
Programs and Projects. [Hyperlink, http://www.gao.gov/cgi-bin/
getrpt?GAO-01-860R] GAO-01-860R. Washington, D.C. June 29, 2001.

Environmental Protection: EPA's Oversight of Nonprofit Grantees' Costs 
Is Limited. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01-366] 
GAO-01-366. Washington, D.C. April 6, 2001.

Environmental Protection: Information on EPA Project Grants and Use of 
Waiver Authority. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-
01-359] GAO-01-359. Washington, D.C. March 9, 2001.

Environmental Research: STAR Grants Focus on Agency Priorities, but 
Management Enhancements Are Possible. [Hyperlink, http://www.gao.gov/
cgi-bin/getrpt?GAO/RCED-00-170] GAO/RCED-00-170. Washington, D.C. 
September 11, 2000.

Environmental Protection: Grants for International Activities and Smart 
Growth. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/RCED-00-
145R] GAO/RCED-00-145R. Washington, D.C. May 31, 2000.

Environmental Protection: Collaborative EPA-State Effort Needed to 
Improve Performance Partnership System. [Hyperlink, http://
www.gao.gov/cgi-bin/getrpt?GAO/T-RCED-00-163] GAO/T-RCED-00-163. 
Washington, D.C. May 2, 2000.

Managing for Results: EPA Faces Challenges in Developing Results-
Oriented Performance Goals and Measures. [Hyperlink, http://
www.gao.gov/cgi-bin/getrpt?GAO/RCED-00-77] GAO/RCED-00-77. Washington, 
D.C. April 28, 2000.

Environmental Protection: Collaborative EPA-State Effort Needed to 
Improve New Performance Partnership System. [Hyperlink, http://
www.gao.gov/cgi-bin/getrpt?GAO/RCED-99-171] GAO/RCED-99-171. 
Washington, D.C. June 21, 1999.

Environmental Protection: Factors Contributing to Lengthy Award Times 
for EPA Grants. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/
RCED-99-204] GAO/RCED-99-204. Washington, D.C. July 14, 1999.

Environmental Protection: EPA's Progress in Closing Completed Grants 
and Contracts. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/RCED-
99-27] GAO/RCED-99-27. Washington, D.C. November 20, 1998.

Dollar Amounts of EPA's Grants and Agreements. [Hyperlink, http://
www.gao.gov/cgi-bin/getrpt?GAO/RCED-96-178R] GAO/RCED-96-178R. 
Washington D.C. May 29, 1996.

(360223):

FOOTNOTES

[1] See, for example, U.S. General Accounting Office, Environmental 
Protection Agency: Problems Persist in Effectively Managing Grants, 
GAO-03-628T (Washington, D.C: June 11, 2003).

[2] U.S. General Accounting Office, Major Management Challenges and 
Program Risks: Environmental Protection Agency, GAO-03-112 
(Washington, D.C. January 2003).

[3] Federal financial assistance includes grants, cooperative 
agreements, loans, loan guarantees, scholarships, and other forms of 
assistance. For this report, we focused on both grants and cooperative 
agreements, and for simplicity, refer to both as "grants."

[4] According to EPA officials, two headquarters' offices, EPA's Office 
of General Counsel and the Office of the Chief Financial Officer 
conduct limited grant activity.

[5] Program offices in Regions 4, 5, 6, 9, and 10 award grants 
directly.

[6] Grant actions include new awards, increase and decrease amendments. 
The 8,070 grant actions involving funding were composed of 4,374 new 
grants, 2,772 increase amendments, and 924 decrease amendments. In 
addition, EPA awarded 1,620 no cost extensions, which did not involve 
funding, in fiscal 2002. 

[7] GAO did not verify EPA's budget data.

[8] The Single Audit Act Amendments of 1996, Pub. L. No. 104-156, 110 
Stat. 1396 (codified at 31 U.S.C. §§ 7501-7507).

[9] The Office of Management and Budget, as authorized by the act, 
increased this amount to $500,000 in federal awards as of June 23, 
2003.

[10] EPA refers to these in-depth reviews as advance monitoring.

[11] See 31 U.S.C. § 3512.

[12] U.S. Environmental Protection Agency, Fiscal Year 2002 Annual 
Report, EPA-190-R-03-001 (Washington, D.C. Jan. 31, 2003).

[13] EPA took these actions through early 2002.

[14] EPA Office of Inspector General, EPA Must Emphasize Importance of 
Pre-Award Reviews for Assistance Agreements, Report No. 2003-P-0007 
(Washington, D.C. Mar. 31, 2003).

[15] Federal Grant and Cooperative Agreement Act of 1977, Pub. L. No. 
95-224, 92 Stat. 3 (codified as amended at 31 U.S.C. §§ 6301-6308).

[16] According to a 2003 National Research Council report, however, 
EPA's Science To Achieve Results (STAR) program effectively awards 
grants for extramural research through its peer-reviewed, competitive 
award process. See National Research Council, The Measure of STAR: 
Review of the U.S. Environmental Protection Agency's Science To Achieve 
Results (STAR) Research Grants Program. (Washington, D.C. 2003).

[17] EPA Office of the Inspector General, EPA's Competitive Practices 
for Assistance Awards, Report No. 2001-P-00008 (Philadelphia, PA: May 
21, 2001). 

[18] EPA Office of Inspector General, Report No. 2001-P-00008. 

[19] The monitoring of key areas is required under EPA Order 5700.4, 
Interim Grantee Compliance Assistance Initiative Policy, (February 
2002). The other four key areas include the (1) progress of work, (2) 
meeting of all programmatic and statutory and regulatory requirements 
are met, (3) equipment purchased under the award is managed and 
accounted for, and (4) financial expenditures. 

[20] U.S. General Accounting Office, Environmental Protection: EPA's 
Oversight of Nonprofit Grantees' Costs Is Limited, GAO-01-366 
(Washington, D.C. Apr. 6, 2001).

[21] OMB has issued three circulars defining allowable costs for 
different types of grantees: A-21, A-87, and A-122.

[22] EPA Office of Inspector General, Procurements Made by Assistance 
Agreement Recipients Should Be Competitive, Report No. 2002-P-00009 
(Philadelphia, PA: Mar. 28, 2002).

[23] EPA Office of Inspector General, Report of Audit on the Center for 
Chesapeake Communities, Report No. E6DEP8-03-0014-9100117 
(Philadelphia, PA: Mar. 31, 1999) and EPA Office of Inspector General, 
Assistance Agreement X993795-01 Awarded by EPA to the Lake 
Wallenpaupack Watershed Management District, Report No. 2002-M-00007 
(Philadelphia, PA: Jan. 18, 2002). 

[24] U.S. General Accounting Office, Environmental Research: STAR 
Grants Focus on Agency Priorities, but Management Enhancements Are 
Possible, GAO/RCED-00-170 (Washington, D.C. Sept. 11, 2000).

[25] EPA Office of Inspector General, Additional Efforts Needed to 
Improve EPA's Oversight of Assistance Agreements, Report No. 2002-P-
00018 (Washington, D.C. Sept. 30, 2002).

[26] Construction grants are grants used for designing and constructing 
facilities such as wastewater treatment plants. Nonconstruction grants 
are grants such as those awarded to states for conducting air quality 
monitoring or grants to universities for conducting environmental 
research. 

[27] A probability sample is one in which each member of the population 
has a known, nonzero probability of selection.

[28] Nonmandated reviews were defined in our analysis as reviews 
conducted for reasons other than statutory or periodic programmatic 
requirements. Of the 893 nonmandated reviews, for 91 of the 
administrative reviews, 191 of the programmatic reviews, and 12 of the 
joint reviews, it was unclear if a protocol was used.

[29] EPA Office of Inspector General, Report No. 2002-P-00018. 

[30] Senior resource officials are typically deputy assistant 
administrators in headquarters offices and assistant regional 
administrators, and are in charge of strengthening agencywide fiscal 
resource management while also ensuring compliance with laws and 
regulations.

[31] U.S. General Accounting Office, Managing for Results: EPA Faces 
Challenges in Developing Results-Oriented Performance Goals and 
Measures, GAO/RCED-00-77 (Washington, D.C. Apr. 28, 2000).

[32] U.S. General Accounting Office, Environmental Protection: 
Information on EPA Project Grants and Use of Waiver Authority, GAO-01-
359 (Washington, D.C. Mar. 9, 2001).

[33] GAO/RCED-00-170. 

[34] The four EPA programs assessed were the Drinking Water State 
Revolving Fund, Leaking Underground Storage Tanks, Nonpoint Source 
Grants, and Tribal General Assistance programs. OMB evaluated these 
programs using its Program Assessment Rating Tool (PART), a 
questionnaire that evaluated four critical areas of performance: 
purpose and design, strategic planning, management, results and 
accountability. These assessments were included in the President's 2004 
budget submission. 

[35] U.S. General Accounting Office, Environmental Protection: 
Collaborative EPA-State Effort Needed to Improve New Performance 
Partnership System, GAO/RCED-99-171 (Washington, D.C. June 21, 1999). 

[36] EPA Office of Inspector General, Surveys, Studies, Investigations, 
and Special Purpose Grants, Report No. 2002-P-00005 (Philadelphia, PA: 
Mar. 21, 2002).

[37] EPA Office of Inspector General, EPA Must Emphasize Importance of 
Pre-Award Reviews for Assistance Agreements, Report No. 2003-P-00007 
(Washington, D.C. Mar. 31, 2003).

[38] EPA Office of Inspector General, Report No. 2002-P-00018.

[39] EPA Office of Inspector General, Report No. 2003-P-00007.

[40] EPA Office of Inspector General, Report No. 2002-P-00018. 

[41] In certain circumstances exempted programs may conduct a 
competition; when they do, they must be conducted in a manner 
consistent with the competition policy. 

[42] For further details on the grants management plan see app. IV.

[43] EPA stated that its internal grants management reviews consist of 
(1) comprehensive management reviews of headquarters and regional 
offices conducted by the Office of Grants and Debarment; (2) grants 
performance measure reviews performed by the Office of Grants and 
Debarment, using performance measures and documentation in the central 
databases; and (3) grants management self-assessments conducted by EPA 
offices.

[44] The senior managers include the Director of the Office of Grants 
and Debarment, the Director of the Grants Administration Division, and 
the Grants Competition Advocate.

[45] U.S. General Accounting Office, Human Capital: A Guide for 
Assessing Strategic Training and Development Efforts in the Federal 
Government, GAO-03-893G (Washington, D.C. July 2003) and U.S. General 
Accounting Office, Results-Oriented Cultures: Creating a Clear Linkage 
Between Individual Performance and Organizational Success, GAO-03-488 
(Washington, D.C. Mar. 14, 2003). 

[46] The health center program provides grants to health centers to 
provide medical care to uninsured, underserved, and vulnerable 
populations in rural and urban areas.

[47] OMB's PART tool rates programs on a four-point scale from 
"effective" to "moderately effective" to "adequate" to "ineffective." A 
fifth rating possibility comes into play with "results not 
demonstrated" if adequate measures or data to gauge the program's 
performance were not available. The other grant program rated 
"effective" was the Medicare Integrity Program, which funds a variety 
of efforts to fight fraud and abuse in the Medicare program.

[48] GAO-03-893G. 

[49] GAO-03-488.

GAO's Mission:

The General Accounting Office, the investigative arm of Congress, 
exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony:

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics.

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading.

Order by Mail or Phone:

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to:

U.S. General Accounting Office

441 G Street NW,

Room LM Washington,

D.C. 20548:

To order by Phone: 	

	Voice: (202) 512-6000:

	TDD: (202) 512-2537:

	Fax: (202) 512-6061:

To Report Fraud, Waste, and Abuse in Federal Programs:

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470:

Public Affairs:

Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.

General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.

20548: