This is the accessible text file for GAO report number GAO-09-687 
entitled 'National Institutes of Health: Completion of Comprehensive 
Risk Management Program Essential to Effective Oversight' which was 
released on September 22, 2009. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Ranking Member, Committee on Finance, U.S. Senate: 

United States Government Accountability Office: 
GAO: 

September 2009: 

National Institutes of Health: 

Completion of Comprehensive Risk Management Program Essential to 
Effective Oversight: 

GAO-09-687: 

GAO Highlights: 

Highlights of GAO-09-687, a report to the Ranking Member, Committee on 
Finance, U.S. Senate. 

Why GAO Did This Study: 

The National Institutes of Health (NIH), an agency of the Department of 
Health and Human Services (HHS), is the primary federal agency for 
supporting medical research. The Office of the Director (OD) is the 
central NIH office responsible for setting policy and overseeing NIH’s 
27 institutes and centers (IC). Allegations involving one institute 
raised questions about areas of oversight by the OD. In light of these 
questions, GAO examined (1) how NIH makes extramural research funding 
decisions and OD monitoring of this process, (2) the design of selected 
internal controls over NIH’s travel and personnel appointment 
processes, and (3) the design of NIH’s new risk management program and 
the program it is replacing. To address these objectives, GAO reviewed 
relevant NIH policies, procedures, and supporting documentation. GAO 
also selected 3 institutes that varied in size for in-depth reviews. 

What GAO Found: 

NIH is required by law to make its extramural research funding 
decisions—funding provided to scientists external to NIH such as those 
at universities—using a dual peer review system. During the first 
level, initial peer review groups assess applications and assign a 
score to them based on their scientific merit. During the second level, 
advisory councils review the applications and their scores and, on the 
basis of this review, recommend to the ICs certain applications for 
funding consideration. IC directors can use their discretion and choose 
to fund applications based on factors in addition to scientific merit, “
skipping” over applications with higher scores or making “exceptions” 
to fund applications with lower scores. GAO found that in fiscal year 
2007, IC directors funded about 19 percent of NIH’s applications for a 
common type of grant based on factors in addition to scientific merit. 
However, the NIH OD does not monitor the extent to which IC directors 
use such discretion when making extramural funding decisions—an action 
that would be consistent with federal internal control standards. 

The NIH OD has established policies and procedures that incorporate key 
internal controls into the travel and personnel appointment processes. 
For example, the processes require multiple levels of review and 
approval. However, there is not an NIH-wide process for risk-based 
monitoring of the effectiveness of controls. Without monitoring actual 
implementation of controls based on assessed risk levels, NIH does not 
have adequate assurance that controls are operating as intended within 
those areas that have been identified as posing risks to the agency’s 
ability to achieve its mission. 

NIH’s Management Control Program, a risk management program updated in 
2004, did not comprehensively address risks to the agency’s overall 
operations and resulted in a lack of sufficient information for 
effective oversight and agencywide risk management. Recognizing this, 
in 2006, NIH began designing a new risk management program, the 
Enterprise Risk Management Program. Although an improvement over the 
earlier program, the design of the new program does not fully address 
the components identified in GAO’s framework for effective risk 
management. For example, the design does not incorporate strategic 
goals and objectives as a precondition for risk management, the 
evaluation of alternative responses to address identified risks, or 
documentation of the rationale for selecting a risk response. Further, 
NIH’s new program is not yet fully implemented, despite an over 3-year 
effort. According to NIH officials, NIH has experienced delays because 
of a change in contractors, balancing staff resources with competing 
demands, and underestimating time needed for implementation. 

What GAO Recommends: 

To help improve oversight, GAO made three recommendations to the 
Director of NIH: (1) monitor the extent to which IC directors use 
discretion in funding decisions, (2) add key components to the 
Enterprise Risk Management Program, and (3) ensure implementation of 
the program. HHS disagreed with the first recommendation, partially 
concurred with the second recommendation, and identified a final date 
for implementation of the program. 

View [hyperlink, http://www.gao.gov/products/GAO-09-687] or key 
components. For more information, contact Linda T. Kohn at (202) 512-
7114 or kohnl@gao.gov or Susan Ragland at (202) 512-8486 or 
raglands@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

NIH Is Required to Use a Peer Review System to Make Extramural Funding 
Decisions; NIH's OD Does Not Monitor Key Decisions in which IC 
Directors Exercise Their Discretion Over Funding: 

Design of NIH's Travel and Personnel Appointment Processes Includes Key 
Control Activities and Some Monitoring Activities but Lacks Systemic 
Risk-Based Monitoring: 

NIH's Management Control Program and Enterprise Risk Management Program 
Do Not Fully Address Key Components of Effective Risk Management: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: NIH Organization and Mission: 

Appendix III: Comments from the National Institutes of Health: 

Appendix IV: GAO Contacts and Staff Acknowledgments: 

Tables: 

Table 1: Extramural Research R01 Grant Applications Funded in Fiscal 
Years 2003 through 2007: 

Table 2: GAO's Risk Management Framework: 

Table 3: Overview of ICs Including Establishment Date, Mission, and 
Fiscal Year 2008 Appropriation: 

Figures: 

Figure 1: Relationship of the Risk Management Framework Components: 

Figure 2: National Cancer Institute's Fiscal Year 2007 Payline for R01 
Grant Applications: 

Figure 3: Organizational Structure of NIH: 

Abbreviations: 

CSR: Center for Scientific Review: 

COSO: Committee of Sponsoring Organizations: 

GPRA: Government Performance and Results Act: 

HHS: Department of Health and Human Services: 

IC: institutes and centers: 

NCI: National Cancer Institute: 

NIAAA: National Institute on Alcohol Abuse and Alcoholism: 

NIDDK: National Institute of Diabetes and Digestive and Kidney 
Diseases: 

NIEHS: National Institute of Environmental Health Sciences: 

NIH: National Institutes of Health: 

OD: Office of the Director: 

OMB: Office of Management and Budget: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

September 11, 2009: 

The Honorable Charles E. Grassley: 
Ranking Member: 
Committee on Finance: 
United States Senate: 

Dear Senator Grassley: 

The National Institutes of Health (NIH) is the primary federal agency 
for supporting medical research in the United States. In fiscal year 
2008, NIH provided $24.4 billion--83 percent of its $29.5 billion 
budget--in extramural research funding, which supports scientists and 
research personnel working at universities, medical schools, and other 
research institutions.[Footnote 1] NIH's extramural research funding 
efforts reflect its large, decentralized organization. NIH comprises 27 
institutes and centers (IC) and an Office of the Director (OD). Each of 
the ICs has its own budget, mission, and staff and focuses on 
particular diseases or research areas, such as cancer or aging issues. 
Twenty-four of the 27 ICs fund extramural research, each with a 
separate appropriation,[Footnote 2] and these ICs make final decisions 
on which extramural research projects to fund following a standard 
process defined by law and NIH policy. As the central office at NIH, 
the OD establishes NIH policy and is responsible for overseeing the 
ICs, including their research funding efforts and their various 
administrative functions, such as hiring personnel and approving 
personnel travel. The OD's oversight responsibilities have grown over 
the years. Between 1985 and 2000, 7 of the 27 ICs were created--and 
these additions have helped to increase the overall complexity of 
overseeing the ICs. More recently, under the American Recovery and 
Reinvestment Act of 2009, NIH received $10.4 billion that NIH plans to 
use in 2009 and 2010 to fund extramural and other research and support 
the construction, renovation, and repair of certain research 
facilities. 

We and others have raised questions about the OD's ability to 
effectively oversee IC activities. For example, in April 2007, we 
reported that NIH had not established clear policies related to 
managing conflicts of interest among senior NIH employees who have 
decision-making responsibilities for NIH's research efforts, [Footnote 
3] which include NIH's extramural research funding. We noted that such 
policies are part of NIH's framework for ensuring the integrity of NIH- 
funded research and recommended that NIH clarify them. NIH agreed with 
our recommendation. In mid-2007 you raised questions over allegations 
of improper travel, personnel appointments, and extramural research 
funding decisions involving the director of one of NIH's ICs, the 
National Institute of Environmental Health Sciences (NIEHS), which 
supports research on environmental influences on the development and 
progression of human disease. Similar questions prompted the House 
Committee on Appropriations to request that NIH conduct a management 
review of NIEHS, which found management and operational problems at the 
Institute.[Footnote 4] 

The above issues focus on how NIH makes extramural funding decisions 
and the quality of its internal control over administrative functions 
such as travel arrangements and personnel appointments. Internal 
control can include the establishment of safeguards, such as 
supervisory reviews, that are incorporated into agency work processes. 
According to federal standards, effectively designed and implemented 
internal control provides reasonable assurance that an agency's 
operations are effective and efficient, its financial reporting 
reliable, and that the agency complies with applicable laws and 
regulations.[Footnote 5] The issues at NIH also raise broader questions 
about NIH's risk management, the process whereby an agency or 
organization systematically identifies risks associated with achieving 
its mission or objectives; assesses the magnitude of those risks; puts 
in place, when necessary, mitigating actions to address those risks; 
and then monitors the effectiveness of those actions. During our 
review, NIH was in the process of implementing its Enterprise Risk 
Management Program, a new risk management program that is replacing the 
NIH Management Control Program--the agency's previous risk management 
program. 

You asked us to examine NIH's oversight of the ICs. Specifically, we 
agreed to provide information on NIH's extramural research funding 
decisions, employee travel arrangements and hiring practices for 
certain employees, and NIH's process for identifying and addressing 
potential risks to its operations. In this report we: 

1. describe how NIH makes extramural research funding decisions and the 
extent to which the NIH's OD monitors this process, 

2. review the design of selected internal controls over NIH's travel 
and personnel appointment processes, and: 

3. review the design of the NIH Management Control Program and the 
Enterprise Risk Management Program to determine if they contain key 
components of an effective risk management program. 

To address these objectives, we reviewed relevant NIH policies, 
procedures, and supporting documentation on (1) the process used across 
NIH for making extramural research funding decisions and efforts by the 
OD to monitor this process, (2) the design of key internal controls for 
employee travel and Title 42 personnel appointments[Footnote 6]-- 
specifically, control and monitoring activities--and (3) the design of 
the NIH Management Control Program and the Enterprise Risk Management 
Program. We also selected 3 ICs--the National Cancer Institute (NCI), 
National Institute of Diabetes and Digestive and Kidney Diseases 
(NIDDK), and National Institute on Alcohol Abuse and Alcoholism 
(NIAAA)--for more in-depth reviews of the process used across NIH for 
making extramural research funding decisions and for more in-depth 
reviews of the design of the ICs' control and monitoring activities for 
travel and Title 42 personnel appointment processes. We selected these 
3 ICs because they vary in size and focus on different disease-specific 
research missions. We interviewed officials from the NIH OD and the 
selected ICs to clarify our understanding of the process used for 
making extramural research funding decisions and the OD's monitoring of 
this process. We also collected data on funding decisions for each of 
the 24 ICs that fund extramural research.[Footnote 7] We performed 
walkthroughs[Footnote 8] at the 3 selected ICs and interviewed 
officials from the NIH OD and the selected ICs to clarify our 
understanding of the design of the ICs' control and monitoring 
activities for travel and Title 42 personnel appointment processes. We 
also interviewed officials from the NIH OD to further our understanding 
of the NIH Management Control Program and the Enterprise Risk 
Management Program. 

As part of our review, we compared the OD's monitoring of the process 
used for making extramural research funding decisions and the design of 
the control and monitoring activities at the three selected ICs to 
GAO's Standards for Internal Control in the Federal Government. 
[Footnote 9] In reviewing the design of the NIH Management Control 
Program and the Enterprise Risk Management Program, we compared these 
designs to our framework for effective risk management.[Footnote 10] 
The scope of our audit did not include testing the implementation of 
internal control over travel and Title 42 personnel appointments. 
Furthermore, we did not review the implementation of either the NIH 
Management Control Program or the Enterprise Risk Management Program 
because, at the time of our review, NIH did not plan to continue the 
Management Control program and the Enterprise Risk Management Program 
was not yet fully implemented. 

Appendix I includes additional details on our scope and methodology. We 
conducted this performance audit from March 2008 to September 2009, in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

Background: 

As the primary federal agency for supporting medical research in the 
United States, NIH's mission is "science in pursuit of fundamental 
knowledge about the nature and behavior of living systems and the 
application of that knowledge to extend healthy life and reduce the 
burdens of illness and disability." NIH is headed by a Director who is 
supported by staff and program offices within the OD and 27 ICs. Each 
of the ICs has its own director and staff. Each IC director reports to 
the OD.[Footnote 11] Appendix II provides more information about NIH's 
organizational structure. 

NIH's ICs were created over time, with each having an explicit mission 
focused on a particular disease, organ system, stage of development, or 
cross-cutting mission, such as providing scientists and researchers 
with the tools they need to understand, detect, treat, and prevent a 
wide range of diseases. The first institute, NCI, was created in 1937, 
and the newest institute, National Institute of Biomedical Imaging and 
Bioengineering, was created in 2000. 

Internal Control: 

Internal control is an integral part of managing an agency.[Footnote 
12] It comprises the plans, methods, and procedures used to meet 
missions, goals, and objectives. Effectively designed and implemented 
internal control provides management with reasonable assurance that the 
following objectives are being achieved: (1) effectiveness and 
efficiency of operations, (2) reliability of financial reporting, and 
(3) compliance with applicable laws and regulations.[Footnote 13] 
Internal control serves as the first line of defense in preventing and 
detecting errors and fraud. The following five elements of internal 
control provide the basis against which internal control is evaluated. 

* Control Environment--Sets the tone for an organization and is the 
foundation for all other standards. Management and employees should 
establish and maintain an environment throughout the organization that 
sets a positive and supportive attitude toward internal control and 
conscientious management. Among others, control environment includes 
management's integrity and ethical values, commitment to competence, 
philosophy and operating style, and organizational structure. 

* Risk Assessment--The identification and analysis of relevant risks 
associated with achieving the objectives and forming a basis for 
determining how risks should be managed. This standard includes an 
assessment of the risks the agency faces from both external and 
internal sources. 

* Control Activities--The policies, procedures, techniques, and 
mechanisms that enforce management's directives. Control activities 
occur at all levels and functions of the agency and include a wide 
range of diverse activities such as approvals, authorizations, 
verifications, and reconciliations. 

* Information and Communication--Information should be recorded and 
communicated to management and others within the entity who need it and 
in a form and within a time frame that enables them to carry out their 
internal control and other responsibilities. In addition to internal 
communications, management should ensure there are adequate means of 
communicating with, and obtaining information from, external 
stakeholders that may have a significant impact on the agency achieving 
its goals. 

* Monitoring--Includes ongoing monitoring in the course of normal 
operations (e.g., regular management and supervisory activities, 
comparisons, and reconciliations) and risk-based monitoring that 
includes separate evaluations of controls' effectiveness whose scope 
and frequency depends primarily on the assessment of risks and 
effectiveness of ongoing monitoring procedures. 

Risk Management: 

One way to help ensure that internal control is continuously monitored 
and improved is through risk management. Risk management helps agencies 
to identify the most significant areas in which to place or enhance 
controls.[Footnote 14] Additionally, based on the assessment of risk 
that is performed as part of an overall risk management program, 
agencies can determine the scope and frequency of control evaluations. 
Risk management is a continuous process whereby an organization 
systematically identifies risks associated with achieving its 
objectives; assesses the magnitude of those risks; puts in place, when 
necessary, mitigating actions to address those risks; and then monitors 
the effectiveness of those actions taken. In addition, because 
governmental, economic, industry, regulatory, and operating conditions 
continually change, risk management provides a mechanism to identify 
and deal with any special risks prompted by such changes. While risk 
management programs do not provide absolute assurance regarding the 
achievement of an agency's objectives, an effective risk management 
program can be particularly useful in a decentralized organization to 
help top management identify potential problems and allocate limited 
resources using a reasonable basis (such as risk). 

In 2005, GAO identified risk management as an area of increasing 
concern, particularly with regard to the need for the completion of 
threat and risk assessments in a variety of areas.[Footnote 15] To help 
address the concern, GAO developed a framework for effective risk 
management activities in the federal government based on best practices 
and authoritative literature.[Footnote 16] This framework includes five 
components that define a risk management program for federal agencies: 
strategic goals, objectives, and constraints; risk assessment; 
alternatives evaluation; management selection; and implementation and 
monitoring. For the purposes of our analysis of NIH's program, we also 
considered two additional components, internal environment and 
information and communications, based on guidance and standards on risk 
management and internal controls.[Footnote 17] Figure 1 illustrates the 
interrelationship of these seven components. The components of the 
framework should operate within an internal environment that supports 
the other components, and pertinent information should be communicated 
between and among internal and external stakeholders as well as 
personnel responsible for carrying out the duties associated with each 
of the components. 

Figure 1: Relationship of the Risk Management Framework Components: 

[Refer to PDF for image: illustration] 

The illustration depicts a circle with Information and Communication at 
the core, surrounded by the following components, each of which is 
interrelated: 

Internal Environment: 
* Strategic goals,objectives,and constraints; 
* Risk assessment; 
* Alternatives evaluation; 
* Management selection; 
* Implementation and monitoring. 

Source: GAO. 

[End of figure] 

NIH Is Required to Use a Peer Review System to Make Extramural Funding 
Decisions; NIH's OD Does Not Monitor Key Decisions in which IC 
Directors Exercise Their Discretion Over Funding: 

NIH is required by law to use a peer review system in its process for 
making extramural research funding decisions. NIH's dual peer review 
system is designed to help ensure the objective evaluation of the 
scientific merit of applications for extramural funding. After NIH's 
peer review process is concluded, IC directors have discretion when 
making final extramural funding decisions and are not required to fund 
applications based strictly on the scores resulting from the evaluation 
of their scientific merit. We found that in fiscal year 2007 IC 
directors decided to fund about 19 percent of NIH's applications for 
RO1 grants, a common type of grant, based on factors in addition to 
these scores. However, NIH's OD does not monitor extramural funding 
decisions in which the IC Directors exercise their discretion. 

By Law, NIH Must Use a Dual Peer Review System Designed to Evaluate 
Scientific Merit of Extramural Funding Applications: 

NIH is required by law to use a peer review system in its process for 
making extramural research funding decisions. This system comprises two 
sequential levels of peer review by panels of experts in various fields 
of research who help NIH identify the most promising extramural grant 
applications to fund, as defined primarily by an assessment of the 
applications' technical and scientific merit.[Footnote 18] According to 
NIH, compared to a single level of peer review, the dual peer review 
system allows for multiple reviews and therefore a more objective 
evaluation of the scientific merit of grant applications. 

Applications for NIH's extramural funding are received by NIH's Center 
for Scientific Review (CSR), which is responsible for assigning each 
application to the first level of peer review. The first level of peer 
review is conducted by what are known as initial peer review groups, to 
which CSR assigns applications for review, based on the applications' 
proposed area of research and the initial peer review groups' area of 
expertise. These initial peer review groups specialize in various 
research areas such as cancer or digestive disorders and are composed 
of scientists, who are often recognized as experts in their field. 
[Footnote 19] Each group meets three times per fiscal year to review 
grant applications. 

The initial peer review groups are responsible for identifying the most 
promising applications for funding, based on an assessment of the 
applications' scientific merit.[Footnote 20] The groups review the 
applications assigned to them and assess their scientific merit, using 
criteria that require reviewers to examine such components as a grant 
application's design and methodology, innovation, and scientific 
significance.[Footnote 21] Using these criteria, the initial peer 
review groups assign a priority score to the applications they 
reviewed, which are used to rank the applications from among those in 
the cohort of applications. After the applications are scored and 
ranked, the information is forwarded to the appropriate IC--based on 
the applications' proposed area of research--for the second level of 
peer review. 

Each IC that funds extramural research has its own advisory council, 
which conducts the second level of peer review.[Footnote 22] Advisory 
councils consist of no more than 18 voting members, two-thirds of whom 
are scientists in the research areas of the IC and one-third of whom 
are leaders of non-science fields.[Footnote 23] Advisory councils meet 
at least three times per fiscal year.[Footnote 24] Under law and NIH 
policy, the advisory councils are responsible for reviewing the 
applications and their priority scores and, on the basis of this 
review, recommending to the ICs certain applications for funding 
consideration. The advisory council may ask for applications to be 
scored a second time, such as if they have questions about whether the 
scientific criteria were applied appropriately. NIH advisory council 
members we interviewed noted that councils only have time to discuss a 
few applications individually, so they consider many applications in 
large groups, particularly in cases when no concerns are apparent about 
the applications or their priority scores. Based on data we reviewed, 
we found that from fiscal year 2003 to 2007, in most cases, only a 
small number of applications were not recommended by advisory councils 
for funding consideration. The advisory councils' recommendations 
conclude NIH's peer review process. 

IC Directors Have Discretion to Make Final Extramural Funding 
Decisions, but NIH's OD Does Not Monitor Decisions in Which IC 
Directors Exercise This Discretion: 

After NIH's peer review process has been concluded, the director of 
each IC is responsible for making final extramural funding decisions. 
In deciding which applications to fund, the IC directors choose 
applications from among those recommended for funding consideration by 
the advisory council.[Footnote 25] In general, IC directors select 
applications for funding based on their priority scores, which reflect 
the evaluation of the applications' scientific merit by NIH's peer 
review process. For each fiscal year, each IC establishes a funding 
line--known as the payline--which roughly corresponds with the number 
of extramural grant applications the IC will be able to fund that year. 
The payline for any given year is based on projections of the total 
funding available at the IC that year for grants, the average dollar 
amount expected to be awarded per application, and the number of 
applications received by the IC. For example, as shown in figure 2, 
based on the amount of funding available for extramural grants, NCI set 
its payline for R01 grants[Footnote 26]--the most common grant 
category--at the 15th percentile for fiscal year 2007. This means that 
NCI expected to have sufficient funding at a minimum for all of the 
applications with scores in the top 15 percent. In general, IC 
directors fund only those applications with priority scores above the 
fiscal year's payline. 

Figure 2: National Cancer Institute's Fiscal Year 2007 Payline for R01 
Grant Applications: 

[Refer to PDF for image: illustration] 

National Cancer Institute’s payline: 15th percentile: 

Application order based on percentile ranking: 
656 applications scored above the 15th percentile; 
3,108 applications scored below the 15th percentile. 

Source: GAO. 

Note: Figure includes only applications that were recommended for 
funding consideration by an initial peer review group and the advisory 
council. The portion of applications that scored above the payline 
appears to be greater than 15 percent because applications that were 
considered for funding multiple times during the year and scored below 
the 15th percentile each time are only counted in this data once. 

[End of figure] 

While the IC directors only fund projects recommended by their advisory 
councils and typically work within the paylines, ultimately they have 
discretion to make final extramural funding decisions. In particular, 
directors are not required to fund applications based strictly on the 
applications' priority scores or the payline. In some instances, IC 
directors decide not to fund applications that scored above the fiscal 
year's payline, such as when the applications duplicate research that 
has already received IC funding. These applications are called "skips." 
For example, of the 656 applications that scored above NCI's payline in 
fiscal year 2007, 3 applications were skipped. Similarly, though the IC 
directors typically do not decide to fund applications with priority 
scores that fall below the fiscal year's payline, in some cases they 
do. These applications are known as "exceptions." For example, of the 
3,108 applications that scored below NCI's payline in 2007, 137 were 
funded as exceptions. In the case of exceptions, the IC directors may 
exercise their discretion and choose to fund these applications based 
on factors in addition to the applications' priority scores. These 
factors can include efforts to support the IC or NIH's research 
priorities. When skipping applications or funding applications as 
exceptions, IC directors are required under NIH policy to document the 
corresponding rationale used. 

In reviewing the IC data, we found that 18.5 percent of NIH's funded 
R01 grant applications were funded as exceptions in fiscal year 2007, 
as shown in table 1. These applications had scientific merit scores 
that were below the payline for their respective ICs and thus were 
funded based on factors in addition to their scientific merit scores. 
This represents a substantial increase from 9.7 percent of funded 
applications that were exceptions in fiscal year 2003. 

Table 1: Extramural Research R01 Grant Applications Funded in Fiscal 
Years 2003 through 2007: 

Fiscal year: Total number of applications funded; 
2003: 6,461; 
2004: 6,167; 
2005: 5,731; 
2006: 5,408; 
2007: 5,715. 

Fiscal year: Number of applications funded above the payline; 
2003: 5,836; 
2004: 5,639; 
2005: 5,159; 
2006: 4,788; 
2007: 4,656. 

Fiscal year: Number of applications funded below the payline - 
exceptions; 
2003: 625; 
2004: 528; 
2005: 572; 
2006: 620; 
2007: 1,059. 

Fiscal year: Percentage of applications funded below the payline - 
exceptions; 
2003: 9.7%; 
2004: 8.6%; 
2005: 10.0%; 
2006: 11.5%; 
2007: 18.5%. 

Source: GAO analysis of NIH grants data. 

[End of table] 

Documentation that we reviewed from three of the ICs--NCI, NIDDK, and 
NIAAA--showed that IC directors funded applications as exceptions for 
various reasons. For example, IC officials cited the NIH-wide 
initiative to fund new investigators as one of the most frequent 
reasons for making the exceptions. In addition, IC officials told us 
that they funded applications as exceptions in order to maintain a 
diverse portfolio of research topics. 

NIH's OD collects information on some aspects of the extramural 
research process. For example, the NIH OD collects information on the 
number of extramural grants funded by each IC; the percentage of 
applications that receive funding; and the priority rankings, by 
percentile, of funded applications. The NIH OD also targets some of 
these collection efforts towards specific types of extramural grants. 
For example, as part of its effort to support new investigators, the 
NIH OD has been collecting data on the number of extramural grants 
awarded to new investigators. 

Although NIH's OD collects some information on the extramural research 
process, it does not monitor key funding decisions made by IC 
directors--specifically, the instances in which IC directors exercise 
their discretion to make skips or exceptions to the funding payline. 
Skips and exceptions represent an area of potential risk because IC 
directors have latitude in making these decisions; monitoring these 
decisions would be consistent with federal internal control standards. 
[Footnote 27] Although ICs are required to document the rationales used 
when skipping applications or funding applications as exceptions, the 
ICs are not required to provide the NIH OD with this documentation, and 
the NIH OD does not collect it. As a result, NIH's OD does not have 
information on the number of applications skipped or funded as 
exceptions and the reasons for these decisions. 

Design of NIH's Travel and Personnel Appointment Processes Includes Key 
Control Activities and Some Monitoring Activities but Lacks Systemic 
Risk-Based Monitoring: 

With regard to the design of NIH-wide travel processes, NIH OD has 
established policies and procedures to help ensure that federal travel 
regulations are followed with regard to issues such as premium class 
travel, per diem expenses, and travel paid by third parties. The key 
control and monitoring activities for travel include reviews and 
approvals which take place during two stages--authorization and 
voucher--of the process. During the authorization stage, the traveler 
receives approval to travel based on the supervisor's approval of the 
mission-relatedness of the trip and an administrative official's 
approval of the method of transportation used, the cost estimates set 
forth for travel expenses, and the availability of funds for 
reimbursement to the traveler. During the voucher stage, the traveler's 
voucher for reimbursement of travel expenses is approved based on an 
administrative official's review of the voucher package which includes 
the traveler's certification of the voucher and required receipts for 
travel expenses. 

The NIH OD has also established policies and procedures to help ensure 
that Title 42 personnel appointment decisions are appropriate. The 
design of NIH-wide key control activities for Title 42 personnel 
appointments includes reviews and approvals which take place during 
three stages--resource determination, appointment selection, and 
compensation--of the process. During the resource determination stage, 
the IC's selecting official identifies a hiring need and the 
administrative official determines whether necessary resources are 
available to meet the hiring need. During the appointment selection 
stage, the IC completes the recruitment including receiving 
applications, conducting candidate interviews, and making a tentative 
selection. For some positions, the NIH Offices of Human Resources, 
Intramural Research, and Extramural Research also play a role in 
preparing the recruitment and approving the selected candidate. 
[Footnote 28] During the compensation stage, final approval for the 
appointment and compensation is given depending upon the position and 
salary level for the candidate. Specifically, if the proposed 
compensation is below the lowest third of a given position's salary 
range, then the IC director makes the final approval; if the proposed 
compensation is above the lowest third of a given position's salary 
range but still within the range, then the IC director makes the final 
approval based on a recommendation from an IC committee; and if the 
proposed compensation is above the salary range or other specified 
limits, then the NIH director makes the final approval based on 
recommendations from an IC committee, the IC director, and an NIH-wide 
committee. 

Overall, we found that the design of the controls included in the NIH- 
wide processes over travel and Title 42 personnel appointments included 
key controls necessary to help ensure these activities were being 
carried out appropriately, except in one key area related to the lack 
of requirements for risk-based monitoring. While controls may appear 
adequate based on written policies and procedures, without monitoring 
actual implementation based on the assessed risk levels, NIH does not 
have adequate assurance that controls are operating as intended within 
those areas that pose risk. NIH policy did not require the ICs to 
perform monitoring that includes risk-based control evaluations. 
Further, although NIH policy required a flexible plan for NIH-wide 
control evaluations that would generally target high-and medium-risk 
areas for review, according to NIH OD officials, such reviews have not 
been performed for over 3 years because they do not have staff to 
perform these reviews. 

At NIH OD and two of the three ICs we reviewed--NCI and NIAAA--we found 
that some monitoring activities were performed over travel and Title 42 
personnel appointments. However, these monitoring activities were 
either not part of systemic risk assessment efforts that lead to 
subsequent monitoring based on assessed risk or not performed on a 
consistent and ongoing basis. Specifically, 

* Because of travel issues previously identified by GAO,[Footnote 29] 
HHS requires each of its operating divisions, which includes NIH, to 
perform quarterly control evaluations of travel cards.[Footnote 30] As 
a result of this requirement, each quarter the NIH OD selected a sample 
of travel transactions from across the ICs and tested compliance with 
federal travel regulations and NIH policies and procedures. For 
example, during each of the first 2 quarters of fiscal year 2008, the 
NIH OD found problems with about 20 percent of the 100 sample items it 
tested. During the third quarter, the NIH OD found problems with about 
30 percent of the 75 sample items it tested. Some of the problems found 
during these quarters included over-or underpayment to travelers, 
failure of travelers to take advantage of lodging tax exemptions, and 
misuse of travel cards. The NIH OD required follow-up actions such as 
reimbursement of overpayment amounts and issuing additional guidance. 
However, these travel control evaluations were not part of a systemic 
process for assessing risk over operations and subsequently monitoring 
or evaluating controls based on assessed risk levels. 

* In 2008, NCI and NIAAA performed control evaluations over travel, and 
NCI performed a control evaluation of personnel appointments (including 
those under Title 42). These control evaluations were performed in 
response to prior audit findings, to prepare for upcoming audits or 
reviews, or to address concerns regarding process inefficiencies. 
However, they were not incorporated into the design of the processes 
and therefore were not performed on a consistent and ongoing basis. 

One of the three ICs we reviewed--NIDDK--had adopted its own risk-based 
program which consisted of assessing the risks over operational areas, 
including travel and personnel appointments, and subsequently 
monitoring the controls over those operational areas. The frequency of 
monitoring depended upon the risk level, and high-risk activities at 
NIDDK were scheduled to be monitored more frequently than low-or medium-
risk activities. The design of NIDDK's program represents a positive 
step towards an effective risk management program. Further details on a 
framework for an effective risk management program are discussed in the 
next section. 

NIH's Management Control Program and Enterprise Risk Management Program 
Do Not Fully Address Key Components of Effective Risk Management: 

The design of the Management Control Program provided NIH with a 
limited ability to identify and address risks to the agency's overall 
operations. Recognizing the need for improvement, in 2006, the NIH OD 
began redesigning its program. However, while an improvement over the 
Management Control Program, the new Enterprise Risk Management Program 
does not fully address all of the components of GAO's framework for 
effective risk management. Further, NIH's Enterprise Risk Management 
Program has not been fully implemented, despite an over 3-year effort, 
and NIH had not yet established milestones for its full implementation. 

NIH's Management Control Program Had Weaknesses: 

NIH's Management Control Program was initially implemented in 1999 and 
updated in 2004. Under the design of this program, risk assessments are 
performed that relate to specific management control areas, such as 
functional areas, systems, or processes (e.g., intramural research 
programs) without relating those areas to potential systemic or 
agencywide risks. If weaknesses are identified within the particular 
area being reviewed, the Management Control Program appropriately 
requires that corrective action plans be developed and implemented to 
correct the weakness and that such actions be monitored after 
implementation to ensure that the weakness has been corrected. 

As designed, NIH's Management Control Program did not address several 
of the components and related key elements included in GAO's framework 
for an effective risk management program. An effective risk management 
program should enable management to proactively identify, assess, and 
mitigate risks. Table 2 outlines the seven components of the risk 
management framework and the key elements within each of these 
components. 

Table 2: GAO's Risk Management Framework: 

Risk management component: Strategic goals, objectives, and 
constraints; 
Description: Addresses what the strategic goals and objectives are 
attempting to achieve and the steps needed to attain these results, and 
considers the constraints under which an agency operates such as 
statute, higher level policy, budget, or other factors beyond 
management's control that may affect an agency's risk management plans; 
Key elements: An agency's risk management program should: 
* Require mission-based strategic goals and objectives, that are 
clearly articulated and measurable, to be set as a pre-condition for 
effective risk management. Without clearly identified strategic goals 
and objectives, an agency cannot effectively identify and address 
potential risks to its mission, prioritize risk, or identify criteria 
against which to measure performance; 
* Require agencies to identify constraints (e.g., legislative 
requirements or resources) that may limit effective risk management. 

Risk management component: Risk assessment; 
Description: Addresses the identification and evaluation of potential 
risks to an agency's ability to achieve its goals and objectives so 
that management can design and implement responses to prevent or 
mitigate identified risks; 
Key elements: An agency's risk management program should: 
* Identify potential events which may adversely affect the agency, 
called risks, and evaluate the events based on likelihood of occurrence 
and impact. For example, an agency may identify and evaluate potential 
risks associated with economic and legislative changes, natural 
disasters, and criminal or terrorist activities; 
* Require continuous identification and evaluation of potential risks 
since governmental, economic, industry, legislative, and operating 
conditions continually change. 

Risk management component: Alternatives evaluation; 
Description: Addresses the identification and evaluation of alternative 
ways in which the agency can act to alter either the likelihood of 
occurrence or the impact of a potential risk; 
Key elements: An agency's risk management program should: 
* Identify alternative ways the agency can respond to prevent or 
mitigate an identified risk. For example, to comply with new 
legislation, an agency may need to revise existing policy and 
procedures or develop new policies and procedures; 
* Evaluate the alternatives identified to consider the effect on 
likelihood of occurrence and impact of a potential risk; 
* Evaluate the alternatives identified to consider the costs and 
benefits. 

Risk management component: Management selection; 
Description: Addresses the selection of a response to mitigate an 
identified risk based on the alternatives evaluated and management 
priorities, such as management's attitude towards risk and how limited 
resources will be targeted; 
Key elements: An agency's risk management program should: 
* Require management to select and document an alternative, such as 
revising or creating a policy or procedure, for addressing an 
identified risk; 
* Require management to document the rationale for selecting the 
alternative. 

Risk management component: Implementation and monitoring; 
Description: Addresses how risk responses will be applied and assessed 
to improve efficiency and effectiveness. In addition, addresses how the 
risk management program will be assessed to determine whether changes 
are needed to improve efficiency and effectiveness; 
Key elements: An agency's risk management program should: 
* Implement management's selected alternative to address risk; 
* Periodically assess management's selected alternative to address 
risk; 
* Periodically assess the efficiency and effectiveness of the entire 
risk management program. 

Risk management component: Internal environment; 
Description: Addresses how management will establish and maintain a 
positive environment that sets the tone throughout the agency and is 
the foundation upon which all other components of risk management 
operate; 
Key elements: An agency's risk management program should: 
* Include an agency's risk management philosophy to help position the 
agency so that it can effectively recognize and manage risk; 
* Require oversight by a high-level senior body within the agency; 
* Incorporate the importance of integrity and ethical values to 
increase the effectiveness of the risk management program since the 
program and its results depend upon the personnel who carry out risk 
activities; 
* Include the way management assigns authority and responsibility to 
help ensure that risk responsibilities are carried out; 
* Hold managers accountable for their assigned duties in the risk 
management program; 
* Require management to organize its risk structure to provide a 
framework for the agency to plan, execute, control, and monitor risk 
activities; 
* Require management to initially train its personnel to help ensure 
that they have the necessary knowledge and skills to perform their 
assigned tasks; 
* Ensure management maintains competence of the agency's personnel by 
providing for continuous training to update personnel on risk 
management practices and techniques. 

Risk management component: Information and communication; 
Description: Addresses the need to identify and communicate pertinent 
information in a form and timeframe that allows personnel to carry out 
their risk management responsibilities; 
Key elements: An agency's risk management program should: 
* Require pertinent information to be collected from and disseminated 
to relevant internal stakeholders in a form and timeframe consistent 
with the agency's risk management needs; 
* Require pertinent information to be collected from and disseminated 
to relevant external stakeholders in a form and timeframe consistent 
with the agency's risk management needs. 

Sources: [hyperlink, http://www.gao.gov/products/GAO-06-91]; The 
Committee of Sponsoring Organizations of the Treadway Commission, 
Enterprise Risk Management--Integrated Framework (Jersey City, N.J.: 
American Institute of Certified Public Accountants, September 2004); 
and [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. 

[End of table] 

The three components of the framework that the Management Control 
Program did not address are strategic goals, objectives, and 
constraints; risk assessment; and information and communication. 
Specifically, the program did not do the following. 

* Link the identification of potential risks with the agency's 
strategic goals and objectives. The design of the Management Control 
Program did not require strategic goals and objectives to be set as a 
precondition for risk management. Without clearly identified strategic 
goals and objectives, an agency is limited in its ability to 
effectively identify and address potential risks to its mission, 
prioritize risk, or identify criteria against which to measure 
performance. 

* Require risk assessments be performed to identify and evaluate 
potential risks that could adversely affect NIH's ability to achieve 
its objectives. The design of the Management Control Program called for 
evaluating the risks within specific functional areas, systems, or 
processes rather than assessing the risks that could adversely affect 
the agency as a whole. 

* Require pertinent information to be collected from and disseminated 
to relevant internal and external stakeholders in a form and time frame 
consistent with the agency's overall risk management needs. The design 
of the Management Control Program allowed for inconsistent and 
incomparable information from the ICs, which can prevent management 
from effectively using the information to help ensure that agency 
objectives are met. 

For a number of years, NIH OD officials recognized that weaknesses 
existed in the agency's Management Control Program, which resulted in a 
lack of sufficient information for effective oversight and agencywide 
risk management. For example, according to NIH OD officials, the 
program (1) did not hold managers responsible for their assigned duties 
in the risk management program, (2) did not require the ICs to 
communicate information in a form that allows NIH to effectively 
identify and manage risk across the agency, and (3) was not overseen by 
a high-level senior body, such as the Steering Committee.[Footnote 31] 
The three weaknesses NIH officials identified in the agency's 
Management Control Program correspond to the following key components 
of our framework for effective risk management: 1) internal 
environment, 2) information and communication, and 3) internal 
environment, respectively. As a result of acknowledged shortcomings, in 
2006 NIH began redesigning its risk management efforts. According to 
NIH OD officials, the new risk management program will improve the 
ability of the NIH OD to proactively identify and mitigate risks before 
they become obstacles to the NIH mission. However, as discussed later, 
NIH has not fully implemented the new Enterprise Risk Management 
Program and has encountered several obstacles in implementing initial 
phases of the program. 

NIH's Enterprise Risk Management Program, while Improved, Does Not 
Fully Address Several Key Components of Effective Risk Management: 

NIH began developing a new risk management program in 2006. The new 
program is designed to consist of a formal six-step methodology for 
managing risks.[Footnote 32] The six steps include: 

* Organize - Identify and train those charged with carrying out risk 
management activities, and define the risk management structure. 
[Footnote 33] 

* Identify and Score[Footnote 34] - Identify and score risks, review 
risks for quality and accuracy, and develop the risk baseline. 

* Assess - Document, analyze, and test processes and controls. 

* Remediate - Develop, review, approve, and execute corrective action 
plans. 

* Monitor - Monitor the risk baseline. 

* Report - Report risk information and results. 

The design of the Enterprise Risk Management Program represents an 
improvement over the 2004 NIH Management Control Program in several key 
areas. Specifically, the new program will allow for improved 
identification, assessment, and mitigation of risks agencywide because 
it includes the following: 

* Risk assessments: The new program requires the identification of 
potential events that could adversely affect the agency and the 
evaluation of those events based on likelihood of occurrence and 
impact. 

* Oversight by a high-level senior body: The design requires the 
Steering Committee to oversee the new risk management program. 

* Information and communication: The design requires that pertinent 
information be collected from and disseminated to relevant internal 
stakeholders in a form and time frame consistent with NIH's risk 
management needs. For example, the program requires a consistent 
methodology for identifying, assessing, and communicating risks across 
NIH, which will allow for consistent, comparable information from each 
of the ICs. 

However, the Enterprise Risk Management Program still does not fully 
address all of the components that we have identified for an effective 
risk management framework. As discussed below, further consideration of 
the risk management framework could significantly improve the design of 
NIH's new risk management program, which, if effectively implemented, 
could assist management in maintaining effective control over the 
agency's decentralized and diverse activities. 

Strategic Goals, Objectives, and Constraints. The Enterprise Risk 
Management Program does not require the NIH OD or ICs to set mission- 
based strategic goals and objectives as a precondition for risk 
management. This is a critical shortcoming because although the risk 
design requires risks to be assessed on the basis of their impact on 
NIH's mission, there is not an NIH-wide strategic plan against which to 
assess risks. Further, while some ICs and NIH OD offices have strategic 
plans for their organizations, the risk management program as designed 
does not call for risks to be assessed on the basis of their impact on 
IC-or NIH OD office-level missions. 

Alternatives Evaluation. Although the Enterprise Risk Management 
Program identifies four different responses the agency can select to 
prevent or mitigate identified risks (creating a new policy, procedure, 
or control; revising an existing policy, procedure, or control; 
streamlining or automating an existing policy, procedure, or control; 
or redesigning the process), the program does not require management to 
evaluate the risk responses identified to consider (1) the effect on 
the likelihood of occurrence and impact of a potential risk and (2) the 
costs and benefits. These types of evaluations could assist management 
in making an informed decision within an environment that includes 
constrained resources. 

Management Selection. The design of the Enterprise Risk Management 
Program does not require management to document the rationale for 
selecting a particular risk response. Such documentation could help 
improve accountability and facilitate analysis of the effectiveness of 
actions taken. 

Implementation and Monitoring. Although the design of the Enterprise 
Risk Management Program requires periodic assessments of the overall 
efficiency and effectiveness of the risk management program, it does 
not offer any detail regarding how these assessments will be performed. 
For example, the program does not provide details such as the 
frequency, scope, or methodology for these reviews. Further, the design 
does not require periodic assessments of implemented risk responses. 
These types of monitoring activities are critical in helping management 
to identify problems with the overall risk management program and to 
determine whether risk responses are preventing or mitigating risks and 
operating as intended. 

Internal Environment. The Enterprise Risk Management Program includes 
many of the elements that define this component. However, the design 
could be improved by (1) incorporating the importance of ethical values 
into the risk management program and (2) ensuring management maintains 
the competence of its personnel by providing for continuous training to 
update personnel on risk management practices and techniques. 

Information and Communication. The design of NIH's Enterprise Risk 
Management Program does not require the collection and dissemination of 
pertinent information to relevant external stakeholders in a form and 
time frame consistent with NIH's risk management needs. For example, 
although the design requires annual reporting, in aggregate, to HHS on 
the adequacy of internal control, it does not require communication 
with other external stakeholders, such as congressional oversight 
committees. 

Implementation of the Enterprise Risk Management Program Has Been 
Hampered by Lack of Milestones: 

The design and implementation of NIH's new risk management program is 
not yet completed, despite an over 3-year effort. Without a sound risk 
management program NIH cannot be reasonably assured that it will be 
able to effectively and proactively identify, assess, and mitigate 
risks before they become problems that affect NIH's ability to achieve 
its mission. During fiscal year 2008, the NIH OD implemented the first 
two steps of the six steps in its new risk management program. NIH had 
(1) organized the risk structure at the NIH OD and ICs, and identified 
and trained personnel responsible for managing risks within NIH OD, and 
(2) identified and scored risk at the NIH OD. NIH OD officials said 
they planned to complete the IC-level implementation of these two steps 
by the end of fiscal year 2009. 

The NIH OD is responsible for the design and implementation of the new 
program, and it has developed a time line with milestones for 
implementing some steps of the program. However, the timeline's 
milestones are not firm, and the NIH OD has revised the timeline to 
accommodate delays. According to NIH OD officials, they have 
experienced delays in designing and implementing the new program 
because of a change in contractors, balancing limited staff resources 
with competing demands, and underestimating the amount of time 
necessary for implementing specific steps of the program. As of the 
completion of our draft report, the NIH OD had not set a date for fully 
implementing the program agencywide. However, in providing written 
comments on a draft of this report, HHS indicated that the Enterprise 
Risk Management Program at NIH was scheduled for full implementation 
throughout NIH by June 2010. (See agency comments and our evaluation 
for additional details.) 

Conclusions: 

While NIH's decentralized structure allows NIH to address a wide range 
of research areas, it also creates significant oversight challenges. 
The ICs operate largely independently--each with its own budget, 
mission, and staff--making it vitally important that NIH and especially 
the OD have the means to ensure that all the ICs operate in accordance 
with NIH's policies and mission. With an annual budget of nearly $30 
billion, plus an additional $10 billion in funding available in 2009 
and 2010 through the American Recovery and Reinvestment Act of 2009, 
the financial stakes are high. 

We found gaps in NIH's ability to monitor key aspects of its extramural 
funding process. Specifically, NIH's OD does not monitor extramural 
funding decisions in which IC directors exercise their discretion to 
skip applications and make exceptions, even though information on these 
decisions is collected at the IC level. Without routine monitoring, 
which is consistent with federal internal control standards, NIH does 
not have the information to be reasonably assured that these decisions 
are appropriate and support the agency's mission. Appropriate funding 
decisions are critical to ensuring an effective use of taxpayer dollars 
and supporting NIH's reputation as the premier federal medical research 
agency in the United States. In reviewing selected administrative 
operations, we also found a key weakness in the design of the controls 
the NIH OD has established for oversight of travel and Title 42 
personnel appointments. Without internal controls that include risk- 
based monitoring of the controls' actual implementation, NIH cannot be 
reasonably assured that these controls are effective and operating as 
intended in areas identified as posing potential risks to NIH. 

Given these issues, a comprehensive risk management program could help 
ensure that such monitoring gaps are identified and addressed. NIH has 
recognized the importance of risk management to its organization and 
has taken steps towards implementing its new Enterprise Risk Management 
Program. Specifically, NIH has organized the risk structure at the NIH 
OD and ICs, identified and trained personnel responsible for managing 
risks within the NIH OD, and made progress in identifying and scoring 
risks at both the NIH OD and the ICs, which represent important steps. 
However, the design of the Enterprise Risk Management Program lacks 
several key components identified in our framework as necessary for 
effective risk management and the program has not yet been fully 
implemented throughout NIH. 

Recommendations for Executive Action: 

To ensure effective oversight of extramural funding decisions, we 
recommend that the Director of NIH establish a process for routine 
monitoring of the extramural funding decisions in which the IC 
directors use their discretion to skip applications or fund 
applications as exceptions. 

To help ensure that NIH has a comprehensive program to effectively 
address potential risks to the agency's mission, including those 
related to the monitoring of extramural research funding decisions, 
travel, and personnel appointments, we recommend that the Director of 
NIH take two actions to complete the design and implementation of NIH's 
Enterprise Risk Management Program: 

* Add key components and related elements needed to achieve 
comprehensive and effective agencywide risk management to the design of 
NIH's Enterprise Risk Management Program, including: 

- mission-based strategic goals and objectives as a precondition for 
risk management and risks to be assessed on the basis of their impact 
on the achievement of these goals and objectives; 

- the evaluation of risk responses to consider the effect on the 
likelihood of occurrence and impact of a potential risk and the costs 
and benefits; 

- the documentation of the rationale for selecting risk responses; 

- additional detail regarding how the assessments of the overall 
efficiency and effectiveness of the risk management program will be 
performed; 

- periodic assessments of implemented risk responses; 

- the importance of ethical values; 

- continuous training to maintain the competence of personnel carrying 
out risk management duties; and: 

- communication with relevant external stakeholders. 

* Identify major milestones, including a final implementation date, to 
help ensure that NIH completes and implements the Enterprise Risk 
Management Program in a reasonable time frame. 

Agency Comments and Our Evaluation: 

The Department of Health and Human Services provided written comments 
on a draft of this report, which are reprinted in appendix III. In 
responding to our draft report, HHS disagreed with the first 
recommendation and partially concurred with the second recommendation. 
In response to the third recommendation, HHS provided new information. 
The following sections summarize HHS's comments on each of our three 
major findings and related recommendations and provide our responses. 

OD Oversight of Extramural Funding Decisions: 

HHS disagreed with our recommendation that the Director of NIH should 
establish a process for routine monitoring of the extramural funding 
decisions in which the IC directors use their discretion to skip 
applications or fund applications as exceptions. In its written 
comments, HHS stated that we implied an inappropriate role for the NIH 
OD. Specifically, HHS said that the OD's role was not to provide input 
on the scientific reasoning for making skips and exceptions, which 
should be left to the judgment of the scientific officials who 
understand the current trends in science and the institute research 
portfolios. HHS further stressed that the ICs are required to document 
the reasons for these decisions and that the documents are available 
for review by the OD upon request. 

Our work shows there would be benefit for the Office of the Director of 
NIH, as part of its responsibility to oversee IC operations, to 
routinely monitor the extent to which IC directors use their discretion 
to skip applications and fund applications as exceptions. This 
monitoring can be consistent with NIH's stated reliance on scientific 
reasoning and the judgment of the scientific officials in making these 
decisions. As we noted in our draft report, when IC directors decide to 
skip applications and fund applications as exceptions, they do so by 
considering factors other than the science-based priority scores 
originally assigned to each application by NIH's initial peer review 
groups and advisory councils. There can be good reasons for the 
decision to skip an application or fund an application as an exception, 
such as the desire to maintain a diverse portfolio of work. Routinely 
monitoring the extent to which IC directors use their discretion to 
skip applications and fund applications as exceptions would position 
the Director of NIH to help ensure that these decisions are consistent 
with NIH policy goals and are therefore appropriate. Such routine 
monitoring would also enable the Director of NIH to identify instances 
in which further review by appropriate officials may be desirable. 
Further, the routine monitoring we recommended is consistent with other 
efforts by the OD to monitor extramural funding decisions. As we noted 
in our draft report, the NIH OD already collects certain information 
related to extramural funding decisions, such as the priority rankings 
of funded applications and the number of extramural grants awarded to 
new investigators in response to an NIH-wide initiative. Finally, NIH 
OD monitoring activities would be consistent with federal internal 
control standards. 

In related comments, HHS drew attention to our finding that the share 
of RO1 grants awarded outside the payline (as exceptions) increased 
substantially from fiscal year 2003 through fiscal year 2007, and noted 
that this increase resulted largely from a corresponding increase in 
the number of RO1 grants awarded to new investigators. We agree with 
HHS, and noted in our draft report that our analysis of NIH's records 
showed that the NIH-wide initiative to fund new investigators was one 
of the most frequently cited reasons for funding an application as an 
exception. HHS further stated that it would like to review our methods 
for quantifying the number of extramural grants funded as exceptions. 
As we indicated in the scope and methodology section of our draft 
report, we based our analysis on data provided by NIH. We noted that 
NIH provided us with information about the payline established by each 
of the 24 ICs for each fiscal year from 2003 through 2007, and the 
number of RO1 grant applications funded relative to each IC's payline 
for each year. 

Design of Controls Over NIH's Travel and Personnel Appointment 
Processes: 

HHS concurred with our finding that the design of NIH's Title 42 
personnel appointment process included key control activities and some 
monitoring but lacked systematic risk-based monitoring. HHS said that 
it intends to incorporate risk-based monitoring into the Title 42 
personnel appointment process. HHS also commented that NIH has 
identified and scored the agency travel process within its Enterprise 
Risk Management Program (discussed in the next section) but that it 
will reassess the travel risk levels to ensure that they are 
appropriate. 

Design of NIH's Management Control Program and Enterprise Risk 
Management Program: 

In response to our recommendation that NIH add key components to the 
design of its Enterprise Risk Management Program to achieve 
comprehensive and effective agencywide risk management, HHS agreed with 
some of our specific recommendations and disagreed with others. We 
identified eight specific items in this area; HHS agreed with four, 
partially agreed with one, and disagreed with three. 

* HHS agreed that the design of NIH's Enterprise Risk Management 
Program should be modified to include the evaluation of risk responses 
to consider the effect on the likelihood of occurrence and impact of a 
potential risk and the costs and benefits. HHS noted that NIH will 
modify its Enterprise Risk Management Guidebook to reflect this 
recommendation. 

* HHS agreed that the design of NIH's Enterprise Risk Management 
Program should be modified to include documentation of the rationale 
for selecting risk responses. HHS noted that it appreciated the 
feedback and has incorporated this element into NIH's processes and 
amended the NIH Enterprise Risk Management Guidebook. 

* HHS agreed that the design of NIH's Enterprise Risk Management 
Program should be modified to include periodic assessment of 
implemented risk responses. 

* HHS agreed that the design of NIH's Enterprise Risk Management 
Program should be modified to include additional detail regarding how 
the assessments of the overall efficiency and effectiveness of the risk 
management program will be performed. However, HHS noted that the NIH 
Enterprise Risk Management Program has already undergone incremental 
evaluation during implementation. HHS also noted that NIH plans to 
develop a program evaluation process and conduct periodic reviews of 
the program in fiscal year 2011. 

* HHS partially agreed that the design of NIH's Enterprise Risk 
Management Program should be modified to include communication with 
relevant external stakeholders. HHS noted that NIH promptly responds to 
all requests for information from external stakeholders. However, HHS 
also noted that the Enterprise Risk Management Program will include 
external communications as it matures. 

* HHS did not agree that the design of NIH's Enterprise Risk Management 
Program should be modified to include mission-based strategic goals and 
objectives as a precondition for risk management and to assess risks on 
the basis of their impact on the achievement of these goals and 
objectives. HHS said that NIH's Enterprise Risk Management Program is 
designed to identify and manage risks before they become obstacles to 
the NIH mission and noted that the ICs establish their own strategic 
goals and objectives. As we noted in the draft report, the design of 
the program does not require the NIH OD or ICs to set mission-based 
strategic goals and objectives as a precondition for risk management, 
nor does the design call for risks to be assessed on the basis of their 
impact on IC-or NIH OD-level missions. We continue to believe that a 
clear and explicit link to strategic goals and objectives would help 
ensure that risks are routinely assessed based on their potential 
impact to achieving NIH's mission and would identify criteria against 
which to measure performance. 

* HHS did not agree that the design of NIH's Enterprise Risk Management 
Program should be modified to include the importance of ethical values. 
HHS said that NIH's risk management program already operates within the 
context of a positive environment in which integrity and ethical values 
play a key role. However, HHS said that NIH would modify the design of 
the Enterprise Risk Management Program as we recommended, by amending 
the Enterprise Risk Management Guidebook to include specific language 
addressing the importance of ethics at NIH. 

* HHS did not agree that the design of NIH's Enterprise Risk Management 
Program should be modified to include continuous training to maintain 
the competence of personnel carrying out risk management duties. 
Nevertheless, HHS stated that NIH has provided training to over 400 
individuals who hold significant risk management roles and noted that 
NIH plans to develop continuous training for all employees on risk 
management. Moreover, HHS said that NIH will modify the design of the 
Enterprise Risk Management Program as we recommended, by modifying the 
Enterprise Risk Management training plan to incorporate ongoing 
training such as training updates and refreshers. 

In response to our recommendation that the Director of NIH should 
identify major milestones, including a final implementation date, to 
help ensure timely implementation of the Enterprise Risk Management 
Program, HHS identified a final implementation date of June 2010. 
Although HHS asserted that NIH's Enterprise Risk Management Program is 
fully functional because NIH has implemented all six steps of the 
program at some level, as we noted in our draft report and as HHS 
confirmed in its written comments, several elements of the program have 
not been implemented across all of NIH. For example, HHS stated that 
steps one and two (identify and score risks) have been implemented 
across all of NIH--including the OD and the ICs--but that steps three 
and four (assess and remediate risks) have been implemented at the OD 
level but not across the ICs. If NIH proceeds with the actions and time 
frames outlined in HHS's comments, it should meet the intent of our 
recommendation. 

HHS stated that the prior risk management program--which our draft 
referred to as the "current" program--was discontinued in 2006. This 
statement is not consistent with the information we gathered during the 
time of our review nor with the policy manual posted on the NIH Web 
site, which states that the Management Control Program was "temporarily 
rescinded effective June 24, 2009,"--1 day after HHS received our 
report for review and comment--and that replacement guidance has not 
been issued. If the prior program has been discontinued and the final 
implementation date for the new program is scheduled for June 2010, NIH 
may have been operating without a fully functioning risk assessment 
program in place, which is a key element of a system of internal 
control. Although we believe our draft report correctly characterized 
the status of NIH's Management Control Program and Enterprise Risk 
Management Program at the time of our review, in response to HHS's 
comments we revised the wording in our report to more clearly 
distinguish between the new Enterprise Risk Management Program and the 
Management Control Program it is replacing. 

In commenting on our evaluation of the NIH Enterprise Risk Management 
Program, HHS questioned the criteria we used in our evaluation. HHS 
stated that it defines risk management as synonymous with internal 
control and that the NIH Enterprise Risk Management Program was 
developed based on the Standards for Internal Control in the Federal 
Government and the Office of Management and Budget (OMB) Circular A- 
123. Thus, HHS suggested that we should revise our report using 
different criteria. We believe that our criteria are appropriate for 
the evaluation. As noted in the draft report, GAO developed the 
framework based on authoritative literature and standards, as well as 
previous GAO reports and testimonies. We consulted the Government 
Performance and Results Act (GPRA) of 1993; the Government Auditing 
Standards, 2003 Revision; GAO's Standards for Internal Control in the 
Federal Government (November 1999); guidance from OMB; the work of the 
President's Commission on Risk Management; consulting papers; and the 
enterprise risk management approach of the Committee of Sponsoring 
Organizations (COSO) of the Treadway Commission. We also reviewed 
numerous risk management frameworks from industry, government, and 
academic sources. 

Furthermore, our draft report noted the relationship between internal 
control and risk management. Specifically, risk management is a 
continuous process through which an organization identifies, assesses, 
and mitigates risks, and through risk management, an organization can 
identify the most significant areas in which to place or enhance 
internal control. Systems of internal control may help an organization 
prevent or reduce risks, such as fraud, waste, abuse, or mismanagement. 
Internal control standards, therefore, provide an important tool for 
use in risk management. For example, in response to our draft report, 
NIH pointed out a variety of management oversight mechanisms, as 
discussed below. Those mechanisms could be considered part of NIH's 
internal controls, but are not part of its risk management program. We 
believe that the framework we used to evaluate NIH's risk management 
program was appropriate. 

In addition, HHS commented that our report implied that the risk 
management program is the sole management oversight mechanism at NIH 
and that we failed to acknowledge other oversight bodies and functions. 
We agree that NIH has many mechanisms for managerial oversight and 
accountability and we cited some of the mechanisms HHS specified in the 
draft report, such as oversight of travel and Title 42 personnel 
appointments. However, it was beyond the scope of our report to 
evaluate the full spectrum of NIH's oversight and accountability 
mechanisms. Further, regardless of the number or type of the other 
oversight mechanisms in place at NIH, these do not in any way diminish 
NIH's need to make its risk management program fully functioning, 
comprehensive, and effective. 

HHS also provided us with technical comments, which we incorporated as 
appropriate. 

As arranged with your office, unless you publicly announce its contents 
earlier, we plan no further distribution of this report until 30 days 
after its issue date. At that time, we will send copies of this report 
to other interested congressional committees, the Secretary of HHS, and 
the Director of NIH. This report will also be available on the GAO Web 
site at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions regarding this report, please 
contact Linda T. Kohn at (202) 512-7114 or kohnl@gao.gov or Susan 
Ragland at (202) 512-8486 or raglands@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. Key contributors to this report are 
listed in appendix IV. 

Sincerely yours, 

Signed by: 

Linda T. Kohn, Director: 
Health Care: 

Signed by: 

Susan Ragland, Director: 
Financial Management and Assurance: 

[End of section] 

Appendix I: Scope and Methodology: 

To gain an understanding of the process used to make extramural 
research funding decisions, we reviewed the laws and regulations 
governing the funding process and National Institutes of Health (NIH) 
policies related to each stage of the process. We also interviewed NIH 
officials with responsibility for establishing these policies and 
overseeing the institutes and centers' (IC) implementation of this 
process. In addition, to develop a detailed understanding of how the 24 
ICs that fund extramural research implement the process, we selected 3 
of the 24 ICs for a more detailed review: the National Cancer Institute 
(NCI), the National Institute on Alcohol Abuse and Alcoholism (NIAAA), 
and the National Institute of Diabetes and Digestive and Kidney 
Diseases (NIDDK). These ICs were chosen because they vary in budget 
size and focus on different disease-specific research missions. We also 
included the Center for Scientific Review (CSR), which does not fund 
extramural research but is responsible for implementing the initial 
steps in the extramural research funding process, including receipt of 
all grant applications. 

At the IC level, we reviewed IC policies and guidance for implementing 
the extramural research funding process and interviewed officials at 
each of the 3 selected ICs plus CSR about their roles in receiving 
applications, facilitating peer review of the applications, and making 
final funding decisions. We also interviewed members of the NCI, NIAAA, 
and NIDDK advisory councils about their role in reviewing and making 
recommendations regarding extramural grant applications. In addition, 
we analyzed selected data from the 24 ICs that fund extramural research 
regarding funding decisions for grants in NIH's R01 category, which is 
the most common of NIH's various grant categories. The R01 grant is the 
original and historically oldest grant mechanism used by NIH. This type 
of grant is awarded to organizations of all types (universities, 
colleges, small businesses, for-profit, foreign and domestic, etc.) to 
support a discrete, specified project to be performed by a named 
investigator or investigators. Specifically, we requested information 
about the paylines each of the 24 ICs established during fiscal years 
2003 through 2007 to be used when making funding decisions. (The 
payline roughly corresponds with the number of extramural grant 
applications an IC will be able to fund each year and is based on 
projections of the total funding available for grants at the IC that 
year, the average dollar amount expected to be awarded per application, 
and the number of applications coming to an IC.) We also requested data 
about the number of R01 grant applications received, scored, and 
recommended by the peer review groups; the total number of grant 
applications funded; and the number of grant applications funded 
relative to each IC's payline that year. We used the data to analyze 
trends in funding decisions over the 5-year period. In order to analyze 
the reasons the ICs cited when funding applications as exceptions to 
the payline for R01 grants, we collected IC documentation for fiscal 
years 2006 and 2007 from NCI, NIAAA, and NIDDK. Because the total 
number of exception decisions made by NCI and NIDDK were large during 
this time frame, we analyzed documents for a random sample of the 
grants awarded as exceptions to the main payline. We also reviewed IC 
documentation related to applications with priority scores above the 
main payline that were not funded by NCI and NIDDK. NIAAA did not 
choose to skip any applications during these fiscal years. 

To ensure that the IC data were sufficiently reliable for our analyses, 
we conducted detailed data reliability assessments of the data that we 
used. We assessed the reliability of the IC data by reviewing existing 
information about the data and the system that produced them and 
interviewing agency officials knowledgeable about the data. We 
determined that the data were sufficiently reliable for the purposes of 
this report. 

To gain an understanding of the design of control and monitoring 
activities over travel and Title 42 personnel appointments, we reviewed 
relevant NIH policies and guidance.[Footnote 35] To further our 
understanding of control and monitoring activities, we also performed 
walkthroughs of the travel and Title 42 personnel appointment processes 
at three ICs--NCI, NIDDK, and NIAAA; these were the three ICs selected 
for our review of the extramural research funding process. 

During our walkthroughs of the travel process, we reviewed 
authorizations, vouchers, and supporting receipts for travel 
transactions at each of the selected ICs. During our walkthroughs of 
the Title 42 personnel appointment process, we reviewed checklists 
showing documents included in the appointment packages, routing slips 
showing who received the appointment packages, and memos documenting 
approvals for Title 42 personnel appointments at each of the selected 
ICs. We interviewed key officials from the NIH OD and the ICs, 
including the: 

* NIH Deputy Director and the NIH Deputy Director for Management to 
clarify our understanding of the differences between the roles of the 
NIH OD and the ICs in the travel and personnel appointment processes 
and the associated control and monitoring activities; 

* NIH Director of Financial Management and the NIH Director and Deputy 
Director of the Office of Human Resources to gain an understanding of 
the control and monitoring activities that the NIH OD performs over 
travel and Title 42 personnel appointments; and: 

* IC Executive Officers (the highest level officials at the ICs that 
oversee administrative activities) and other specialists within the ICs 
to clarify our understanding of control and monitoring activities in 
the travel and Title 42 personnel appointments at the IC level. 

We compared the design of the processes to GAO's Standards for Internal 
Control in the Federal Government[Footnote 36] to determine if the 
processes as designed included appropriate control and monitoring 
activities. While the design of control activities is based on NIH-wide 
policies and procedures, monitoring activities vary at the individual 
ICs. Therefore, our review of monitoring activities for travel and 
Title 42 personnel appointments at these selected ICs cannot be 
generalized to the other ICs. The scope of our audit did not include 
testing the implementation of controls over travel and Title 42 
personnel appointments. 

To gain an understanding of the design of the NIH Management Control 
Program, we reviewed relevant NIH policy and supporting documentation. 
Specifically, we reviewed relevant NIH policies[Footnote 37] and the 
NIH OD's fiscal year 2008 guidance to the ICs on reporting risk 
management activities. To gain an understanding of the design of the 
Enterprise Risk Management Program we reviewed NIH draft guidance. 
[Footnote 38] We also reviewed the time lines for implementing the 
Enterprise Risk Management Program to determine the estimated 
implementation dates. We interviewed key officials from the NIH OD 
including the: 

* NIH Deputy Director and the NIH Deputy Director for Management to 
gain a high-level understanding of how the Enterprise Risk Management 
Program will address recent oversight issues at NIEHS and help NIH to 
better manage its decentralized organization; 

* NIH Director of Financial Management to understand the risk 
activities NIH performed for fiscal year 2008 as part of the NIH 
Management Control Program; and: 

* NIH Director for the Office of Management Assessment--the office with 
primary responsibility for designing and implementing the new risk 
management program--to understand current risk activities at NIH, to 
clarify the design of the new risk management program, and to further 
our understanding of the implementation time line for the new risk 
management program as well as the cause for delays in implementation. 

We compared elements of the NIH Management Control Program and the 
Enterprise Risk Management Program to our risk management framework 
[Footnote 39] to determine if the designs contain the key components of 
an effective risk management program. We did not review the 
implementation of either the NIH Management Control Program or the 
Enterprise Risk Management Program because, at the time of our review, 
NIH did not plan to continue the Management Control Program and the 
Enterprise Risk Management Program was not yet fully implemented. 

We conducted this performance audit from March 2008 to September 2009, 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

[End of section] 

Appendix II: NIH Organization and Mission: 

As the primary federal agency for supporting medical research in the 
United States, the National Institutes of Health's (NIH) mission is 
"science in pursuit of fundamental knowledge about the nature and 
behavior of living systems and the application of that knowledge to 
extend healthy life and reduce the burdens of illness and disability". 
NIH is headed by a Director who is supported by 11 staff offices and 1 
program office within the NIH Office of the Director (OD) and 27 
institutes and centers (IC). Figure 3 depicts the organizational 
structure of NIH. 

Figure 3: Organizational Structure of NIH: 

[Refer to PDF for image: organizational chart] 

Top level: 
Immediate Office of the Director; 

Reporting to the Immediate Office of the Director: 

Office of the Director Program Office: Division of Program 
Coordination, Planning, and Strategic Initiatives; 

Office of the Director Staff Offices: 
* Office of Extramural Research; 
* Office of Intramural Research; 
* Office of Management/Chief Financial Officer; 
* Office of Science Policy; 
* Office of Communications and Public Liaison; 
* Office of Equal Opportunity and Diversity Management; 
* Office of Legislative Policy and Analysis; 
* Executive Office; 
* Office of the Ombudsman/Center for Cooperative Resolution; 
* NIH Ethics Office; 
* Office of the Chief Information Officer. 

Second level, direct relationship with Immediate Office of the 
Director: 

National Cancer Institute; 
National Eye Institute; 
National Heart, Lung, and Blood Institute; 
National Human Genome Research Institute; 
National Institute on Aging; 
National Institute on Alcohol Abuse and Alcoholism; 
National Institute of Allergy and Infectious Diseases; 
National Institute of Arthritis and Musculoskeletal and Skin Diseases; 
National Institute of Biomedical Imaging and Bioengineering; 
Eunice Kennedy Shriver National Institute of Child Health and Human 
Development; 
National Institute on Deafness and Other Communication Disorders; 
National Institute of Dental and Craniofacial Research; 
National Institute of Diabetes and Digestive and Kidney Diseases; 
National Institute on Drug Abuse; 
National Institute of Environmental Health Sciences; 
National Institute of General Medical Sciences; 
National Institute of Mental Health; 
National Institute of Neurological Disorders and Stroke; 
National Institute of Nursing Research; 
National Library of Medicine; 
John E. Fogarty International Center for Advanced Study in the Health 
Sciences; 
National Center for Complementary and Alternative Medicine; 
National Center on Minority Health and Health Disparities; 
National Center for Research Resources. 

Third level, direct relationship with Immediate Office of the Director: 

Clinical Center; 
Center for Information Technology; 
Center for Scientific Review. 

Source: NIH. 

[End of figure] 

The ICs, which were established over time, each have an explicit 
mission focused on a particular disease or organ system, an area of 
human health and development, or aspects of research support.[Footnote 
40] The first institute, the National Cancer Institute, was established 
in 1937, and the newest institute, the National Institute of Biomedical 
Imaging and Bioengineering, was established in 2000. Research funded by 
NIH can be conducted by scientists in NIH laboratories and Clinical 
Center--called intramural research--or by nonfederal scientists at 
universities, academic health centers, hospitals, and independent 
research institutions--called extramural research. Table 3 depicts a 
time line of the establishment of the 27 ICs and their respective 
missions and fiscal year 2008 appropriations. 

Table 3: Overview of ICs Including Establishment Date, Mission, and 
Fiscal Year 2008 Appropriation: 

IC: National Cancer Institute; 
Year established: 1937; 
Mission: Conducts and supports research that will lead to a future in 
which we can prevent cancer, identify cancers that do develop at the 
earliest stage, eliminate cancers through innovative treatment 
interventions, and biologically control those cancers that we cannot 
eliminate so they become manageable, chronic diseases; 
FY 2008 appropriation (in 000s): $4,830,647. 

IC: Center for Scientific Review; 
Year established: 1946; 
Mission: Conducts initial peer reviews of the majority of research and 
research-training applications submitted to NIH; 
FY 2008 appropriation (in 000s): N/A[A]. 

IC: National Institute of Allergy and Infectious Diseases; 
Year established: 1948; 
Mission: Leads research that strives to understand, treat, and 
ultimately prevent the myriad infectious, immunologic, and allergic 
diseases that threaten millions of human lives; 
FY 2008 appropriation (in 000s): $4,583,344. 

IC: National Heart, Lung, and Blood Institute; 
Year established: 1948; 
Mission: Provides leadership for a national program in diseases of the 
heart, blood vessels, lung, and blood; blood resources; and sleep 
disorders; 
FY 2008 appropriation (in 000s): $2,937,654. 

IC: National Institute of Dental and Craniofacial Research; 
Year established: 1948; 
Mission: Provides leadership for a national research program designed 
to understand, treat, and ultimately prevent the infectious and 
inherited craniofacial-oral-dental diseases and disorders that 
compromise millions of human lives; 
FY 2008 appropriation (in 000s): $392,233. 

IC: National Institute of Mental Health; 
Year established: 1949; 
Mission: Provides national leadership dedicated to understanding, 
treating, and preventing mental illnesses through basic research on the 
brain and behavior, and through clinical, epidemiological, and services 
research; 
FY 2008 appropriation (in 000s): $1,412,951. 

IC: National Institute of Diabetes and Digestive and Kidney Diseases; 
Year established: 1950; 
Mission: Conducts and supports basic and applied research and provides 
leadership for a national program in diabetes, endocrinology, and 
metabolic diseases; digestive diseases and nutrition; and kidney, 
urologic, and hematologic diseases; 
FY 2008 appropriation (in 000s): $1,715,761. 

IC: National Institute of Neurological Disorders and Stroke; 
Year established: 1950; 
Mission: Seeks to reduce the burden of neurological diseases by 
supporting and conducting research, both basic and clinical, on the 
normal and diseased nervous system, fostering the training of 
investigators in the basic and clinical neurosciences, and seeking 
better understanding, diagnosis, treatment, and prevention of 
neurological disorders; 
FY 2008 appropriation (in 000s): $1,552,113. 

IC: Clinical Center; 
Year established: 1953; 
Mission: Provides the patient care, services, and environment needed to 
initiate and support the highest quality conduct of and training in 
clinical research; 
FY 2008 appropriation (in 000s): N/A[A]. 

IC: National Library of Medicine; 
Year established: 1956; 
Mission: Collects, organizes, and makes available biomedical science 
information to scientists, health professionals, and the public; 
FY 2008 appropriation (in 000s): $322,212. 

IC: National Institute of General Medical Sciences; 
Year established: 1962; 
Mission: Supports basic biomedical research that is not targeted to 
specific diseases but rather funds studies on genes, proteins, and 
cells, as well as on fundamental processes like communication within 
and between cells, how our bodies use energy, and how we respond to 
medicines; 
FY 2008 appropriation (in 000s): $1,946,104. 

IC: Eunice Kennedy Shriver National Institute of Child Health and Human 
Development; 
Year established: 1962; 
Mission: Leads research on fertility, pregnancy, growth, development, 
and medical rehabilitation that strives to ensure that every child is 
born healthy and wanted and grows up free from disease and disability; 
FY 2008 appropriation (in 000s): $1,261,381. 

IC: National Center for Research Resources; 
Year established: 1962; 
Mission: Provides laboratory scientists and clinical researchers with 
the environments and tools they need to understand, detect, treat, and 
prevent a wide range of diseases; 
FY 2008 appropriation (in 000s): $1,155,560. 

IC: Center for Information Technology; 
Year established: 1964; 
Mission: Incorporates the power of modern computers into the biomedical 
programs and administrative procedures of NIH by focusing on three 
primary activities: conducting-computational biosciences research, 
developing computer systems, and providing computer facilities; 
FY 2008 appropriation (in 000s): N/A[A]. 

IC: National Eye Institute; 
Year established: 1968; 
Mission: Conducts and supports research that helps prevent and treat 
eye diseases and other disorders of vision; 
FY 2008 appropriation (in 000s): $670,664. 

IC: John E. Fogarty International Center for Advanced Study in the 
Health Sciences; 
Year established: 1968; 
Mission: Promotes and supports scientific research and training 
internationally to reduce disparities in global health; 
FY 2008 appropriation (in 000s): $66,912. 

IC: National Institute of Environmental Health Sciences; 
Year established: 1969; 
Mission: Reduces the burden of human illness and dysfunction from 
environmental causes by, defining how environmental exposures, genetic 
susceptibility, and age interact to affect an individual's health; 
FY 2008 appropriation (in 000s): $645,669. 

IC: National Institute on Alcohol Abuse and Alcoholism; 
Year established: 1970; 
Mission: Conducts research focused on improving the treatment and 
prevention of alcoholism and alcohol-related problems to reduce the 
enormous health, social, and economic consequences of this disease; 
FY 2008 appropriation (in 000s): $438,579. 

IC: National Institute on Drug Abuse; 
Year established: 1973; 
Mission: Supports and conducts research across a broad range of 
disciplines and rapid and effective dissemination of results of that 
research to improve drug abuse and addiction prevention, treatment, and 
policy; 
FY 2008 appropriation (in 000s): $1,006,022. 

IC: National Institute on Aging; 
Year established: 1974; 
Mission: Leads a national program of research on the biomedical, 
social, and behavioral aspects of the aging process; the prevention of 
age-related diseases and disabilities; and the promotion of a better 
quality of life for all older Americans; 
FY 2008 appropriation (in 000s): $1,052,830. 

IC: National Institute of Arthritis and Musculoskeletal and Skin 
Diseases; 
Year established: 1986; 
Mission: Supports research into the causes, treatment, and prevention 
of arthritis and musculoskeletal and skin diseases, the training of 
basic and clinical scientists to carry out this research, and the 
dissemination of information on research progress in these diseases; 
FY 2008 appropriation (in 000s): $511,291. 

IC: National Institute of Nursing Research; 
Year established: 1986; 
Mission: Supports clinical and basic research to establish a scientific 
basis for the care of individuals across the life span--including 
managing patients during illness and recovery to reducing risks for 
disease and disability; promoting healthy lifestyles; promoting quality 
of life in those with chronic illness; and caring for individuals at 
the end of life; 
FY 2008 appropriation (in 000s): $138,207. 

IC: National Institute on Deafness and Other Communication Disorders; 
Year established: 1988; 
Mission: Conducts and supports biomedical research and research 
training on normal and disordered processes of hearing, balance, smell, 
taste, voice, speech, and language that affect 46 million Americans; 
FY 2008 appropriation (in 000s): $396,234. 

IC: National Human Genome Research Institute; 
Year established: 1989; 
Mission: Supports the NIH component of the Human Genome Project, a 
worldwide research effort designed to analyze the structure of human 
DNA and determine the location of the estimated 30,000 to 40,000 human 
genes; 
FY 2008 appropriation (in 000s): $489,368. 

IC: National Center on Minority Health and Health Disparities; 
Year established: 1993; 
Mission: Promotes minority health and leads, coordinates, supports, and 
assesses NIH efforts to reduce and ultimately eliminate health 
disparities among minority and other medically underserved communities. 
Conducts and supports basic, clinical, social, and behavioral research; 
promotes research infrastructure and training; fosters emerging 
programs; disseminates information; and reaches out to minority and 
other medically underserved communities; 
FY 2008 appropriation (in 000s): $200,630. 

IC: National Center for Complementary and Alternative Medicine; 
Year established: 1999; 
Mission: Explores complementary and alternative medical practices in 
the context of rigorous science; trains researchers; and disseminates 
authoritative information; 
FY 2008 appropriation (in 000s): $122,224. 

IC: National Institute of Biomedical Imaging and Bioengineering; 
Year established: 2000; 
Mission: Improves health by promoting fundamental discoveries, design 
and development, and translation and assessment of technological 
capabilities in biomedical imaging and bioengineering; 
FY 2008 appropriation (in 000s): $300,233. 

Source: NIH. 

[A] The IC does not fund research and does not receive a separate 
appropriation but rather is funded through the NIH Management Fund. 

[End of table] 

[End of section] 

Appendix III: Comments from the National Institutes of Health: 

Department Of Health & Human Services: 
Office Of The Secretary: 
Assistant Secretary For Legislation: 
Washington, DC 20201: 
July 21, 2009: 

Linda T. Kohn: 
Director, Health Care: 
U.S. Government Accountability Office: 
441 G Street N.W. 
Washington, DC 20548: 

Dear Ms. Kohn: 

Enclosed are comments on the U.S. Government Accountability Office's 
(GAO) report entitled: National Institutes Of Health: Completion of 
Comprehensive Risk Management Program Essential to Effective Oversight 
(GAO-09-687). 

The Department appreciates the opportunity to review this report before 
its publication. 

Sincerely, 

Signed by: 

Barbara Pisaro Clark: 
Acting Assistant Secretary for Legislation: 

Attachment: 

[End of letter] 

General Comments Of The Department Of Health And Human Services (HHS) 
On The Government Accountability Office's (GAO) Draft Report Entitled: 
"National Institutes Of Health: Completion Of Comprehensive Risk 
Management Program Essential To Effective Oversight" (GAO-09-687): 

The National Institutes of Health (NIH) appreciates the review 
conducted by GAO and the opportunity to provide clarifications, 
corrections, and additional supporting documentation on this draft 
report. NIH respectfully submits the following general comments. 
Technical comments are included as a separate attachment. 

GAO's Overall Conclusion. Completion of Comprehensive Risk Management 
Program Essential to Effective Oversight: 

GAO's draft report implies that the NIH Enterprise Risk Management 
Program is the sole management oversight mechanism at NIH and fails to 
acknowledge other management, governance, and oversight bodies and 
functions. 

As emphasized by the title of the report, it appears that GAO equates 
the NIH Risk Management Program with overall governance and management 
of the agency. This is a false assumption. NIH offers the following 
clarification. 

While the risk management program at NIH is an important tool used by 
managers NIH-wide for decision-making, it is only one of many 
mechanisms available to the Office of the Director (OD) for effective 
oversight and accountability over the agency's 27 Institutes and 
Centers (IC). 

OD is responsible for a number of programs that manage risk, reasonably 
ensure internal control, provide governance, and drive performance for 
the agency. As an Attachment to this response, NIH has enumerated many, 
but not all, of the oversight functions of NIH-wide activities that OD 
performs. For example, the NIH Steering Committee, chaired by the NIH 
Director and composed of 10 IC Directors, works as an efficient and 
transparent forum for trans-NIH governance and streamlined decision-
making. Further, standing working groups provide recommendations to the 
NIH Steering Committee on issues such as intramural and extramural 
research, facilities, budget, and information technology. 

Another example of an OD oversight mechanism is the NIH Ethics Advisory 
Committee that provides centralized, consistent, and rigorous reviews 
of requests to engage in outside activities and awards that bestow 
gifts over $2,500. The Office of Extramural Research manages the Peer 
Review process, promotes scientific integrity, and manages research 
risks to patients. Likewise, the Office of Intramural Research manages 
the Institutional Review Board process for intramural research 
performed by NIH scientists. 

GAO Finding: NIH's Current and Proposed Risk Management Programs Do Not 
Fully Address Key Components of Effective Risk Management (p.19): 

1. Clarification: NIH's Enterprise Risk Management Program is based on 
widely-accepted Federal Government internal control standards. 

On page 19, GAO states that the program "does not fully address all of 
the components of GAO's framework for effective risk management." GAO's 
report relies on a comparison of the MIT Enterprise Risk Management 
Program with a risk management process published in an appendix to a 
report entitled, Risk Management: Further Refinements Needed to Assess 
Risks and Prioritize Protective Measures at Ports and Other Critical 
Infrastructure (GAO-06-91, Dec. 2005). 

In a footnote on page 7 of the report, GAO states that "risk management 
does not replace, but rather incorporates and expands on internal 
control." However, NIH defines risk management as synonymous with 
internal control. The title "Risk Management" was used to distinguish 
the new internal control program from the former internal control 
program, which ended in 2006. NIH believed that the Risk Management 
program would bring new enthusiasm to the internal control program from 
the scientific and management communities at NIH. NIH suggests deleting 
the footnote. 

The criteria to which GAO compared the NIH Enterprise Risk Management 
Program are not the same criteria that NIH used in developing its Risk 
Management Program. Specifically, NIH used the Standards for Internal 
Control in the Federal Government; the Office of Management and Budget 
(OMB) Circular A-123, Management's Responsibility for Internal Control; 
the GAO Internal Control and Management Evaluation Tool; and the 
Guidance Manual for OMB circular A-123 Assessments published by the 
Department of Health and Human Services. We believe that these 
frameworks provide robust guidance to effectively manage risk and 
reasonably ensure internal control. Although there is some overlap 
between the guidance GAO used and the guidance NIH used, NIH chose its 
criteria because it meets the OMB and HHS standards for an internal 
control program. 

While the fundamentals of the NIH Risk Management Program adhere to the 
above frameworks, NIH tailored the content and approach of the Program 
to have it work effectively in a scientific research environment. NIH 
strongly believes that these adaptations are critically important to 
the Program's success. For example, the NIH Program has developed a set 
of standardized criteria to evaluate and prioritize risks. Development 
of these criteria included input from a wide array of scientific and 
management personnel at NIH to make the criteria applicable to a wide 
range of risks at the agency. NIH suggests that GAO revise its report 
accordingly. 

2. Correction: GAO's draft report refers to a "current program" that is 
in fact a retired risk management program that has not been used in two 
years. The "proposed program" cited in the draft report is the NIH 
Enterprise Risk Management Program currently operating at NIH. 

On page 19, GAO indicates that NIH implemented its "current" program in 
1999 and updated it in 2004. This statement is incorrect.
The "current" program that was in place in 1999 and 2004 and was 
retired by NIH in 2006. All references to the "current" program in 
GAO's report correspond to the outdated program and are neither timely 
nor relevant to the activities currently being conducted and 
implemented as part of the NIH Enterprise Risk Management Program. The 
program identified by GAO as "proposed" is in fact the current program 
that is in operation at NIH. 

NIH recommends that GAO correctly refer to the retired program as well 
as the Enterprise Risk Management Program that is currently being 
implemented and operated at NIH. In addition, NIH recommends that GAO 
move all references to the retired program (pages 19-23), including 
related findings, to a background section or remove them entirely from 
the report. NIH has provided technical comments on each reference made 
to the retired program so that GAO may appropriately correct this 
error. 

3. NIH does not concur with GAO's finding that the NIII Enterprise Risk 
Management Program "does not require NIH OD or ICs to set mission-based 
strategic goals and objectives as a precondition for risk management;" 
the report fails to acknowledge that all 27 NIH ICs have strategic 
plans and that the Program is designed to "support the research mission 
and vision of NIB."[Footnote 41] 

On page 24 of the Strategic Goals, Objectives, and Constraints section 
of the draft report, GAO states that the program "does not require NIH 
OD or ICs to set mission-based strategic goals and objectives as a 
precondition for risk management." NIH recommends that GAO delete this 
finding and the associated recommendation. 

The NIH Enterprise Risk Management Program "is designed to proactively 
identify and manage risks before they become obstacles to the NIH 
mission."[Footnote 42] The NIH Enterprise Risk Management Program 
provides explicit guidance for all Program stakeholders to "Think 
strategically. Consider the goals, objectives and mission of the 
organization. Any event or condition that could prevent or inhibit the 
accomplishment of the organization's goals, objectives or mission 
should be documented as a risk."[Footnote 43] 

The ICs establish their own scientific research strategic goals and 
objectives. IC-level goals and objectives do not contradict each other, 
are relevant to the broader NIH-wide mission, and include measurement 
criteria. However, there is a direct link between each IC's operational 
strategies and those of NIH as a whole. NIH is the steward of medical 
and behavioral research for the nation. One of the four goals of the 
agency is to "exemplify and promote the highest level of scientific 
integrity, public accountability, and social responsibility in the 
conduct of science."[Footnote 44] In realizing this goal, the NIH 
Deputy Director for Management provides leadership and direction to 
programs such as the Enterprise Risk Management Program. 

The Program is part of a management strategy to support the scientific 
research mission of NIH. The NIH Deputy Director for Management's (DDM) 
Strategic Plan includes four goals: 

* Goal 1 - Improving Human Capital Planning and Management; 
* Goal 2 - Leveraging Information for Data Driven Decision-Making to 
achieve Performance Excellence; 
* Goal 3 - Employing Proactive Risk Management to Enhance Program 
Performance; 
* Goal 4 - Enhancing Internal Communications. 

The mission of the management community is to enable NIH to pursue its 
biomedical research mission of scientific discovery and advancement of 
knowledge by serving as a valued partner that provides timely, high 
quality, and responsive programs and services in a manner that reflects 
a commitment to excellence and the preservation of public trust. 

4. NIH concurs with GAO's finding and corresponding recommendation 
regarding alternatives evaluation, including evaluation of risk 
responses. 

On page 25 under the Alternatives Evaluation section of the draft 
report, GAO states that the program "does not require management to 
evaluate the risk responses identified to consider (1) the effect on 
the likelihood of occurrence and impact of a potential risk and (2) the 
costs and benefits." 

We appreciate this valuable feedback to improve our Program. As a 
result of GAO's review, findings and recommendations, NIH will update 
the Enterprise Risk Management Guidebook to reflect this 
recommendation. The proposed risk response should be reviewed by the 
Program and discussed during the Remediate phase to determine 
appropriateness. The resulting assignment of the risk response will 
consider alternatives based upon various factors, including cost and 
benefits. 

NIH is in the process of updating its policy regarding enterprise risk 
management and internal controls found in Manual Chapter 1750. 
Revisions to the policy will be reflected in updates to its Enterprise 
Risk Management Guidebook to address ongoing monitoring of risk 
strategies and potential alternatives. 

5. NIH concurs with GAO's finding and corresponding recommendation 
regarding documentation of management selection. 

On page 25 under the Management Selection section of the draft report, 
GAO states that the program does "not require management to document 
the rationale for selecting a particular risk response." We appreciate 
this feedback and have incorporated this clement into our processes and 
amended the Enterprise Risk Management Guidebook. 

6. NIH concurs with GAO's Implementation and Monitoring finding 
regarding how the assessments of the overall effectiveness of the risk 
management program should be performed. However, NIH offers the 
following clarifications. 

On page 25, within the Implementation and Monitoring section of the 
draft report, GAO states that the program "does not offer any detail 
regarding how assessments will be performed" and "the design does not 
require periodic assessments of implemented risk responses." 

The NIH Enterprise Risk Management Program has already undergone 
several incremental evaluations. First, upon completion of the pilot, 
the Program conducted an evaluation to gather stakeholder feedback. 
This feedback was used to further refine the NIH risk management 
methodology and the tools that support it. Results of the pilot 
evaluation are documented in a pilot test report that was completed at 
the conclusion of pilot test activities. 

In addition, after the completion of the risk identification and 
scoring steps in the first phase of implementation with the NIH OD, the 
Program conducted an internal evaluation, Program staff evaluated the 
process and outcomes and developed lessons leaned and recommendations 
for improvement to guide the Program in the implementation of Phase 2 
with the NIH ICs. Results of this evaluation are documented in the OD 
Baseline Report. 

An additional staff evaluation, documentation of lessons learned and 
development of recommendations for further Program improvements is 
currently underway and is expected to be finalized by August 15, 2009.
A more formal evaluation of the Program is being planned and will 
likely include a follow-up of the Risk Culture Survey that was 
conducted during the HHS audit in 2006. The Risk Culture Survey 
examines potential strengths and weakness in the risk management and 
control environment. It measures the impact of the Program on the 
internal control environment at NIH, focusing on Leadership and 
Strategy, Accountability and Reinforcement, People and Communication, 
Risk Management and Infrastructure. These four areas include sub-
components that include ethics, tone at the top, and training. 

NIH has identified target milestone periods for ongoing evaluation in 
the Risk Management portion of the NIH Deputy Director for Management's 
Administrative Management Strategic Plan. This includes developing a 
program evaluation process and conducting periodic reviews. NIH 
anticipates that by June 30, 2010, NIH will conduct a follow-up Risk 
Culture Survey and will develop a program evaluation process, to 
include determining the frequency, scope and methodology for the 
reviews. In FY 2011, NIH will conduct reviews of the overall efficiency 
and effectiveness of the NIH Enterprise Risk Management Program. 

7. NIH does not concur with the finding and corresponding 
recommendation regarding the Program's internal environment. NIH does 
incorporate the importance of ethical values into the Program and 
maintains the competence of its risk management personnel by providing 
training. 

NIH operates within an environment in which ethical values play a key 
role: 

On page 25, GAO states that the Risk Management Program could be 
improved by "incorporating the importance of ethical values." The 
Program already operates within the context of a positive environment 
in which integrity and ethical values play a key role. NIH has a formal 
code of conduct; senior management has established an ethical tone and 
consistently models and enforces conscientious and competent 
leadership; management takes disciplinary action whenever appropriate 
or necessary and NIH has an extensive set of guidance on ethics. The 
NIH Ethics Office, within the OD, works in tandem with each individual 
IC Ethics Program to provide mandatory ethics training-both for new and 
current employees-that exceeds federal requirements. The importance of 
ethical values is being reinforced constantly through other means and 
in other ways that more directly affect internal controls. 

Furthermore, the report does not provide any evidence to support this 
statement and does not state how this would be achieved. We ask that 
the finding and recommendation be deleted from the report. 
Nevertheless, NIH will amend the Enterprise Risk Management Guidebook 
to include specific language addressing the importance of ethics at 
NIH. 

NIH maintains the competence of its risk management personnel by 
providing training: 

On page 25, GAO states that the program could be improved by "ensuring 
management maintains the competence of its personnel by providing for 
continuous training to update personnel on risk management practices 
and techniques." It should be highlighted that NIH provided GAO with 
documentation about training given to OD and IC staff who hold 
significant risk management roles, as well as the completion dates of 
training. The information shows that NIH does ensure the competence of 
staff working in risk management and does not support GAO's conclusion 
about the lack of training on risk management practices and techniques. 
NIH does not understand how GAO reached this conclusion, and the report 
does not provide an explanation or supporting evidence for this 
funding. 

To date, the NIH Enterprise Risk Management Program has provided 
training to over 400 individuals who hold significant risk management 
roles. This includes training specifically tailored for all OD Office 
Directors and IC Executive Officers. In addition, specific training 
focusing on the methodology and the identification of risks was 
provided to OD Office staff and IC leadership staff identified by OD 
Office Directors and IC Executive Officers as having important risk 
management roles. Additional training data is available to GAO upon 
request. 

The number of individuals trained to date is a result of the NIH 
Enterprise Risk Management training plan, which establishes targets for 
continuous training. NIH has also developed a formal risk management 
training course that is available to all NIH employees through the NIH 
Training Center and is developing training for NIH Administrative 
Officers. Furthermore, the DDM Strategic Plan includes a strategy for 
developing role-based and general awareness risk management training to 
reinforce a culture of risk awareness among leadership and staff. The 
training, as well as risk management communications, help ensure that 
all employees understand their role in conducting operations in a 
manner that manages risk. NIH has not provided training updates and 
refreshers because the program first began reaching personnel in 2008 
and 2009. However, the NIH Enterprise Risk Management training plan 
will incorporate these as the Program continues. Therefore, NIH 
recommends that GAO change its finding and revise its recommendation 
for NIH to "fully implement its plan for ongoing training." 

8. NIH partially concurs with GAO's finding that the NIH Enterprise 
Risk Management Program should "require the collection and 
dissemination of pertinent information to relevant external 
stakeholders." 

On page 26 within the Information and Communication component, GAO 
states that "the program does not require the collection and 
dissemination of pertinent information to relevant external 
stakeholders in a form and timeframe consistent with NIH's risk 
management needs." NIH offers the following clarification. GAO places 
emphasis on the communications with internal stakeholders cited in the 
GAO framework, Risk Management. Further Refinements Needed to Assess 
Risks and Prioritize Protective Measures at Ports and Other Critical 
Infrastructure (GAO-06-91). 

NIH promptly responds to all requests for information from external 
stakeholders, including those related to internal controls and risk 
management. The Risk Management Program provides NIH and HHS internal 
stakeholders with timely and accurate risk management data. For 
example, the Program provides the NIH Risk Management Council and NIH 
Risk Management Senior Assessment Team with risk management reports, 
program status updates, and updates on emerging risk management issues 
such as the Recovery Act. Another example of internal risk management 
communications is the Program's delivery to HHS of the NIH annual 
Federal Managers' Financial Integrity Act (FMFIA) statement of 
assurance attesting that NIH federal programs have effective and 
efficient controls in place that meet the objectives of the FMFIA and 
OMB Circular A-123. 

The Program will include external communications and exchanges as it 
matures. 

9. NIH does not concur with GAO's finding that the Enterprise Risk 
Management Program has been hampered by a lack of milestones. Although 
the Program has encountered some delays, the Program maintains a 
schedule of milestones. 

NIH has implemented all six steps of the Enterprise Risk Management 
methodology. 

On pages 19 and 26, the GAO report states that the NIH program has "not 
yet [been] fully implemented despite an over 3-year effort." This 
statement is incorrect. 

During the field work phase of its audit, GAO inquired about whether 
the risk management methodology had been implemented NIH-wide. NIH 
stated that as of November 19, 2008, the new risk management 
methodology had not been fully implemented across NIH. 

However, since November 2008, NIH has implemented steps l and 2 
(Organize and Identify and Score) of the methodology across all 27 ICs, 
as well as steps 3 and 4 (Assess and Remediate) of the methodology 
across OD. The creation of a comprehensive NIH enterprise baseline risk 
inventory is a result of the agency's implementation of the Identify 
and Score steps of the methodology across NIH. Furthermore, On May 7, 
2009, NIH hosted the first Risk Management Council (RMC) meeting. The 
RMC provides guidance on program implementation and operations, has 
oversight over the completion of risk management activities, and 
reports results to the NIH Risk Management Senior Assessment Team. 
Because these two risk management governance structures are 
operational, NIH executed Steps 5 and 6 (Monitor and Report) of the 
methodology. Therefore, NIH believes that it has executed all six steps 
of its Enterprise Risk Management Program for a fully functioning 
Program. 

NIH began developing the Enterprise Risk Management Program in August 
2007. In just two years, NIH has designed and executed its Enterprise 
Risk Management Program to conform to GAO, OMB, and HHS guidance. This 
represents extraordinary progress. NIH recognizes the need for 
continued progress to fully implement and continuously improve the 
Program. NIH defines full implementation as an initial completion of 
the Program's first two steps (Organize and Identify and Score) and the 
continuous operation of the remaining four steps: Assess, Remediate, 
Monitor and Report. The NIH Enterprise Risk Management Program is 
scheduled for full implementation by June 2010. 

The Program maintains a schedule of milestones: 

On pages 19 and 26, GAO's draft report indicates that the NIH program 
has "been hampered by a lack of milestones." This statement is also 
incorrect. 

According to the GAO document, Performance Measurement and Evaluation, 
GAO/GGD-98-26, page 3, a "program may be any activity, project, 
function, or policy that has an identifiable purpose or set of 
objectives." The NIH Enterprise Risk Management Program maintains a 
schedule of milestones and defines the Program's goals and objectives 
in the NIH Management Strategic Plan discussed in Response #4 of this 
document. During the fieldwork phase of GAO's audit, NIH provided GAO 
with a Work Breakdown Structure document that contained significant 
milestones such as the following: 

* Complete a pilot of the Enterprise Risk Management Program 
methodology; 
* Implement steps I through 4 of the methodology across OD; 
* Implement steps I and 2 of the methodology across all 27 ICs; 
* Conduct risk management training to individuals who hold significant 
risk management roles; 
* Implement both risk management governance structures: the Risk 
Management Council and the Risk Management Senior Assessment Team. 

The development and implementation of the Program demonstrates the 
remarkable level of effort and commitment NIH has invested over the 
past two years. 

GAO does not provide agencies with an standard timeline for filly 
implementing a Risk Management Program: 

On page 29, GAO recommends that NIH complete and implement its program 
in a reasonable timeframe. GAO does not define what constitutes a 
"reasonable" timeframe. NIH offers the following clarification. 

In the December 2005 report, Risk Management Further Refinements Needed 
to Assess Risks and Prioritize Protective Measures at Ports and Other 
Critical Infrastructure (GAO-06-91), pages 99 and 101, GAO acknowledges 
that the "[GAO] risk management framework has been used to evaluate 
activities related to security and combating terrorism" and "is 
intended to be a starting point for risk management activities and will 
likely evolve as processes mature and lessons are learned." While the 
framework is a valuable tool for establishing a "full cycle of related 
activities from strategic planning through implementation and 
monitoring" and is used to "inform agency officials and decision makers 
of the basic components of a risk management system," it does not 
provide agencies with a standard timeline for fully implementing a risk 
management program. 

NIH would like to know the criteria GAO used as a basis for its 
conclusion and asks that this finding be modified in the report. 

GAO Finding: NIH Is Required to Use a Peer Review System to Make 
Extramural Funding Decisions; NIH's OD Does Not Monitor Key Decisions 
In Which IC Directors Exercise Their Discretion Over Funding Decisions 
(p.9): 

1. NIH does not concur with the recommendation that would require an 
oversight role for the OD that is inconsistent with the IC Director's 
authority to make grant award decisions. 

GAO states that IC directors can use their discretion and choose to 
fund applications on the basis of factors other than scientific merit, 
"skipping" over applications with higher scores or making "exceptions" 
to fund applications with lower scores. NIH stresses that while IC 
directors authorize these actions, their decisions are not made in 
isolation or without consultation, and review. The NIH OD ensures that 
there is a process in place that documents these decisions and that 
these documents are available upon request, should questions arise. 

ICs are required to document the rationales used when skipping 
applications or funding applications as exceptions. While the ICs are 
not required to routinely provide the NIH OD with this documentation, 
these data are available, upon request, for analysis by the NIH OD for 
assessing compliance or for other purposes. The GAO recommendation 
should be revised accordingly to recognize that the role of the NIH OD 
for this activity is to ensure that documentation procedures are in 
place, not to have input about the specific scientific reasoning about 
skipping applications or funding applications as exceptions. 

GAO implies a role for the OD that is not scientifically appropriate. 
Specific reasons for skips and exceptions must and should rely on the 
judgment of scientific officials who understand the current trends in 
science, as well as the portfolios of the institute. 

2. In response to Page 15, Table 1, "Extramural Research R01 Grant 
Applications Funded in FY03 -FY07", NIH would like to highlight that 
R01 grants awarded outside the payline from FY03 through FY07 were in 
fact a result of the increase in the number of R01 grants awarded to 
new investigators. 

Maintaining a viable research workforce is considered essential to the 
vitality of health-related research. That means new investigators must 
enter the pool of NIH funded Principal Investigators at a reasonable 
rate to replace those who choose to leave or leave because their 
applications are no longer competitive. In some cases, an adequate 
supply of new investigators is dependent on funding applications that 
receive review scores outside the normal funding range, as shown in the 
chart below. It should be pointed out that these applications are still 
well within the range of scores that are considered to be highly 
meritorious. In order to protect the viability of the extramural 
workforce the NIH reaches for additional applications from New 
Investigators. 

During FY07 through FY 2009, the OD designed and implemented policies 
to support new investigators. The policies were designed to reverse the 
steady decline in the number of new investigators that started in FY03. 
Annually, OD has presented guidelines to the Institute Directors. Those 
guidelines arc available at [hyperlink, 
http://grants.nih.gov/gtants/new_investigators/indix.htm]. Each year 
the OD sets New Investigator targets for the ICs and then tracks awards 
to New Investigators during the course of the year. Over the past three 
fiscal years the NIH has reached the established targets. The 
importance of new investigators to the continued success of the NIH 
extramural programs is well understood. This information has been 
clearly articulated in notices that have appeared in the NIH Guide for 
Grants and Contracts and in other NIH publications and presentations. 

NIH created the graph below to illustrate the increase in the raise-to-
pay awards as a result of the increase in R01 awards to new 
investigators. 

Figure: NIH R01 Awards Outside the Pay Line Awarded to New 
Investigators: 

[Refer to PDF for image: line graph] 

Fiscal year: 2003; 
Number of R01 Awards Outside the Pay Line: 625; 
Number of R01 Awards Outside the Pay Line Awarded to New Investigators: 
124. 

Fiscal year: 2004; 
Number of R01 Awards Outside the Pay Line: 528; 
Number of R01 Awards Outside the Pay Line Awarded to New Investigators: 
133. 

Fiscal year: 2005; 
Number of R01 Awards Outside the Pay Line: 572; 
Number of R01 Awards Outside the Pay Line Awarded to New Investigators: 
176. 

Fiscal year: 2006; 
Number of R01 Awards Outside the Pay Line: 620; 
Number of R01 Awards Outside the Pay Line Awarded to New Investigators: 
203. 

Fiscal year: 2007; 
Number of R01 Awards Outside the Pay Line: 1,059; 
Number of R01 Awards Outside the Pay Line Awarded to New Investigators: 
532. 

[End of figure] 

3. NIH would like to review GAO's method and approach for quantifying 
out of order funding. 

GAO Finding: Design of NIH's Travel and Personnel Appointment Processes 
Include Key Control Activities and Some Monitoring Activities but Lacks 
Systemic Risk-Based Monitoring (p.16): 

1. NIH concurs with the finding that the design of NIH's Title 42 
personnel appointment process includes key control activities and some 
monitoring but lacks systemic risk-based monitoring. 

On page 16, GAO states: "The NIH OD has also established policies and 
procedures to help ensure that Title 42 personnel appointment decisions 
are appropriate." The report continues on page 17 to say that "the 
design of the controls included in the NIH-wide processes over travel 
and Title 42 personnel appointments included key controls necessary to 
help ensure these activities were being carried out appropriately, 
except in one key area related to the lack of requirements for risk-
based management." This key area was identified as systemic monitoring 
that "includes risk-based control evaluations." Further the report 
states that while some monitoring was going on, it was neither a part 
of a systemic risk assessment plan and was not performed on an on-going 
basis. 

In regard to the Title 42 personnel appointment process, NIH agrees 
with the GAO finding that at the present, NIH does not have in place a 
systemic monitoring program for risk-based evaluations. While 
evaluations are done, as the report notes, they do not meet the test of 
being part of a "systemic risk assessment plan" or are not performed on 
a consistent and on-going basis. NIH will incorporate risk-based 
monitoring into the Title 42 personnel appointment process. In 
addition, NIH will add the Title 42 personnel appointment process as a 
risk area within the NIH Enterprise Risk Management Program to ensure 
that the risk is monitored and assessed. 

NIH identified and scored the agency travel process as a risk area 
within the NIH Enterprise Risk Management Program. Therefore, travel 
risks are being monitored. NIH will reassess the travel risk levels to 
ensure that they are appropriate. 

Table: The National Institute of Health: Inventory of OD Oversight 
Mechanisms for the GAO Governance and Oversight Review: 

52 items are listed in the inventory [original copy is illegible] 

Refer to PDF for information. 

[End of section] 

Appendix IV: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

Linda T. Kohn (202) 512-7114 or kohnl@gao.gov: 

Susan Ragland (202) 512-8486 or raglands@gao.gov: 

Acknowledgments: 

In addition to the contacts named above, Paul Caban and Jenny Grover, 
Assistant Directors; Jehan Abdel-Gawad; Deyanna Beeler; Francine 
Delvecchio; Patrick Frey; Krister Friday; Natalie Herzog; Cynthia 
Jackson; Kelli Jones; Judy Lee; Lisa Motley; Kara Patton; Will Simerl; 
Jessica Smith; and Matt Zaun made key contributions to this report. 

[End of section] 

Footnotes: 

[1] NIH also supports intramural research, which is performed by NIH 
scientists in NIH laboratories. 

[2] The three centers that do not fund extramural research and do not 
receive separate appropriations (Center for Scientific Review, Center 
for Information Technology, and the Clinical Center) are funded through 
the NIH Management Fund, which is funded using a portion of other NIH 
appropriations. See 42 U.S.C. § 290. 

[3] GAO, NIH Conflict of Interest: Recusal Policies for Senior 
Employees Need Clarification, [hyperlink, 
http://www.gao.gov/products/GAO-07-319] (Washington, D.C.: Apr. 30, 
2007). 

[4] H.R. Rep. No. 110-231, at 161-62 (2007); NIH Office of Management 
Assessment, Management Review: National Institute of Environmental 
Health Sciences (Apr. 9, 2008). 

[5] GAO, Standards for Internal Control in the Federal Government, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] 
(Washington, D.C.: November 1999). 

[6] Under two provisions of title 42 United States Code, NIH has 
additional hiring flexibilities not permitted under title 5 authorities 
related to the general schedule and senior executive service. These 
flexibilities are referred to as "title 42" personnel appointments. 
Specifically, title 42 authorities allow NIH to hire scientists at 
salary levels comparable to those outside of the federal government. In 
2008, under these authorities, NIH could hire scientists with salary 
levels up to $250,000. However, maximum pay for the general schedule 
was $149,000 and for the senior executive service was $172,200 in 2008. 
See 42 U.S.C. § 209 (f),(g). 

[7] Specifically, we reviewed funding decisions made for R01 grants, 
the original grant mechanism used by NIH, which is a common type of 
grant awarded to organizations of all types (universities, colleges, 
small businesses, for-profit, foreign and domestic, etc.) to support a 
discrete, specified project to be performed by a specific investigator 
or group of investigators. 

[8] A walkthrough is a method used to develop an understanding of key 
processes and controls in which an auditor traces a transaction through 
the organization's procedures. 

[9] See [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. 

[10] See table 2 for GAO's framework for effective risk management. GAO 
developed the framework based on authoritative literature and 
standards, as well as previous GAO reports and testimonies. We 
consulted the Government Performance and Results Act (GPRA) of 1993; 
the Government Auditing Standards, 2003 Revision; GAO's Standards for 
Internal Control in the Federal Government (November 1999); guidance 
from the Office of Management and Budget (OMB); the work of the 
President's Commission on Risk Management; consulting papers; and the 
enterprise risk management approach of the Committee of Sponsoring 
Organizations (COSO) of the Treadway Commission. We reviewed numerous 
frameworks from industry, government, and academic sources. GAO, 
"Appendix I: A Risk Management Framework" of Risk Management: Further 
Refinements Needed to Assess Risks and Prioritize Protective Measures 
at Ports and Other Critical Infrastructure, [hyperlink, 
http://www.gao.gov/products/GAO-06-91] (Washington, D.C.: Dec. 15, 
2005). 

[11] The Director of NIH is appointed by the President, with Senate 
confirmation. The President also appoints the director of NCI, while 
the Secretary of the Department of Health and Human Services (HHS) 
appoints the other IC directors. 

[12] See [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. 

[13] See 31 U.S.C. § 3512(c). 

[14] Risk management does not replace, but rather incorporates and 
expands on internal control. Thus, risk management provides a more 
robust and extensive focus to effectively identify, assess, and manage 
risk. 

[15] GAO, High-Risk Series: An Update, [hyperlink, 
http://www.gao.gov/products/GAO-05-207] (Washington, D.C.: January 
2005). 

[16] See table 2. GAO developed the framework based on authoritative 
literature and standards, as well as previous GAO reports and 
testimonies. We consulted the Government Performance and Results Act 
(GPRA) of 1993; the Government Auditing Standards, 2003 Revision; GAO's 
Standards for Internal Control in the Federal Government (November 
1999); guidance from the Office of Management and Budget (OMB); the 
work of the President's Commission on Risk Management; consulting 
papers; and the enterprise risk management approach of the Committee of 
Sponsoring Organizations (COSO) of the Treadway Commission. We reviewed 
numerous frameworks from industry, government, and academic sources. 
GAO, "Appendix I: A Risk Management Framework" of Risk Management: 
Further Refinements Needed to Assess Risks and Prioritize Protective 
Measures at Ports and Other Critical Infrastructure, [hyperlink, 
http://www.gao.gov/products/GAO-06-91] (Washington, D.C.: Dec. 15, 
2005). 

[17] The Committee of Sponsoring Organizations of the Treadway 
Commission, Enterprise Risk Management--Integrated Framework (Jersey 
City, N.J.: American Institute of Certified Public Accountants, 
September 2004) and [hyperlink, 
http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. 

[18] See 42 U.S.C. §§ 282(b)(9); 289a(a); 289a-1(a)(2). The Secretary 
of HHS promulgated regulations expanding on the use of peer review by 
groups appointed by the Director of NIH and the directors of the 
national research institutes. 

[19] The composition of the initial peer review groups is specified in 
42 C.F.R. § 52h.4 (2008). Based on these criteria, NIH staff select the 
initial peer reviewers, who generally agree to participate for 4 years. 

[20] See 42 C.F.R. § 52h.7 (2008). 

[21] 42 C.F.R. § 52h.8 (2008) directs peer review groups to assess each 
proposed research project taking into account the following criteria, 
among other pertinent factors: (a) its significance, (b) the adequacy 
of its approach and methodology, (c) its innovativeness and 
originality, (d) the qualifications and experience of its principal 
investigator and staff, (e) the scientific environment and reasonable 
availability of resources for it, (f) the adequacy of its plans to 
include both genders, minorities, children, and special populations as 
appropriate for its scientific goals, (g) the reasonableness of its 
budget and duration, and (h) the adequacy of its protections for 
humans, animals, and the environment. 

[22] 42 U.S.C. § 284a. Although the law setting forth the requirements 
for advisory councils is specific to institutes, each center that funds 
extramural research has an advisory council substantially similar to 
those of the institutes. See 42 U.S.C. §§ 287a (National Center for 
Research Resources), 287c-21(b) (National Center for Complementary and 
Alternative Medicine), 287c-31(j) (National Center on Minority Health 
and Health Disparities). 

[23] Advisory councils also include ex officio members, who are 
nonvoting. Voting members generally serve 4-year terms. At the NCI, the 
President appoints voting advisory council members, and the members 
serve 6-year terms. For all other advisory councils, the Secretary of 
HHS appoints voting members. 

[24] By law, the advisory councils for NCI and the National Heart, 
Lung, and Blood Institute must meet at least four times per fiscal 
year. 42 U.S.C. § 284a(h)(2). 

[25] NIH may not approve or fund any application unless it has been 
recommended for approval by a majority of the members of the initial 
peer review group and a majority of the voting members of the advisory 
council. The initial peer review groups recommend applications for 
approval via the scoring system. 42 U.S.C. § 289a-1(a)(2). 

[26] The R01 grant is the original grant mechanism used by NIH. This 
type of grant is awarded to organizations of all types (universities, 
colleges, small businesses, for-profit, foreign and domestic, etc.) to 
support a discrete, specified project to be performed by a named 
investigator or investigators. 

[27] See GAO, [hyperlink, 
http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. The Department of 
Health and Human Services, of which NIH is a component, is required to 
establish and maintain an effective system of internal control, 
consistent with the standards prescribed by the Comptroller General. 31 
U.S.C. § 3512(c), (d). 

[28] Examples of positions that would require more involvement from the 
NIH-level offices include senior-level employees such as tenure-track 
investigators, senior investigators, senior scientists, and senior 
clinicians. 

[29] GAO, Department of Health and Human Services: Controls Over Travel 
Program Are Generally Effective, but Some Improvements Are Needed, 
[hyperlink, http://www.gao.gov/products/GAO-03-334] (Washington, D.C.: 
Feb. 21, 2003). 

[30] Travel cards are a type of charge card used for official travel- 
related expenses. 

[31] The Steering Committee, which is chaired by the NIH director and 
composed of 10 IC directors who serve on a rotating basis, is NIH's 
most senior-level governing body. The Steering Committee is responsible 
for addressing NIH-wide issues, other than those that relate to 
science. 

[32] NIH Office of Management Assessment, NIH Enterprise Risk 
Management Program, Enterprise Risk Management Guidebook: A Step-By- 
Step Guide (March 2009, Draft). 

[33] Risk management structure is a segmentation of discrete, mission- 
oriented subsets of an organization to facilitate risk management 
activities at the lower level. 

[34] Scoring risks includes assessing the risk based on likelihood of 
occurrence and impact. Based on the assessment, risks are assigned a 
points value, which allows for quantitative comparison and ranking of 
risks across NIH. 

[35] For travel, we reviewed NIH manual chapters 1500-01: Introduction 
to Official Government Travel (Jan. 5, 2004), 1500-02: Traveler 
Responsibilities (May 13, 2008), and 1500-08: Acceptance of Payment 
from a Nonfederal Source to Cover Travel Expenses [Sponsored Travel] 
(Jan. 23, 2006). For Title 42 personnel appointments, we reviewed Title 
42 Pay Model--NIH (Dec. 21, 2004) and NIH manual chapter 2300-575-2: 
Title 42 Recruitment and Retention Incentives (May 4, 2000). 

[36] GAO, Standards for Internal Control in the Federal Government, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] 
(Washington, D.C.: November 1999). 

[37] NIH Manual Chapter 1750 - Management Control Program (Nov. 15, 
2004). 

[38] NIH Office of Management Assessment, NIH Enterprise Risk 
Management Program, Enterprise Risk Management Guidebook: A Step-By- 
Step Guide (March 2009, Draft). 

[39] See table 2. GAO developed the framework based on authoritative 
literature and standards, as well as previous GAO reports and 
testimonies. We consulted the Government Performance and Results Act 
(GPRA) of 1993; the Government Auditing Standards, 2003 Revision; GAO's 
Standards for Internal Control in the Federal Government (November 
1999); guidance from the Office of Management and Budget (OMB); the 
work of the President's Commission on Risk Management; consulting 
papers; and the enterprise risk management approach of the Committee of 
Sponsoring Organizations (COSO) of the Treadway Commission. We reviewed 
numerous frameworks from industry, government, and academic sources. 
GAO, "Appendix I: A Risk Management Framework" of Risk Management: 
Further Refinements Needed to Assess Risks and Prioritize Protective 
Measures at Ports and Other Critical Infrastructure, [hyperlink, 
http://www.gao.gov/products/GAO-06-91] (Washington, D.C.: Dec. 15, 
2005). 

[40] Prior to 1985, Congress either created ICs itself or gave others 
(e.g., the Surgeon General or the Secretary of HHS) the authority to 
create ICs through individual laws. Since 1985, the Secretary of HHS 
has had the authority to establish, reorganize, or abolish ICs. Pub. L. 
No. 99-158, 99 Stat. 820 (1985). 

[41] NIH Enterprise Risk Management Guidebook, page 6. 

[42] Ibid. 

[43] NIH Enterprise Risk Management Guidebook, page 25. 

[44] NIH website, [hyperlink, http://www.nih.gov/about]. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: