This is the accessible text file for GAO report number GAO-14-55 
entitled 'VA Health Care: Improvements Needed in Processes Used to 
Address Providers' Actions That Contribute to Adverse Events' which 
was released on December 3, 2013. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

VA Health Care: 

Improvements Needed in Processes Used to Address Providers' Actions 
That Contribute to Adverse Events: 

VA Adverse Events: 

Report to Congressional Requesters: 

December 2013: 

GAO-14-55: 

GAO Highlights: 

Highlights of GAO-14-55, a report to congressional requesters. 

Why GAO Did This Study: 

Adverse events—-clinical incidents that may pose the risk of injury to 
a patient as the result of a medical intervention rather than the 
patient’s underlying health condition—-can occur in all health care 
delivery settings. VAMCs can use one or more of the protected 
(confidential and nonpunitive) and nonprotected processes to evaluate 
the role of individual providers in adverse events. GAO was asked to 
review the extent to which processes used to respond to adverse events 
are carried out across VAMCs. In this report, GAO examined (1) VAMCs’ 
adherence to VHA’s protected peer review process, and the extent to 
which VHA monitors this process, and (2) VAMCs’ adherence to VHA’s 
nonprotected processes and the extent to which VHA monitors these 
processes. To conduct this work, GAO visited four VAMCs selected for 
variation in size, complexity of surgeries typically performed, and 
location. GAO reviewed VHA policies and federal internal control 
standards and analyzed data from the four selected VAMCs. GAO also 
interviewed VHA and VA OIG officials, as well as officials from VISNs 
of the four selected VAMCs. 

What GAO Found: 

The Department of Veterans Affairs (VA) medical centers did not adhere 
to certain policy elements of the protected peer review process, and 
monitoring by VA’s Veterans Health Administration (VHA) is limited. 
According to policy issued by VHA, protected peer review may be used 
by VA medical centers (VAMC) when there is a need to determine whether 
a provider’s actions associated with an adverse event were clinically 
appropriate—that is, whether another provider with similar expertise 
would have taken similar action. Despite VAMC officials’ general 
understanding of the protected peer review process, none of the VAMCs 
GAO visited adhered to all four protected peer review policy elements 
selected for review, including the timely completion of reviews, and 
the timely development of peer review triggers that signal the need 
for further review of a provider’s care. Failure of VAMCs to adhere 
to the protected peer review policy elements may result in missed 
opportunities to identify providers who pose a risk to patient safety. 
Veterans Integrated Service Networks (VISN), responsible for oversight 
of VAMCs, monitor VAMCs’ protected peer review processes through 
quarterly data submissions and annual site visits. A VHA official said 
that VHA monitors the process by reviewing and analyzing the 
aggregated quarterly data submitted by VAMCs through the VISNs. The VA 
Office of the Inspector General (OIG) also conducts oversight of the 
protected peer review process as part of a larger review of VAMCs’ 
operations. While the VISNs and VA OIG have reviewed VAMCs 
establishment of peer review triggers to prompt further review of a 
provider‘s care, neither they nor VHA has monitored their 
implementation. As such, VHA cannot provide reasonable assurance that 
VAMCs are using the peer review triggers as intended, as a risk 
assessment tool. This weakens VAMCs’ ability to ensure they are 
identifying providers that are unable to deliver safe, quality patient 
care. 

VAMCs’ adherence to the nonprotected focused professional practice 
evaluation (FPPE) process is unclear due to gaps in VHA’s policy on 
documentation requirements, and VHA does not routinely monitor 
nonprotected processes. An FPPE for cause is a time-limited evaluation 
during which the VAMC assesses the provider’s professional competence 
when a question arises regarding the provider’s ability to provide 
safe, quality patient care. Information collected through the FPPE can 
be used to inform adverse actions, such as limiting the provider’s 
scope of care. Although VAMC officials were generally aware of the 
FPPE process, there are gaps in VHA’s policy regarding how these 
evaluations should be documented and what information should be 
included, which limited GAO's ability to assess VAMCs’ adherence to the 
process. For example, one VAMC provided GAO with documentation labeled 
as an FPPE and identified by the service chief as an FPPE; however, 
the quality manager said a formal FPPE was not conducted and that the 
documentation was actually part of a protected peer review. These 
differing views illustrate that, even within the same facility, gaps 
in VHA’s policy on documenting FPPEs create a lack of clarity and 
opportunities for misinterpretation and inappropriate use. Moreover, 
the gaps in VHA’s policy may hinder VAMCs’ ability to appropriately 
document the evaluation of a provider’s skills, support any actions 
initiated, and track provider-specific incidents over time. There is 
no routine monitoring of FPPEs for cause by VHA, VISNs, or VA OIG. 

What GAO Recommends: 

GAO recommends that VA take action to ensure VAMCs adhere to certain 
elements of the peer review policy, require VAMCs to report data on 
implementation of protected peer review triggers, and develop more 
specific policy to help guide the FPPE process, including 
documentation requirements. In its written comments, VA generally 
concurred with GAO’s conclusions and recommendations. 

View GAO-14-55. For more information, contact Debra Draper at (202) 
512-7114 or draperd@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

VAMCs Did Not Adhere to Certain Policy Elements of the Protected Peer 
Review Process, and Monitoring by VHA Is Limited: 

VAMC's Adherence to the FPPE Process Is Unclear Due to Gaps in Policy 
Addressing Documentation Requirements; VHA Does Not Routinely Monitor 
Nonprotected Processes: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Example of a VA Medical Center's (VAMC) Focused 
Professional Practice Evaluation (FPPE) Template: 

Appendix II: Comments from the Department of Veterans Affairs: 

Appendix III: GAO Contacts and Staff Acknowledgments: 

Tables: 

Table 1: VA Site Visit Locations and Characteristics: 

Table 2: Adherence to Selected Protected Peer Review Policy Elements 
at Four VA Medical Centers (VAMC) GAO Visited, Fiscal Years 2009 
through 2011: 

Table 3: Date Established and Description of Peer Review Triggers at 
Four VA Medical Centers (VAMC) GAO Visited: 

Table 4: Protected Peer Review Process Monitoring Activities by 
Selected VA and Veterans Health Administration (VHA) Organizational 
Components: 

Figures: 

Figure 1: Example of Decision Process Used by VA Medical Center (VAMC) 
Officials When Responding to an Adverse Event: 

Figure 2: VA and Veterans Health Administration (VHA) Organizational 
Components Involved in Monitoring VA Medical Centers' (VAMC) Adverse 
Events and Related Processes: 

Abbreviations: 

ADUSH: Assistant Deputy Under Secretary for Health: 

AIB: administrative investigation board: 

DUSHOM: Deputy Under Secretary for Health for Operations and 
Management: 

FPPE: focused professional practice evaluation: 

OIG: Office of Inspector General: 

VA: Department of Veterans Affairs: 

VAMC: VA medical center: 

VHA: Veterans Health Administration: 

VISN: Veterans Integrated Service Network: 

[End of section] 

GAO:
United States Government Accountability Office: 
441 G St. N.W. 
Washington, DC 20548: 

December 3, 2013: 

The Honorable Patty Murray: 
Chairman: 
Committee on the Budget: 
United States Senate: 

The Honorable Bernard Sanders: 
Chairman: 
Committee on Veterans' Affairs: 
United States Senate: 

The Honorable Michael Michaud: 
Ranking Member: 
Committee on Veterans' Affairs: 
House of Representatives: 

The Honorable Eddie Bernice Johnson: 
House of Representatives: 

The Department of Veterans Affairs' (VA) Veterans Health 
Administration (VHA) operates one of the largest health care delivery 
systems in the United States. VHA's health care system includes 152 VA 
medical centers (VAMC) that offer a variety of inpatient and 
outpatient services, ranging from routine examinations to complex 
surgical procedures. According to VHA officials, these services are 
provided by a variety of health care professionals, including over 
39,000 physicians and dentists. As in all health care delivery 
settings like VAMCs, adverse events--clinical incidents that may pose 
the risk of injury to a patient as the result of a medical 
intervention or the lack of an appropriate intervention, such as a 
missed or delayed diagnosis, rather than that patient's underlying 
medical condition--may occur, resulting in potential or actual harm to 
patients.[Footnote 1] In an analysis of VHA data published in 2011, 
for example, researchers found that there were 101 reported surgical 
adverse events--events pertaining to incorrect surgical procedures, 
such as a procedure performed on the wrong patient or with the wrong 
implant--across VAMCs from mid-2006 to 2009.[Footnote 2] 

VHA requires that VAMCs take appropriate action to report and evaluate 
adverse events.[Footnote 3] VHA generally gives VAMCs discretion in 
choosing which process to use in responding to reported adverse 
events. VAMCs generally use one or more of the following three 
processes to evaluate the role of individual providers in adverse 
events: peer review for quality management, focused professional 
practice evaluation (FPPE), and administrative investigation board 
(AIB).[Footnote 4] Peer review for quality management is used when 
there is a need to determine whether a provider's actions associated 
with an adverse event were clinically appropriate--that is, whether 
another provider with similar expertise would have taken similar 
action. An FPPE is a process whereby the VAMC evaluates a provider's 
competence in a specific area of clinical expertise, such as a 
surgical procedure, when a question arises regarding the provider's 
ability to provide safe, quality patient care.[Footnote 5] An AIB can 
be used to investigate whether an adverse event was the result of a 
provider's professional misconduct or from potential system 
deficiencies related to VHA policies or procedures.[Footnote 6] 

In evaluating the role of providers in adverse events, VHA emphasizes 
the use of protected processes that are confidential and nonpunitive, 
which they believe better facilitate quality management efforts at the 
individual provider level. The peer review for quality management is a 
protected process.[Footnote 7] The information collected is 
confidential and protected under federal law, and therefore cannot be 
used to inform adverse actions against providers, such as limiting a 
provider's clinical privileges.[Footnote 8] However, according to VHA 
officials, if a provider's ability to provide safe, quality patient 
care is in question, then VAMCs may use a nonprotected process to 
address such concerns.[Footnote 9] FPPEs and AIBs are nonprotected 
processes, and, therefore, the information collected through these 
processes can be used to inform adverse actions against providers. 
When warranted on the basis of the outcome of these types of 
processes, VAMC officials may take a variety of actions, such as 
training and proctoring a provider, major adverse actions, 
disciplinary actions, or privileging actions.[Footnote 10] 

Adverse events raise questions about the quality of care VAMCs provide 
to veterans. You asked us to review any gaps in the processes that 
VAMCs use for responding to adverse events and the extent to which the 
processes are carried out across VHA's health care system. In this 
report, we examined: 

1. VAMCs' adherence to VHA's protected peer review process and the 
extent to which VHA monitors this process, and: 

2. VAMCs' adherence to VHA's nonprotected processes and the extent to 
which VHA monitors these processes. 

To examine VAMCs' adherence to protected and nonprotected processes, 
we reviewed VHA's policies on protected peer review and FPPEs, and 
VA's policy on AIBs.[Footnote 11] Additionally, we reviewed these 
processes in the context of federal internal control standards for 
risk assessment, information and communications, control activities, 
and monitoring.[Footnote 12] We also visited four VAMCs, which were 
selected on the basis of geographic variation using the location in 
one of VA's 21 Veterans Integrated Service Networks (VISN) as a proxy, 
VAMC size based on the number of unique patients served, and level of 
surgical complexity (see table 1).[Footnote 13] 

Table 1: VA Site Visit Locations and Characteristics: 

VA medical center (VAMC) location: Dallas, Texas; 
Veterans Integrated Service Network (VISN): 17; 
VAMC size[A]: 112,000; 
Surgical complexity rating[B]: Complex. 

VA medical center (VAMC) location: Nashville, Tennessee; 
Veterans Integrated Service Network (VISN): 9; 
VAMC size[A]: 84,000; 
Surgical complexity rating[B]: Complex. 

VA medical center (VAMC) location: Seattle, Washington; 
Veterans Integrated Service Network (VISN): 20; 
VAMC size[A]: 87,000; 
Surgical complexity rating[B]: Complex. 

VA medical center (VAMC) location: Augusta, Maine; 
Veterans Integrated Service Network (VISN): 1; 
VAMC size[A]: 40,000; 
Surgical complexity rating[B]: Intermediate. 

Source: GAO analysis of Veterans Health Administration (VHA) 
information. 

[A] Number of unique patients rounded to the nearest thousand. 

[B] VHA assigns each VAMC a surgical complexity rating of standard, 
intermediate, or complex. This classification system uses multiple 
variables to measure surgical complexity arrayed along four 
categories: facilities, equipment, workload, and staffing. These 
ratings were implemented May 7, 2010. 

[End of table] 

We also requested data from each VAMC on protected peer reviews, 
FPPEs, and AIBs from fiscal years 2009 through 2011 in order to review 
the four VAMCs' adherence to the applicable policies and within the 
context of relevant federal internal control standards.[Footnote 14] 
Specifically, we reviewed VAMCs' adherence to several elements of 
VHA's protected peer review policy for which VAMCs are required to 
collect data;[Footnote 15] these selected elements include timeliness 
of peer review completion and committee review of select peer review 
results. VHA requires VAMCs to establish peer review or professional 
activity triggers that signal the need to conduct a detailed 
assessment of a provider's care, which can then lead to an FPPE. 
[Footnote 16] We reviewed only whether VAMCs established peer review 
triggers, since VAMCs collect peer review data.[Footnote 17] To assess 
the reliability of the protected peer review data we analyzed, we 
reviewed VHA's protected peer review policy on data reporting 
requirements, interviewed VHA and VAMC officials familiar with the 
data, and conducted checks on the data, looking for missing values, 
outliers, or other anomalies. We found some records that contained 
missing, incomplete, or inaccurate data and excluded those data from 
our analysis. We determined that the deletions had no material effect 
on our analysis and that the data, as analyzed, were sufficiently 
reliable for our purposes. 

We examined VAMCs' adherence to VHA's FPPE process by reviewing VAMCs' 
available FPPE documentation for clarity and ease of tracking what 
steps the VAMC had taken to conduct the FPPEs from initiation to 
completion. We reviewed VA's policy on AIBs and interviewed VAMC 
officials about their use of AIBs to investigate providers involved in 
adverse events. We also reviewed previous GAO work on VA's AIB 
process, including the extent to which VA collected data on AIBs and 
how VA used the results of AIB investigations.[Footnote 18] We were 
unable to analyze VAMCs' implementation of certain elements of VA's 
AIB policy because at the time of our review VAMCs had not retained 
all of the documentation necessary for us to conduct a review for 
processes conducted from fiscal years 2009 through 2011. According to 
the AIB policy,[Footnote 19] the investigation files need to be 
retained in accordance with applicable records retention schedules 
and, at a minimum, until any corrective action is completed and the 
time frame for any claims or appeals has passed; we did not assess how 
long VAMCs should have kept each of the investigation files.[Footnote 
20] 

To examine the extent to which VHA monitors VAMCs' use of protected 
and nonprotected processes to respond to adverse events, we 
interviewed VHA officials from the Office of the Assistant Deputy 
Under Secretary for Health (ADUSH) for Quality, Safety and Value, 
including the Director of Risk Management and the Director of 
Credentialing and Privileging; the Office of the Deputy Under 
Secretary for Health for Operations and Management (DUSHOM); and the 
Office of the Medical Inspector. We also interviewed officials from 
the VA Office of Inspector General (OIG) Office of Healthcare 
Inspections, as well as the chief medical officers of the four VISNs 
for the VAMCs we visited. 

We limited our review to describing VHA's protected and nonprotected 
processes for responding to adverse events that involve individual 
providers, excluding from our scope processes, such as root cause 
analysis,[Footnote 21] that focus primarily on events that involve 
systems or processes issues. We focused our review on VHA physicians 
and dentists, hereafter referred to as providers, because these are 
the primary types of licensed independent providers at VAMCs. The 
findings of our four site visits cannot be generalized to other VAMCs. 

We conducted this performance audit from July 2012 to December 2013 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

Background: 

VAMCs conduct an initial review of cases that are identified as 
possible adverse events to determine how best to respond and which 
process to use to determine the facts of the case, such as protected 
peer review, FPPE, or AIB. Because VAMCs generally have discretion in 
which of these processes they choose to use to respond to an adverse 
event, different VAMCs may choose different processes in response to 
experiencing similar adverse events. Based on the nature of the 
adverse event and the information gleaned through a particular review 
process, a VAMC may decide to conduct multiple types of reviews, both 
protected and nonprotected processes, as appropriate. Information 
collected through protected review processes, including protected peer 
review, cannot be used to inform adverse actions against a provider; 
information collected through nonprotected processes, including FPPEs 
and AIBs, can be used to support a VAMC's decision to take adverse 
action against a provider. According to VHA policy, VAMCs can use both 
protected and nonprotected processes concurrently or consecutively as 
long as protected and nonprotected processes and data collection are 
kept separate. According to VHA officials, if a VAMC is using a 
protected process to review an event and realizes that a nonprotected 
review may be necessary, the protected process should be stopped and 
the VAMC should start a nonprotected review. See figure 1 for an 
illustration of the decision process a VAMC official might use when 
deciding how to respond to an adverse event. 

Figure 1: Example of Decision Process Used by VA Medical Center (VAMC) 
Officials When Responding to an Adverse Event: 

[Refer to PDF for image: process map] 

1. Identify potential adverse event. 

2. Preliminary review of adverse event by VAMC officials. 

3. Determine if adverse event requires further review: 
No: No further review; 
Yes: continue. 

4. Choose type of review process: 
* Protected: 
Findings requiring further review; or: 
Information cannot be used to inform adverse action against a provider; 
* Nonprotected: 
Findings requiring further review; or: 
Information can be used to inform adverse action against a provider. 

Source: GAO. 

[End of figure] 

Protected Process: 

According to VHA's protected peer review policy, peer review is 
required under certain circumstances, such as a death that appears 
related to a hospital-incurred incident or a complication from 
treatment and a suicide within 30 days of a clinical encounter with a 
VA health care professional.[Footnote 22] Peer review may be 
considered in other circumstances, such as when there is an unexpected 
or negative outcome. Once VAMC officials decide to conduct a protected 
peer review, a peer reviewer is assigned to evaluate the care 
delivered and the actions taken by the provider.[Footnote 23] The peer 
reviewer makes an initial determination of whether the provider should 
have taken different action when providing patient care and 
preliminarily assigns one of the following three levels of care: 

* Level of care 1 - the most experienced, competent providers would 
have managed the case in a similar manner; 

* Level of care 2 - the most experienced, competent providers might 
have managed the case differently; or: 

* Level of care 3 - the most experienced, competent providers would 
have managed the case differently. 

According to VHA's peer review policy, the initial peer review should 
be completed within 45 calendar days from determination of the need 
for peer review.[Footnote 24] If the peer-reviewed case is assigned a 
level of care 2 or 3, it must be referred to the VAMC's peer review 
committee for further review.[Footnote 25] After conducting a further 
review of the facts of the case, and receiving further input from the 
provider under review, the peer review committee either validates the 
initial level of care or assigns a higher or lower level of care. The 
peer review committee's level of care rating is final and must be 
completed within 120 calendar days from determination of the need for 
peer review. The peer review committee can also make recommendations 
for nonpunitive, nondisciplinary actions, as appropriate, such as 
reviewing and revising local policy, to improve the quality of care 
delivered. The final level of care rating and any recommendations for 
improvement are reported to the provider's supervisor, who gives the 
provider feedback that is based on the peer review committee's 
findings. According to VHA officials, VAMCs conduct approximately 
23,000 protected peer reviews systemwide annually. 

VHA also has a contract with an external organization that is used to 
audit protected peer review.[Footnote 26] VAMCs may request external 
protected peer review expertise if there are no qualified peers 
available at the VAMC. According to a VISN official, external review 
may also be requested if the VAMC needs to ensure an independent peer 
review, for example, if all the providers in the same clinical 
specialty were involved in the event. VHA also requires each VAMC to 
submit quarterly a sample of cases that were recently peer-reviewed, 
for a secondary peer review by this external organization.[Footnote 27] 

According to VHA's protected peer review policy, each VAMC is required 
to develop peer review or professional activity triggers to signal the 
need for further assessment of a provider's clinical care.[Footnote 
28] The triggers are specific to an individual VAMC. If, after a 
detailed assessment, concerns arise about a provider's ability to 
deliver safe, quality patient care, then an FPPE would be conducted. 
[Footnote 29] For example, if a provider meets a VAMC's peer review 
triggers by receiving three peer review level of care ratings of 3--
meaning that the most experienced, competent providers would have 
managed the cases differently--within a 12-month period, then VAMC 
officials would be prompted to conduct a detailed assessment of the 
provider's care, and address concerns about the provider's ability to 
deliver safe, quality patient care by conducting an FPPE. 

Nonprotected Processes: 

According to VHA policy, an FPPE may be used when a question arises 
regarding a provider's ability to provide safe, quality patient care, 
such as whether documentation of patient encounters is late or 
insufficient, or whether diagnoses are accurate. An FPPE may also be 
used when a provider meets or exceeds a VAMC's protected peer review 
triggers and a detailed assessment indicates the need for further 
review. According to VHA and VAMC officials, an FPPE begins by 
providing an opportunity for a provider to improve his or her 
performance in area(s) of identified concern. The FPPE is a time-
limited period during which medical staff leadership assesses the 
provider's professional performance and ability to improve. According 
to VAMC officials, if the provider's performance improves, VAMC 
officials may decide that the FPPE process is completed and the 
provider may be returned to routine monitoring. If a provider's 
performance does not improve, then medical staff leaders may decide to 
continue the FPPE for an additional period of time or determine if a 
privileging action should be taken, such as reducing or revoking a 
provider's privileges.[Footnote 30] The FPPE may include a review of 
the provider's care, either through medical record review, direct 
observation, or discussions with other individuals involved in the 
care of patients. FPPEs conducted when a question arises regarding a 
provider's ability to provide safe, quality patient care are not 
conducted often, according to VHA and VAMC officials. According to a 
VHA official, about 100 FPPEs of this kind are conducted VHA-wide each 
year.[Footnote 31] 

VA typically uses AIBs to examine nonclinical issues for which the 
facts are in dispute, such as allegations of employee misconduct, 
according to VHA and VAMC officials. These officials also said VAMCs 
may use AIBs to investigate issues of an individual provider's 
clinical competence, but this is not typical.[Footnote 32] According 
to VA's AIB policy, the convening authority--typically the VAMC 
director--appoints members to an AIB and defines the scope and 
authority of the investigation.[Footnote 33] The AIB collects and 
analyzes evidence and develops a report, including findings and 
conclusions. The VAMC director may use an AIB's findings to inform 
decisions of whether to take adverse action against a provider and, if 
so, what type of action to pursue. In our 2012 report on AIBs, we 
found that VAMCs and VISNs conducted a total of 1,136 investigations 
nationwide from fiscal year 2009 through 2011, but because VA does 
not track the types of matters investigated, it is unclear how many of 
these were related to clinical competence.[Footnote 34] 

VA and VHA Monitoring Responsibilities: 

There are several VA and VHA organizational components that are 
involved in monitoring VAMCs' adverse events and related processes. 
VHA's Office of the ADUSH for Quality, Safety and Value is responsible 
for establishing VHA policies for protected peer review and FPPEs and 
for providing guidance to VAMCs on using those processes. VHA's Office 
of the DUSHOM oversees the VISNs and provides the VISNs broad and 
general operational direction and guidance; VISN directors are tasked 
with oversight of the protected peer review process. The Office of the 
Medical Inspector addresses health care problems to monitor and 
improve the quality of care provided by VHA; veterans may report 
problems with medical care received at VAMCs directly to the Office of 
the Medical Inspector. The VA OIG Office of Healthcare Inspections 
inspects individual health care issues and performs quality program 
assistance reviews of VAMC operations. See figure 2 for a simplified 
organizational chart of the relationship among these entities. 

Figure 2: VA and Veterans Health Administration (VHA) Organizational 
Components Involved in Monitoring VA Medical Centers' (VAMC) Adverse 
Events and Related Processes: 

[Refer to PDF for image: organizational chart] 

Top level: Secretary of the Department of Veterans Affairs (VA). 

Second level: 
Under Secretary for Health: Veterans Health Administration (VHA); 
Inspector General (OIG). 

Third level, reporting to Under Secretary for Health: Veterans Health 
Administration (VHA): 
* Principal Deputy Under Secretary for Health; 
* Medical Inspector. 

Third level, reporting to Inspector General (OIG): 
* Assistant Inspector General for Healthcare Inspections. 

Fourth level, reporting to Principal Deputy Under Secretary for Health: 
* Deputy Under Secretary for Health for Operations and Management 
(DUSHOM): 
- Veterans Integrated Service Network (VISN) Directors; 
* Assistant Deputy Under Secretary for Health (ADUSH) for Quality, 
Safety and Value: 
- Director of Safety and Risk Awareness: Director of Risk Management; 
Director of Credentialing and Privileging. 

Source: GAO analysis of VA data. 

[End of figure] 

VAMCs Did Not Adhere to Certain Policy Elements of the Protected Peer 
Review Process, and Monitoring by VHA Is Limited: 

VAMCs Did Not Adhere to Certain Policy Elements of the Protected Peer 
Review Process: 

According to VHA's protected peer review policy, and supported by 
federal internal control standards for risk assessment and information 
and communications, VHA's protected peer review policy requirements 
should ensure that identified patient safety risks are mitigated and 
lead to organizational improvements and optimal patient outcomes. 
Additionally, federal internal control standards state that agencies 
should have reliable information relating to internal events to 
effectively run and control their operations; this information should 
be identified, captured, and communicated in sufficient detail and at 
the appropriate time to the right people. 

VAMC officials from all four sites we visited demonstrated a general 
understanding of the process as described in the protected peer review 
policy. For example, officials from each of the four VAMCs knew that 
peer review was a protected nonpunitive process and officials from 
three of the four VAMCs were able to describe the steps of the 
process. However, our analysis of peer review data for fiscal years 
2009 through 2011 provided by the four VAMCs showed that none of the 
VAMCs adhered to all four VHA protected peer review policy elements 
selected for review: (1) completing the initial peer review within 45 
calendar days, (2) completing the final peer review within 120 
calendar days, (3) sending all initial level of care 2 and 3 peer 
reviews to the peer review committee, and (4) developing protected 
peer review triggers. Additionally, these peer review data included 
varying amounts of missing and inaccurate data, which affected our 
ability to fully analyze these data. See table 2 for a summary of the 
four VAMCs' adherence to selected protected peer review policy 
elements. 

Table 2: Adherence to Selected Protected Peer Review Policy Elements 
at Four VA Medical Centers (VAMC) GAO Visited, Fiscal Years 2009 
through 2011: 

VAMC: A; 
Total number of protected peer reviews[A]: 262; 
Percentage of initial peer reviews completed within 45 calendar days: 
77; 
Percentage of final peer reviews completed within 120 calendar days: 
89; 
Percentage of initial peer reviews rated level of care 2 and 3 sent to 
peer review committee: 100. 

VAMC: B; 
Total number of protected peer reviews[A]: 354[B]; 
Percentage of initial peer reviews completed within 45 calendar days: 
100; 
Percentage of final peer reviews completed within 120 calendar days: 
97[C]; 
Percentage of initial peer reviews rated level of care 2 and 3 sent to 
peer review committee: 99. 

VAMC: C; 
Total number of protected peer reviews[A]: 1,000; 
Percentage of initial peer reviews completed within 45 calendar days: 
Cannot determine[D]; 
Percentage of final peer reviews completed within 120 calendar days: 
Cannot determine[D]; 
Percentage of initial peer reviews rated level of care 2 and 3 sent to 
peer review committee: 79[E]. 

VAMC: D; 
Total number of protected peer reviews[A]: 307; 
Percentage of initial peer reviews completed within 45 calendar days: 
79; 
Percentage of final peer reviews completed within 120 calendar days: 
89; 
Percentage of initial peer reviews rated level of care 2 and 3 sent to 
peer review committee: 96. 

Source: GAO analysis of Veterans Health Administration (VHA) data. 

Note: Peer review level of care 2 means that the most experienced, 
competent provider might have managed the case differently; peer 
review level of care 3 means that the most experienced, competent 
provider would have managed the case differently. 

[A] These totals do not include peer-reviewed cases for which the date 
the case was initially sent to the peer reviewer was missing from 
VHA's data. The total number of protected peer reviews for each 
facility does not reflect the numbers that were used to determine 
adherence for the three policy elements listed in this table. Each 
percentage calculation used a different denominator because of 
variations in the quality of the data sets across each of the policy 
elements and fiscal years. 

[B] Includes the number of protected peer reviews conducted in fiscal 
years 2010 and 2011 only, because the data for fiscal year 2009 were 
missing the date the peer review was sent and received by the peer 
reviewer. 

[C] Includes data from fiscal years 2010 and 2011 only. We could not 
determine adherence to this policy element because the data provided 
by this VAMC for fiscal year 2009 were missing the date the peer 
review was sent and received by the peer reviewer. 

[D] The peer review data provided by this VAMC for fiscal years 2009 
through 2011 did not include the date the case was sent for initial 
peer review; it only included the date of the initial review. As a 
result, we could not determine whether initial peer reviews were 
completed within 45 calendar days or whether final peer reviews were 
completed within 120 calendar days for fiscal years 2009 through 2011. 

[E] To determine the percentage of initial peer reviews rated level of 
care 2 and 3 sent to the peer review committee, we used evidence of a 
final peer review level as a proxy that a case was sent to the peer 
review committee for further review. 

[End of table] 

Completing initial peer reviews within 45 calendar days. In our 
analysis of the timeliness policy elements for fiscal years 2009 
through 2011, we found that VAMCs A and D completed 77 to 79 percent 
of initial peer reviews within 45 calendar days. VAMC B completed 100 
percent of initial peer reviews within 45 calendar days for the 2 
fiscal years for which data were available--2010 and 2011. We could 
not determine the completion percentage for VAMC C because the data 
provided by the facility did not contain all of the information 
needed. According to a VAMC C official, the peer review data provided 
to us was compiled just before our site visit and was based on a 
review of past records; the missing data elements could not be located 
during that review. Officials from VAMCs A and D told us that it was 
difficult to get peer reviewers to abide by the 45-day rule, and, 
according to officials from VAMC D, the heavy workload of those in 
charge of peer review tracking during our study period affected their 
ability to ensure peer reviewers' timely completion. According to the 
official in charge of peer review tracking at VAMC D, a routine 
process is now in place at this facility to remind peer reviewers to 
complete their reviews after 30 of the 45 calendar days allotted have 
elapsed. 

Completing the final peer reviews within 120 calendar days. VAMCs A 
and D each completed 89 percent of final peer reviews within 120 
calendar days for fiscal years 2009 through 2011; VAMC B completed 97 
percent for fiscal years 2010 through 2011.[Footnote 35] As previously 
noted, we were unable to determine the completion percentage for VAMC 
C. An official at VAMC A told us that, in addition to the delays in 
completing the initial peer reviews within 45 calendar days, the peer 
review committee might not have always been able to review all peer-
reviewed cases at its monthly meeting if there had been a particularly 
large number of cases sent to the committee that month. 

Sending level of care 2 and 3 peer reviews to the peer review 
committee. We found that VAMCs A and D sent 96 to 100 percent of their 
initial level of care 2 and 3 peer reviews to the peer review 
committee for fiscal years 2009 through 2011; VAMC B sent 99 percent 
for fiscal years 2010 through 2011.[Footnote 36] VAMC C sent 79 
percent of its initial level of care 2 and 3 peer reviews to the peer 
review committee. Officials from VAMC C told us that a possible reason 
for the low adherence rate could be that the peer review committee and 
one of the VAMC's service line committees tasked with sending level of 
care 2 and 3 cases to the peer review committee for further review did 
not communicate well during this time period. According to these 
officials, communication and coordination between these two committees 
has recently improved. 

Developing protected peer review triggers. We found that some VAMCs 
did not develop peer review triggers in a timely way. VHA's protected 
peer review policy issued in January 2008 required that VAMCs develop 
criteria that may define the need for further review or 
action.[Footnote 37] VHA's 2010 update of this policy specified that 
VAMCs were required to establish peer review or professional activity 
triggers. The 2010 policy update provides one example of what these 
triggers could include--three peer review level of care 3 ratings 
for a provider within 12 consecutive months. Further, federal internal 
control standards for risk assessment state that management should 
identify internal risks and undertake a thorough and complete analysis 
of the possible effects. Additionally, the standards state that risk 
assessment should include establishment of criteria for determining 
low, medium, and high risk levels. 

We found that while all four VAMCs we visited developed peer review 
triggers, two of the four VAMCs developed peer review triggers 
approximately two years after the updated policy was issued. (See 
table 3.) 

Table 3: Date Established and Description of Peer Review Triggers at 
Four VA Medical Centers (VAMC) GAO Visited: 

VAMC: A; 
Date trigger established: May 2012; 
Description of trigger: Three level of care 3s within 12 consecutive 
months. 

VAMC: B; 
Date trigger established: July 2008; 
Description of trigger: Two level of care 3s or four level of care 2s 
within 12 consecutive months. 

VAMC: C; 
Date trigger established: August 2012; 
Description of trigger: Two level of care 3s or three level of care 2s 
within 12 consecutive months. 

VAMC: D; 
Date trigger established: December 2010; 
Description of trigger: Two level of care 3s or three level of care 2s 
within 6 consecutive months. 

Source: GAO analysis of VHA data. 

Note: Peer review level of care 2 indicates that the most experienced, 
competent provider might have managed the case differently. Peer 
review level of care 3 indicates that the most experienced, competent 
provider would have managed the case differently. 

[End of table] 

According to some VAMC officials we spoke with, the delay in 
developing peer review triggers was the result, at least in part, of a 
lack of guidance from VHA. When asked about peer review triggers, 
officials from VAMCs B and C told us they did not receive any 
assistance from VHA in developing triggers and that the guidance 
provided by VHA was vague. Officials from VAMCs A and C, the two VAMCs 
that did not initially develop peer review triggers, told us that 
while they had informal triggers that could prompt the need for a 
nonprotected review, such as an FPPE, they had not formally documented 
the triggers. 

When VAMCs fail to complete peer reviews in a timely manner and send 
all level of care 2 and 3 initial peer reviews to the peer review 
committee, they put patients' safety at risk through potential 
exposure to substandard care. VAMCs may fail to identify problematic 
providers in a timely manner and take the appropriate actions. 
Additionally, by not submitting initial level of care 2 and 3 peer 
reviews to the peer review committee for further evaluation, VAMCs are 
not ensuring that the initial ratings assigned were appropriate. 
Moreover, the delayed establishment of the peer review triggers by 
some VAMCs may have resulted in missed opportunities to identify 
providers who posed a risk to patient safety and to conduct an FPPE, 
which would have allowed any warranted action to be taken against the 
provider. 

VHA and VA OIG Monitor VAMCs' Protected Peer Review Process: 

Within VHA, the VISNs and the Office of Risk Management monitor VAMCs' 
protected peer review processes. Officials from the VISNs that oversee 
the four VAMCs we visited told us they monitor VAMCs' protected peer 
review processes through quarterly data monitoring and annual site 
visits, as required by VHA's protected peer review policy.[Footnote 
38] Officials from these VISNs said they monitor peer review data that 
VAMCs are required to submit quarterly.[Footnote 39] Data elements 
that are to be submitted by VAMCs and reviewed by the VISN include, 
but are not limited to, the number of peer reviews completed, the 
assigned level of care ratings by the initial peer reviewer and by the 
peer review committee, the number of assigned level of care ratings 
changed to a higher or lower level by the peer review committee, and 
the timeliness of the reviews. After completing their review of VAMCs' 
quarterly data, VISN officials are to send aggregated peer review data 
to VHA's office of the ADUSH for Quality, Safety and Value through a 
shared electronic database. 

In addition to quarterly review of the data, officials from all four 
VISNs said they conduct annual site visits to the VAMCs within the 
VISN; these site visits include a multifaceted review of VAMCs' 
quality management operations. However, we did not review 
documentation from the VISNs' site visits; one VISN official told us 
that the VISN does not keep formal records and that they are required 
only to attest to their completion. The scope of VISNs' annual site 
visits covers a broad variety of topics, including VAMCs' protected 
peer review processes, according to VISN officials. All VISN officials 
we spoke with told us they typically chose which elements of the peer 
review process to review based on the focus of recent inspections by 
other entities, such as The Joint Commission and the VA OIG.[Footnote 
40] For example, officials we spoke with from one of the four VISNs we 
spoke with said they reviewed in a 2012 site visit whether the VAMCs 
implemented the protected peer review policy elements for timeliness--
specifically, completing final peer reviews within 120 calendar days; 
VA OIG reported in January 2011 that a VAMC within the VISN was not 
compliant with this timeliness requirement. 

The Director of Risk Management told us the Office of Risk Management 
monitors protected peer review processes by reviewing and analyzing 
aggregated data submitted quarterly by VAMCs through the VISNs, as 
required by VHA's protected peer review policy.[Footnote 41] The 
Director of Risk Management told us a staff member in that office 
reviews the data at the VISN and VAMC levels, including the total 
number of peer reviews; initial peer review level of care 1, 2, or 3 
ratings; total number of peer reviews sent to the peer review 
committee; and number of peer reviews with the rating level of care 
changed by the peer review committee. The Director of Risk Management 
said that if an outlier is identified in the aggregated data, it is 
brought to the attention of the relevant VISN officials. An official 
from the Office of Risk Management reported that the office produces 
quarterly reports from analysis of the VHA systemwide aggregated 
protected peer review data and that these quarterly reports are shared 
with each VISN. According to VHA's fourth quarter report for fiscal 
year 2012, ratings for 22 percent of peer-reviewed cases were changed 
by the VAMCs' peer review committees. VHA's analysis found that the 
peer review committees were more likely to improve the peer review 
rating by decreasing the assigned rating level, such as decreasing a 
level of care 3 to a level of care 1 or 2, or a level of care 2 to a 
level of care 1. 

In addition to monitoring VISNs' aggregated protected peer review 
data, VHA's Director of Risk Management said the office communicates 
regularly with the Office of the DUSHOM, as required by VHA's 
protected peer review policy and supported by federal internal control 
standards for information and communications. According to VHA's 
protected peer review policy, the DUSHOM's responsibilities include 
establishing and maintaining the peer review program in coordination 
with the ADUSH for Quality, Safety and Value, and providing direction 
and guidance on data elements that VAMCs must report through VISNs 
to VHA.[Footnote 42] Federal internal control standards state that 
mechanisms should exist to allow the easy flow of information down, 
across, and up through the organization, and easy communications 
should exist between functional activities. These standards also state 
that responsibility for decision-making should be clearly linked to the 
assignment of authority, and individuals should be held accountable 
accordingly.[Footnote 43] The Director of Risk Management said that 
the office periodically gives brief overviews of the office's work, 
including protected peer review findings, to the Under Secretary for 
Health and to the DUSHOM. 

Beyond VHA monitoring of VAMCs' protected peer review process, the VA 
OIG also routinely reviews certain policy elements of the process 
through its Combined Assessment Program, which reviews each VAMC every 
2 to 3 years.[Footnote 44] In the last 5 years, the VA OIG has 
reviewed VAMCs' compliance with several requirements, including (1) 
peer review committee must submit quarterly reports to the medical 
executive committee,[Footnote 45] (2) peer review committee must 
analyze protected peer reviews for trends in follow-up items and 
recommendations, and (3) protected peer reviews by the initial 
reviewer and the peer review committee must be completed timely. The 
VA OIG has conducted at least one Combined Assessment Program review 
since 2011 at each of the four VAMCs we visited.[Footnote 46] See 
table 4 for a summary of protected peer review monitoring activities 
conducted by VISNs, VHA's Office of Risk Management, and VA OIG. 

Table 4: Protected Peer Review Process Monitoring Activities by 
Selected VA and Veterans Health Administration (VHA) Organizational 
Components: 

VA/VHA organizational component: VHA Office of Quality, Safety and 
Value, Office of Risk Management; 
Data monitoring: [Check]; 
Site visits: [Empty]. 

VA/VHA organizational component: Veterans Integrated Service Networks 
(VISN); 
Data monitoring: [Check]; 
Site visits: [Check]. 

VA/VHA organizational component: VA Office of Inspector General Office 
of Healthcare Inspections; 
Data monitoring: [Empty]; 
Site visits: [Check]. 

Source: GAO. 

[End of table] 

According to officials from the Office of Risk Management, routine 
monitoring of certain policy elements of VAMCs' protected peer review 
processes has enabled VHA to make changes in policy and improve 
protected peer review monitoring activities. VHA's Director of Risk 
Management said that her office is responsible for making changes to 
policy in response to VA OIG findings related to the protected peer 
review process. For example, in a 2008 report on VISNs' oversight of 
the protected peer review process, the VA OIG found that VISNs failed 
to substantially comply with the requirement to conduct periodic 
inspections; the VA OIG recommended that VHA clarify its peer review 
policy to define periodic site visits required of VISNs.[Footnote 47] 
VHA's 2008 revision of the protected peer review policy redefined site 
visits as annual to result in more frequent monitoring by VISNs. 
[Footnote 48] VHA also improved peer review data monitoring 
activities. According to the Director of Risk Management, a national 
survey of VAMC risk managers was conducted in 2011 to better 
understand how protected peer review data are used. Based on survey 
data and a literature review, VHA officials said they determined that 
VAMCs and VISNs needed additional data on the specific areas of care 
that were being peer-reviewed as well as information about significant 
problems identified and frequently cited issues. VHA officials said 
that in fiscal year 2012, they expanded the required quarterly 
protected peer review data set that VAMCs report to the VISN to 
include the commonly peer-reviewed aspects of care, such as the choice 
of a diagnostic test, performance of a procedure or treatment, and 
addressing abnormal results of diagnostic tests. 

VHA and VA OIG Monitoring of Peer Review Triggers Is Limited: 

According to VHA's protected peer review policy, VISNs and the Office 
of Risk Management play a role in monitoring VAMCs' implementation of 
protected peer review processes, which include peer review triggers. 
In addition, federal internal control standards state that management 
needs to comprehensively identify risks and analyze the possible 
effects; these peer review triggers serve as part of VHA's risk 
assessment tool to help identify issues of risk to patient safety 
and improve the organization.[Footnote 49] 

Officials of the VISNs for the four VAMCs we visited and the VA OIG 
told us they have reviewed the establishment of protected peer review 
triggers by VAMCs. However, VISNs, VHA, and VA OIG have not monitored 
whether the triggers have actually been implemented. Officials we 
interviewed from two of the four VISNs said their VISNs reviewed 
VAMCs' peer review triggers during the required annual site visits; an 
official from a third VISN said that the VISN has confirmed that most 
VAMCs have established protected peer review triggers. Officials from 
the fourth VISN told us they have reviewed types of triggers in place 
at VAMCs, but they have not confirmed that VAMCs have established 
protected peer review triggers. Officials from all four VISNs told us 
they typically do not monitor whether VAMCs have implemented the 
established protected peer review triggers, including monitoring how 
many FPPEs for cause have been triggered. One VISN official explained 
that the VISN cannot monitor every aspect of every policy and 
regulation that govern VAMCs' operations; therefore, they choose to 
focus their monitoring efforts on the elements of particular 
importance where other entities--such as the VA OIG and The Joint 
Commission--have found evidence of noncompliance. 

VHA's Director of Risk Management told us that the office does not 
monitor whether VAMCs have established peer review triggers. Further, 
the official told us the office has not monitored how VAMCs have 
implemented peer review triggers or tracked how many providers may be 
exceeding the triggers and are subject to FPPEs for cause. The official 
also noted that VHA has not asked VAMCs to document their peer review 
triggers and has not asked the VA OIG to look specifically, during 
Combined Assessment Program reviews, at whether VAMCs have established 
triggers. 

Officials from the VA OIG told us they did monitor VAMCs' 
establishment of peer review triggers as part of Combined Assessment 
Program reviews conducted in fiscal year 2009. The OIG review found 97 
percent compliance with establishing peer review triggers across the 
44 VAMCs OIG officials visited; the four VAMCs we visited were not 
among the 44 VAMCs included in this review.[Footnote 50] According to 
VA OIG officials, they decided not to review the establishment of 
peer review triggers in subsequent Combined Assessment Program reviews 
because of the high compliance rate in 2009; instead, subsequent 
Combined Assessment Program reviews focused on requirements with which 
VAMCs had not been in compliance, such as requiring the peer review 
committee to submit quarterly reports on protected peer review to the 
medical executive committee.[Footnote 51, 52] VA OIG officials told us 
they have not reviewed whether VAMCs have implemented the triggers. 

Because neither VHA's Office of Risk Management nor VA OIG review 
whether peer review triggers have been implemented, VHA cannot provide 
reasonable assurance that VAMCs are using the triggers as a risk 
assessment tool as intended. Failure to do so weakens VAMCs' ability 
to ensure patient safety, and officials cannot be assured that the use 
of these triggers meets the intended goal of identifying providers 
that are not delivering safe, quality patient care. 

VAMC's Adherence to the FPPE Process Is Unclear Due to Gaps in Policy 
Addressing Documentation Requirements; VHA Does Not Routinely Monitor 
Nonprotected Processes: 

Gaps in Policy Create Lack of Clarity as to How VAMCs Are to Document 
the FPPE Process; Requirements for AIBs Generally Are Clear: 

FPPEs. According to federal internal control standards for control 
activities, written documentation should exist for all significant 
events that occur within an agency; this documentation should be 
readily available for examination, and it should be complete and 
accurate in order to facilitate tracing the event from initiation 
through processing to completion. In documenting FPPEs, building 
strong and complete evidence on each case is important to support the 
outcome of the evaluation, as well as to track the identified area of 
concern over time. 

VHA's FPPE policy provides a general definition of an FPPE, that it 
can be used for cause (when a question arises regarding a provider's 
ability to provide safe, quality patient care), that the criteria for 
the FPPE should be defined by the VAMC in advance, and that the 
results of the FPPE must be documented in the provider's profile. 
[Footnote 53] However, there are gaps in VHA's policy regarding how 
these evaluations should be documented and what information should be 
included, which limited our ability to assess VAMCs' adherence to the 
FPPE policy. Officials from two of the VAMCs we visited told us there 
are no standardized guidelines on how the FPPE process should be 
structured. According to the Director of Credentialing and Privileging, 
VHA's policy on FPPEs was intended to allow VAMCs flexibility in the 
design of the evaluation to accommodate the variety of ways that 
issues are identified and the types of issues that may be addressed 
(see box for example of an FPPE). 

Side bar: 

Example of an FPPE at one VAMC we visited; There are multiple ways 
that an FPPE may be prompted, including multiple patient complaints or 
exceeding a VAMC's peer review trigger. At one VAMC we visited, a 
provider exceeded a peer review trigger--receiving two or more level 
of care 3 peer review ratings within 6 months--which prompted a 
detailed assessment, which included a retrospective evaluation of 25 
percent of the provider's medical records over the previous 12-month 
period. The medical records were evaluated specifically for patient 
evaluations, outcomes, and documentation. Upon completion of the 
assessment, VAMC officials determined that the provider's 
documentation, including patient discharge summaries, patient 
transfer, medication review, and disclosure notes, was inadequate. The 
evaluator recommended initiating an FPPE, for which the provider would 
be instructed on how to properly document these types of summaries and 
notes. The evaluator also recommended that the provider write the 
notes for each patient seen before moving to see other patients, 
instead of writing the notes for all patients at the end of the day. 
VAMC officials also initiated a performance improvement plan for the 
provider, which included monitoring his medical record documentation 
over a 6-month period. When officials determined that the provider had 
not improved after this period, the FPPE was extended to continue 
monitoring aspects of his clinical care. The provider retired six 
months later. 

[End of side bar] 

Officials from the VAMCs we visited were generally aware that FPPEs 
can be used to address concerns about the quality of a provider's 
care; are time limited; and are not disciplinary, but could ultimately 
be used to take adverse action against a provider, if necessary. While 
VAMC officials were generally aware of the FPPE process, we found that 
the four VAMCs we visited varied widely in their documentation of 
FPPEs, attributable at least in part to the lack of specificity in 
VHA's FPPE policy regarding documentation requirements. In reviewing 
FPPEs conducted between fiscal years 2009 and 2011 we found the 
following: 

* One of the four VAMCs provided a completed template for each of its 
FPPEs, including the purpose of the FPPE, specifying what triggered 
the review, the time period for review, comments by the evaluator, an 
action plan based on the review, and evidence of concurrence with the 
review by the applicable service chief. (See appendix I for an example 
of an FPPE template used at one of the VAMCs we visited.) 

* Two other VAMCs provided various combinations of documents as 
evidence of their FPPEs, including professional standards board 
minutes and emails and letters from evaluators. These documents 
contained varying amounts of information detailing the circumstances 
prompting the FPPE, comments from evaluators, and follow-up actions, 
if any.[Footnote 54] 

* The fourth VAMC initially provided us with documentation of one 
FPPE, including a completed template identifying the clinical service 
involved, the method of evaluation, the evaluator's findings, and the 
service chief's conclusions; and several documents, each with focused 
professional practice evaluation labeled at the top, specifying the 
medical records evaluated for the FPPE and the evaluator's comments on 
each case. The VAMC's service chief told us that an FPPE had been 
conducted for cause for this provider, but the VAMC's quality manager 
said a formal FPPE had not been conducted and that the documentation 
we received was part of a protected peer review process. This 
disagreement illustrates that even within the same facility the 
interpretation of VHA's policy on FPPEs differs, which can lead to 
potentially inappropriate use. 

Gaps in VHA's policy on FPPE documentation requirements create a lack 
of clarity and therefore may affect VAMCs' ability to appropriately 
document the evaluation of providers' skills, support any actions 
initiated, and track provider-specific FPPE-related incidents over 
time. For example, if FPPEs are not well documented, VAMC officials 
may have limited knowledge of the findings to proceed with any actions 
and limited ability to track that such evaluations were conducted. As 
a result, if another adverse event subsequently occurred involving the 
same provider, the VAMC may not be aware of any prior findings. One 
VAMC official stressed to us the importance of thorough documentation 
of an FPPE, even if the determination is made that the provider 
delivered safe, quality patient care and no adverse action is needed. 
Without adequate documentation, a VAMC may conduct an FPPE that 
complies with VHA's policy, and determine that adverse action is 
needed on the basis of the evaluation's findings, but ultimately may be 
unable to take the action because the documented evidence is 
insufficient. Officials at one VAMC said they did not believe that 
the evidence gathered from an FPPE was strong enough to hold up 
against a provider's appeal of an adverse action. 

AIBs. Another type of nonprotected review that VAMC officials may 
choose to address an adverse event is an AIB.[Footnote 55] In our 
review of VA's AIB policy, we found that the policy generally provides 
clear guidance on the requirements, including documentation. For 
example, the policy specifies steps on how VAMC officials determine 
the need for an AIB, select the board members, and write the charge 
letter that convenes the AIB, and also provides a number of templates 
and checklists, including templates for the charge letter and 
investigative report, and checklists for collecting evidence, 
conducting interviews, and writing the report.[Footnote 56] 

When asked about AIBs, VAMC officials told us an AIB could be used 
when (1) a case is egregious; (2) there is a previous incident 
involving the same provider; (3) the VAMC needs or wants to take 
privileging or disciplinary action; and (4) the facts of a case are in 
dispute or when there is a high level of complexity and ambiguity in 
the case. For example, at one VAMC we visited, an AIB was convened to 
investigate the facts and circumstances regarding the care and 
treatment immediately preceding a veteran's death. The AIB was charged 
with investigating allegations that a provider's care deviated from 
standard practice procedures, including prescribing inappropriate 
doses of medications, not ensuring appropriate monitoring or 
appropriate interval of physical assessments, and providing 
substandard documentation and communication. 

VHA Does Not Routinely Monitor Nonprotected Processes: 

FPPEs. According to federal internal control standards for monitoring, 
agencies should assess the quality of performance over time and 
provide reasonable assurance that deficiencies are detected and 
promptly resolved. Further, VHA's credentialing and privileging policy 
states that the DUSHOM is responsible for ensuring that VISN directors 
maintain an appropriate credentialing and privileging process, which 
includes FPPEs, consistent with VHA policy.[Footnote 57] The VISN 
Chief Medical Officer is responsible for oversight of the 
credentialing and privileging process of the VAMCs within the VISN. 

Officials we interviewed from three of the four VISNs we interviewed 
told us they do not monitor FPPEs conducted in response to questions 
about a provider's ability to deliver safe, quality patient care. 
Officials from two of those three VISNs said that they have examined a 
sample of FPPEs during annual site visit reviews of VAMC operations; 
however, the sample of FPPEs would include mostly FPPEs for newly 
hired providers, since there are very few FPPEs conducted in response 
to questions that have arisen about a provider's ability to deliver 
safe, quality patient care. An official from the fourth VISN said he 
monitors FPPEs during annual site visits to VAMCs, including FPPEs 
conducted in response to questions that have arisen about a provider's 
ability to deliver safe, quality patient care. 

Officials from the office of the DUSHOM and the Office of 
Credentialing and Privileging said they do not monitor FPPEs conducted 
in response to questions about a provider's ability to deliver safe, 
quality patient care. The Office of Credentialing and Privileging does 
not monitor these FPPEs because there are so few of them that the cost 
of reviewing a process that occurs so infrequently would outweigh the 
benefit. According to VHA officials, if an FPPE leads to a proposed 
reduction or revocation of clinical privileges, the Director of 
Credentialing and Privileging is frequently consulted to ensure that 
appropriate due process is afforded the provider.[Footnote 58] 

Similar to the VISNs, VA OIG officials told us they do not monitor 
FPPEs conducted when a question arises regarding a provider's ability 
to deliver safe, quality patient care. However, VA OIG officials said 
that, during site visits for Combined Assessment Program reviews, they 
do review policy elements of the process for FPPEs for providers newly 
appointed to the VAMCs' medical staff or for providers requesting new 
privileges.[Footnote 59] Because none of these entities monitor FPPEs 
conducted when a question arises regarding a provider's ability to 
deliver safe, quality patient care, VHA cannot be assured that the 
process is working as intended or whether VAMCs need additional 
guidance or training about the process. 

AIBs. Officials from two of the four VISNs we interviewed said they 
routinely monitor VAMCs' AIB activity, including confirming whether 
the AIBs were completed timely and what kinds of issues were 
addressed. Officials from the other two VISNs said they do not monitor 
AIBs. Because the AIB directive is a VA policy, there are no explicit 
requirements for VHA to monitor AIBs; however, federal internal 
control standards for information and communications state that 
relevant, reliable, and timely information is needed by an agency to 
achieve its objectives and to control its operations.[Footnote 60] In 
our 2012 report on AIBs, we found that VA does not collect and analyze 
aggregated data on AIB investigations, and at that time, VHA officials 
told us that there were no plans to do so.[Footnote 61] In our report, 
we recommended that VA establish a process to collect and analyze 
aggregated data from AIB investigations conducted within VHA. Having 
these types of data may provide VA with valuable information to 
systematically gauge the extent to which matters investigated by AIBs 
are occurring throughout VHA and to take corrective action, if needed, 
to reduce the likelihood of future occurrences. According to VHA 
officials, after our 2012 report was issued, VHA created a workgroup 
to examine our recommendation and concluded that monitoring AIBs was 
not necessary or warranted. 

Conclusions: 

VAMCs' adherence to policies on protected and nonprotected processes 
for responding to providers' actions that contribute to adverse events 
helps ensure that quality care is provided to veterans and that safety 
risks are minimized. Having clear and detailed guidance in policy for 
these processes is critical to helping VAMC officials identify and 
address adverse events, including providers' contributing actions, in 
a timely and appropriate manner. Although officials at the four VAMCs 
we reviewed generally understood the protected peer review process, we 
found that none of these four VAMCs adhered to all four VHA protected 
peer review policy elements selected for review, such as completing 
peer reviews within required time frames and sending the required peer-
reviewed cases to the peer review committee for further assessment. As 
such, VHA may be missing opportunities for improvements both in the 
practice of individual providers and organizationally. VHA also may be 
missing opportunities to identify and intervene early with providers 
whose care may pose a risk to patient safety if VAMC officials have 
not established or implemented peer review triggers that would 
initiate a detailed assessment of a provider's care. Assisting with 
and monitoring VAMCs' development and use of these peer review 
triggers will help VHA ensure that the protected peer review process 
contributes to organizational improvements and favorable patient 
outcomes, as intended by VHA policy. 

FPPEs and AIBs are nonprotected processes that VAMCs use to address 
adverse events involving individual providers. However, gaps in VHA's 
FPPE policy on documentation requirements have created a lack of 
clarity for VAMCs on how to appropriately document the process. 
Inadequate documentation of the FPPE process may result in VAMC 
officials being unable to take adverse action against a provider when 
necessary. Providing more specific policy guidance for FPPEs would 
better support VAMCs' use of this process, including when officials 
determine that they may need to take adverse action against a 
provider. Although VHA officials reiterated that they do not have 
plans to collect and analyze aggregated AIB data as we recommended in 
our 2012 report, we continue to believe that this is a potentially 
important quality improvement tool for use by VHA. 

Recommendations for Executive Action: 

To improve VHA's use of the protected peer review and nonprotected 
processes to respond to individual providers involved in adverse 
events or when questions arise regarding providers' ability to deliver 
safe, quality patient care, we are making five recommendations. 

To address protected peer review process requirements, we recommend 
that the Secretary of Veterans Affairs direct the Under Secretary for 
Health to: 

* ensure that VAMCs send all required initial peer reviews (level of 
care 2 and 3) to the peer review committee; 

* ensure VAMCs' peer review committees complete final peer reviews 
within 120 calendar days; 

* provide clear guidance and assistance on the purpose, development, 
and implementation of peer review triggers; and: 

* require VAMCs to periodically provide data on protected peer review 
triggers, including the number of providers that have exceeded the 
triggers as part of the protected peer review data VAMCs report to 
VISNs on a quarterly basis. 

To address the nonprotected FPPE process, we recommend that the 
Secretary of Veterans Affairs direct the Under Secretary for Health to 
develop more specific policy on the FPPE process, including 
documentation requirements such as the FPPE's purpose, time period 
covered, evaluator's assessment, and the summary of actions to be 
taken. 

Agency Comments and OUr Evaluation: 

VA provided written comments on a draft of this report, which we have 
reprinted in appendix II. In its comments, VA generally concurred with 
our conclusions and our five recommendations, and described the 
agency's plans to implement each of our recommendations. VA also 
provided technical comments, which we have incorporated as appropriate. 

In response to our first and second recommendations that VA ensure 
that all initial peer reviews (levels of care 2 and 3) be sent to the 
peer review committee for review and that the committee's reviews be 
completed within 120 days, VA stated that it will provide refresher 
education to key staff, such as chiefs of staff, risk managers, and 
VISN officials. VA anticipates that its planned actions will be 
completed by December 31, 2013. 

In response to our third recommendation that VA provide clear guidance 
and assistance on the purpose, development, and implementation of peer 
review triggers, VA stated that refresher education on this policy 
requirement, which also encompasses professional activity triggers, 
was communicated to key staff, such as chiefs of staff, risk managers, 
and VISN officials, through a conference call. Additionally, VA stated 
that staff in the VHA Risk Management Program, Office of Quality, 
Safety and Value will be available to provide consultative assistance 
to facilities that are unclear on how to implement this requirement. 
VA anticipates completion of these activities by December 31, 2013. 

In response to our fourth recommendation that VA require VAMCs to 
periodically provide data on peer review triggers, VA concurred. VA 
stated that VAMCs will be required to submit a deidentified, summary 
report discussing trends and analysis of aggregate data on peer review 
activity with their quarterly submission to the VISN. VA stated that 
this new requirement will be included in the fiscal year 2014 revision 
of VHA Directive 2010-025, Peer Review for Quality Management, which 
establishes that the VAMC's medical executive committee is responsible 
for determining peer review or professional activity trigger levels. 
VA anticipates completion of these activities by September 30, 2014. 
VA disagreed with the latter part of our recommendation that the data 
submitted should include the number of providers that have exceeded 
the triggers. VA stated that reviewing aggregate data for the number 
of providers who exceeded trigger thresholds would represent skewed 
data, which VA officials believe is not reflective of the quality of 
care provided by those providers submitted for triggered reviews. VA 
stated larger facilities may appear to have artificially higher levels 
of providers referred for detailed assessments than those of smaller 
facilities. We agree with VA that VAMC clinical leadership, with VISN 
oversight, input, and support, is the preferred means to handle 
trigger thresholds and data analysis; however, VA did not specify what 
data on protected peer review triggers, that VAMCs would be required 
to report. We maintain that it is important for VAMCs and the VISNs to 
review whether the peer review triggers are implemented as intended. 
Part of this review should include monitoring how many providers have 
exceeded the trigger thresholds. Additionally, we believe that 
collecting and reporting such data will help the VISNs and VHA ensure 
that the protected peer review process contributes to organizational 
improvements and favorable patient outcomes. 

In response to our fifth recommendation that VA develop more 
specific policy on the FPPE process, including documentation 
requirements such as the FPPE's purpose, time period covered, 
evaluator's assessment, and the summary of actions to be taken, VA 
stated that it will develop guidance on the FPPE process that will 
begin with a description of the process for a detailed assessment and 
define the FPPE for cause process if an opportunity to improve is 
indicated. VA further noted that the guidance will end with an 
overview of the adverse action process to be initiated when the 
provider does not demonstrate adequate improvement and a reduction or 
revocation of clinical privileges appears to be indicated. VA 
anticipates completion of these activities by September 30, 2014. 
While we understand VA's intention of expediting dissemination of 
information about the FPPE process through guidance, we believe that 
it is important for the guidance to be included in the next formal 
iteration of VHA policy. 

We are sending copies of this report to the appropriate congressional
committees, the Secretary of Veterans Affairs, the Under Secretary for
Health, and other interested parties. In addition, the report is 
available at no charge on the GAO website at [hyperlink, 
http://www.gao.gov]. 

If you or your staffs have any questions about this report, please 
contact me at (202) 512-7114 or draperd@gao.gov. Contact points for 
our Offices of Congressional Relations and Public Affairs may be found 
on the last page of this report. GAO staff who made major 
contributions to this report are listed in appendix III. 

Signed by: 

Debra A. Draper: 
Director, Health Care: 

[End of section] 

Appendix I: Example of a VA Medical Center's (VAMC) Focused 
Professional Practice Evaluation (FPPE) Template: 

This FPPE form is a full recreation of a template used by one of the 
VAMCs we visited. 

VA Medical Center: 

Focused Professional Practice Evaluation: 

Provider Name: 

Date Of Appointment: 

Purpose Of Focused Professional Practice Evaluation: 

* New provider with clinical privileges has the credentials to suggest 
competency but a period of evaluation is needed to confirm the new 
provider’s competence in this organization. 

* Provider has requested a new clinical privilege. 

* Provider requires supervision for a procedure(s) and/or treatment 
modality. 

* Service has concerns about a provider’s competence in relation to:
- Sentinel event; 
- Provider-specific tort settlement; 
- Substantiated provider-specific complaint; 
- Significant safety violation; 
- Repeated or egregious unprofessional behavior. 

* Concerns raised by Medical Staff regarding provider’s performance: 
- Other: 
- Specify: 

Time Period For Review: 
From: 
To: 

Competency Assessment: 

1. Patient Care: 

Provides patient care that is compassionate, appropriate and effective 
for the promotion of health, prevention of illness and treatment of 
disease. 

Provider Name: 

Measures of Competency: 
* Medical Record Review; 
* Clinical Inspection/Observation of Actual Treatment Procedures; 
* Morbidity and Mortality Data; 
* Other: 
Specify: 

Competency Rating: 
Unsatisfactory/Marginal Performance: 1: 2: 3: 
Fully Satisfactory Performance: 4: 5: 6: 
Outstanding Performance: 7: 8: 9: 

Comments: 

2. Medical/Clinical Knowledge: 

Demonstrates knowledge of established and evolving biomedical, 
clinical and social sciences, and applies knowledge to patient care 
and the education of others.New provider with clinical privileges has 
the credentials to suggest competency but a period of evaluation is 
needed to confirm the new provider’s competence in this organization. 

Measures of Competency: 
* Case Management; 
* Completion Orientation/Mandatory Training; 
* Current Licensure; 
* Current BLS; 
* Current ACLS; 
* Other: 
Specify: 

Competency Rating: 
Unsatisfactory/Marginal Performance: 1: 2: 3: 
Fully Satisfactory Performance: 4: 5: 6: 
Outstanding Performance: 7: 8: 9: 

Comments: 

3. Practice-Based Learning And Improvement: 

Uses scientific evidence and methods to investigate, evaluate, and 
improve patient care practices. 

Measures of Competency: 
* Compliant with Peer Review Recommendations; 
* Professional CME; 
* Conference/Journal/Study Club Presentations; 
* Other: 
Specify: 

Competency Rating: 
Unsatisfactory/Marginal Performance: 1: 2: 3: 
Fully Satisfactory Performance: 4: 5: 6: 
Outstanding Performance: 7: 8: 9: 

Comments: 

4. Interpersonal And Communication Skills: 

Demonstrates interpersonal and communication skills to establish and 
maintain professional relationships with patients, families, and other 
members of the healthcare team. 

Measures of Competency: 
* Discussion with other individuals involved in the care of provider’s 
patients; 
* Staff/patient complaints; 
* Staff/patient compliments; 
* Other: 
Specify: 

Competency Rating: 
Unsatisfactory/Marginal Performance: 1: 2: 3: 
Fully Satisfactory Performance: 4: 5: 6: 
Outstanding Performance: 7: 8: 9: 

Comments: 

5. Professionalism: 

Demonstrates behaviors that reflect a commitment to continuous 
professional development, ethical practice, an understanding and 
sensitivity to diversity, and a responsible attitude toward patients, 
the medical profession, and society. 

Measures of Competency: 
* Professional Behavior; 
* Compliance with Bylaws, Directives, Policies; 
* Meets Administrative Responsibilities in Timely Manner; 
* Participates Medical Center Committees, Staff Meetings; 
* Other: 
Specify: 

Competency Rating: 
Unsatisfactory/Marginal Performance: 1: 2: 3: 
Fully Satisfactory Performance: 4: 5: 6: 
Outstanding Performance: 7: 8: 9: 

Comments: 

6. System-Based Practice: 

Demonstrates an understanding of the contexts and systems in which 
health care is provided and the ability to apply this knowledge to 
improve and optimize health care. 

Measures of Competency: 
* Compliant with Team Systems Design/Organization Processes; 
* Time-Out Procedure/Process Compliance; 
* Informed Consent Process Compliance; 
* Other: 
Specify: 

Competency Rating
Unsatisfactory/Marginal Performance: 1: 2: 3: 
Fully Satisfactory Performance: 4: 5: 6: 
Outstanding Performance: 7: 8: 9: 

Comments: 

Overall Rating: 
Unsatisfactory/Marginal Performance: 1: 2: 3: 
Fully Satisfactory Performance: 4: 5: 6: 
Outstanding Performance: 7: 8: 9: 

Comments: 

Provider Name: 

Action: 

* No concerns regarding competency; proceed with Ongoing Professional 
Practice Evaluation 

* Extent time period for focused professional practice evaluation. 

Specified Time Period: 
From: 
To: 

Specified Clinical Privileges:
* All; 
* One or more; 
* Specify: 

* Initiate plan for performance improvement. 

Specify: 

* It is recommended that this competence assessment be addressed with 
the provider. 

* Findings of competency assessment discussed with the provider? 
Yes; 
No. 

Print Reviewer’s Name and Title: 
Date: 

Reviewer’s Signature: 
Date: 

* Concur; 
* Do not Concur. 

Actions to be taken: 

Signature of Service Chief: 
Date: 

Source: VHA. 

[End of section] 

Appendix II: Comments from the Department of Veterans Affairs: 

Department of Veterans Affairs: 
Washington DC 20420 

November 5, 2013: 

Ms. Debra A. Draper: 
Director, Health Care: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Ms. Draper: 

The Department of Veterans Affairs (VA) has reviewed the Government
Accountability Office's (GAO) draft report, "VA Health Care: 
Improvements Needed in Processes Used to Address Providers' Actions 
That Contribute to Adverse Events" (GAO-14-55). VA generally agrees 
with GAO's conclusions and concurs with GAO's recommendations to the 
Department. 

The enclosure specifically addresses GAO's recommendations and 
provides technical comments to the draft report. VA appreciates the 
opportunity to comment on your draft report. 

Sincerely, 

Signed by: 

Jose D. Riojas: 
Chief of Staff: 

Enclosure: 

Department of Veterans Affairs (VA) Response to Government 
Accountability Office (GAO) Draft Report: 

"VA Health Care: Improvements Needed in Processes Used to Address 
Providers' Actions That Contribute to Adverse Events" (GAO-14-55): 

GAO Recommendation: To improve VHA's use of the protected peer review 
and nonprotected processes to respond to individual providers involved 
in adverse events or when questions arise regarding providers' ability 
to deliver safe, quality care, we are making five recommendations. 

To address the protected peer review process, we recommend that the 
Secretary of VA direct the Under Secretary for Health to: 

Recommendation 1: Ensure that VAMCs send all required initial peer 
reviews (level of care 2 and 3) to the peer review committee. 

VA Comment: Concur. The Veterans Health Administration (VHA) Directive 
2010-025, Peer Review for Quality Management, dated June 3, 2010, 
requires the Peer Review Committee to reconsider all peer review for 
quality management cases within the facility completed by individual 
initial peer reviewers when the level of review is determined to be a 
Level 2 or Level 3. The involved provider will have an opportunity to 
submit written comments or appear before the Peer Review Committee 
prior to the committee's determination of a final level assignment. 

Refresher education on this policy requirement will be communicated to 
key constituent groups, e.g., Chiefs of Staff, Risk Managers, Veterans 
Integrated System Network (VISN) Chief Medical Officers, and VISN 
Quality Management Officers, to reinforce compliance with this 
directive. This information was communicated on the quarterly Risk 
Managers call held on September 24, 2013. Anticipated completion date 
is December 31, 2013. 

Recommendation 2: Ensure VAMCs' peer review committees complete final 
peer reviews within 120 calendar days. 

VA Comment: Concur. VHA Directive 2010-025 states that the Peer Review 
Committee must complete a final review of each referred case within 
120 days from the determination that a peer review is necessary. The 
exception for a delay, or an extension beyond 120 days, needs to be 
requested in writing and approved by the facility Director. 

Medical centers are required to submit a quarterly update of their 
compliance to their VISN. The VISN does a roll-up of this peer review 
monitor, along with other peer review data, for submission to the VHA 
Risk Management Program for analysis. VISNs that have medical centers 
not meeting this requirement are contacted to ensure an action plan 
has been developed to maintain compliance. 

Refresher education on this policy requirement was communicated to key 
constituent groups, e.g., Chiefs of Staff, Risk Managers, VISN Chief 
Medical Officers, and VISN Quality Management Officers, to reinforce 
compliance with this directive. This information was communicated on 
the quarterly Risk Manager call held on September 24, 2013. 
Anticipated completion date is December 31, 2013. 

Recommendation 3: Provide clear guidance and assistance on the 
purpose, development and implementation of peer review triggers. 

VA Comment: Concur. VHA Directive 2010-025 states that the Medical 
Executive Committee is responsible for establishing peer review or 
professional activity triggers, e.g., three Level 3 assignments to a 
provider in a rolling 12-month period, or professional actions that 
lead to a focused review of a provider's clinical care. 

Refresher education on this policy requirement was communicated to key 
constituent groups, e.g., Chiefs of Staff, Risk Managers, VISN Chief 
Medical Officers, and VISN Quality Management Officers, to reinforce 
compliance with this directive. Staff in the VHA Risk Management 
Program, Office of Quality, Safety, and Value will be available to 
provide consultative assistance to facilities that are unclear on how 
to implement this requirement. This information was communicated on 
the quarterly Risk Manager call held on September 24, 2013. 
Anticipated completion date is December 31, 2013. 

Recommendation 4: Require VAMCs to periodically provide data on 
protected peer review triggers, including the number of providers that 
have exceeded the triggers as part of the protected peer review data 
VAMCs report to VISNs on a quarterly basis. 

VA Comment: Concur. 

VA concurs that VA medical centers (VAMC) should be required to 
periodically provide data on protected peer review triggers to the 
VISNs but disagrees with including the number of providers that have 
exceeded the triggers. 

The Peer Review program is a quality assurance activity. VHA Directive 
2010-025, Peer Review for Quality Management, states that the Medical 
Executive Committee is responsible for determining the trigger levels. 
This is a locally driven decision-making process that allows the 
clinical leadership to provide appropriate oversight on potential 
problems in their facility and to understand how and why adverse 
events may occur within VA medical facilities. 

Reviewing aggregate data for the number of providers who exceeded 
trigger thresholds would represent skewed data, which is not 
reflective of the quality of care provided by those providers 
submitted for triggered reviews. Facilities vary across the Nation by 
size and complexity. Very large facilities that see more critically 
ill patients and perform more high-risk procedures would have more 
opportunities for the provider's care to meet or exceed the trigger 
thresholds. Conversely, smaller facilities that do not see critical 
patients and whose care is outpatient based, will have fewer 
opportunities for providers to meet or exceed the trigger thresholds. 
By comparing national aggregate data, larger facilities may appear to 
have artificially high levels of providers referred for focused 
reviews than those of smaller facilities. Higher facility complexity 
levels have a disproportionally higher number of providers in clinical 
sub-specialties (e.g., neurosurgery and cardiothoracic surgery), and 
this will also play a role in the probability of a provider being 
referred for a focused review due to activating a trigger. Further, 
while small facilities may be less likely to reach a threshold, they 
also have greater risk of severe variations in the numbers of events. 
Though the quality of care could remain constant, by chance alone a 
trigger may occur. It is important that facilities be able to assess 
the context in which events happen, not be bound by arbitrary triggers. 

VISN and medical centers should determine and review their own 
triggers, which allows for consideration and analysis based on 
regional and local variations in the complexity of care provided at 
any given facility. Local clinical leadership, with VISN oversight, 
input and support is the preferred means to handle trigger thresholds 
and data analysis. 

VHA Directive 2010-025, Peer Review for Quality Management establishes 
that the Medical Executive Committee is responsible for determining 
the trigger levels. The facility monitors this process and takes 
appropriate follow up action with involved providers. The directive is 
scheduled to be updated this fiscal year. The updated directive will 
include a provision that requires facilities to submit a de-
identified, summary report discussing trends and analysis of aggregate 
data with their quarterly submission to the VISN of peer review 
activity. 

Target completion date: September 30, 2014. 

GAO Recommendation: To address the nonprotected FPPE process, we 
recommend that the Secretary of VA direct the Under Secretary for 
Health to: 

Recommendation 5: develop more specific policy on the FPPE process, 
including documentation requirements such as the FPPE's purpose, time 
period covered, evaluator's assessment, and the summary of actions to 
be taken. 

VA Comment: Concur. VHA concurs with this recommendation and will 
develop guidance on the Focused Professional Practice Evaluation 
(FPPE) process beginning with initial privileges through Ongoing 
Professional Practice Evaluation (OPPE) for ongoing monitoring of 
provider competency. This guidance will describe the processes to be 
used when OPPE identifies concerns with the care delivered by a 
provider. The guidance will begin with a description of the process 
for a focused review and define the FPPE for cause process if an 
opportunity to improve is indicated. The guidance will end with an 
overview of the adverse action process to be initiated when the 
provider does not demonstrate adequate improvement and a reduction or 
revocation of clinical privileges appears indicated. Anticipated 
completion date is September 30, 2014. 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Debra A. Draper, (202)512-7114 or draperd@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Marcia Mann, Assistant Director;
Ashley Dixon; Mariel Lifshitz; Katie McConnell; Elizabeth Morrison;
Lisa Motley; Ann Tynan; and Michael Zose made key contributions to this
report. 

[End of section] 

Footnotes: 

[1] Adverse events may not always be attributable to an error made by 
a provider; for example a patient fall occurring while under medical 
care may be considered an adverse event, but be unrelated to patient 
care. 

[2] Julia Neily et al., "Incorrect Surgical Procedures within and 
Outside of the Operating Room, A Follow-up Report," Archives of 
Surgery, vol. 146, no. 11 (2011). 

[3] According to VHA officials, VHA requires adverse events to be 
reported to appropriate VAMC officials through each VAMC's incident 
reporting system, or through other methods such as a phone call or 
email to the applicable medical center's patient safety manager or 
risk manager. In addition to identifying and reporting adverse events, 
VAMCs have a quality management program that enables them to identify 
and respond to other quality-of-care issues that may not rise to the 
level of an adverse event. VHA uses the terms quality management, 
quality improvement, and quality assurance interchangeably. 

[4] See GAO, Veterans Health Care: Veterans Health Administration 
Processes for Responding to Reported Adverse Events, [hyperlink, 
http://www.gao.gov/products/GAO-12-827R] (Washington, D.C.: Aug. 24, 
2012). In this report we found that VAMCs may also use root cause 
analysis to evaluate the systems or process causes for an adverse 
event; we did not include root cause analysis in this report because 
we focused on the processes used to evaluate actions of individual 
providers, not processes used to address systems or process issues. 

[5] FPPEs are also conducted at the time of a provider's initial 
appointment to the VAMC's medical staff, or when a provider applies 
for new or additional privileges; we excluded these types of FPPEs 
from the scope of our review. Privileges are the authority granted to 
a physician or dentist by the VAMC to provide patient care in the 
facility. Privileges are limited by the VAMC's capabilities and the 
individual's professional license, education, training, experience, 
and competence. Hereafter, we use the term FPPE to refer to an FPPE 
conducted for cause, such as when questions arise about a privileged 
provider's ability to deliver safe, quality care. 

[6] An example of a system deficiency would be a failure of a VAMC's 
pharmacy to ensure patients receive the prescribed medication. 

[7] Hereafter, we refer to peer review for quality management as 
protected peer review. 

[8] Under federal law, records and documents created as part of VHA's 
medical quality assurance program are confidential and privileged and 
may only be disclosed under limited circumstances. See 38 U.S.C. § 
5705; 38 C.F.R. §§ 17.500-17.511. VHA's medical quality assurance 
program consists of systematic health care reviews carried out by or 
for VHA for the purposes of improving the quality of medical care or 
improving the utilization of health care resources in VHA medical 
facilities. The protected peer review process is part of VHA's medical 
quality assurance program, and, as such, documents generated through 
these processes are confidential and privileged. 

[9] VHA policy refers to nonprotected processes generally as 
management reviews. 

[10] According to VA policy, major adverse actions include suspension, 
transfer, reduction in grade, reduction in basic pay, and termination. 
Disciplinary actions include admonishments, which are official letters 
of censure for minor deficiencies in competence or conduct; and 
reprimands, which are more severe letters of censure for deficiencies 
in competence or conduct. Department of Veterans Affairs, Employee/ 
Management Relations, VA Directive 5021, v.3 (Washington, D.C.: Aug. 
28, 2007) and Employee/Management Relations, VA Handbook 5021, v.10 
(Washington, D.C.: Nov. 8, 2012), accessed April 17, 2013, [hyperlink, 
http://www.va.gov/vapubs/search_action.cfm?dType=2]). Privileging 
actions include reducing, revoking, suspending, denying, or failing to 
renew a provider's privileges. Veterans Health Administration, 
Credentialing and Privileging, VHA Handbook 1100.19 (Washington, D.C.: 
Nov. 14, 2008). 

[11] According to VHA officials, protected peer review and FPPE 
processes are specific to VHA. In contrast, the AIB process is used 
across VA, by VHA as well as VA's other administrations--the Veterans 
Benefits Administration and the National Cemetery Administration. 

[12] GAO, Internal Control Standards: Internal Control Management and 
Evaluation Tool, [hyperlink, http://www.gao.gov/products/GAO-01-1008G] 
(Washington, D.C.: Aug. 2001). 

[13] VISNs are regional systems of care that oversee the day-to-day 
functions of VAMCs that are within their network. Each VAMC is 
assigned to a single VISN. VHA assigns each VAMC a surgical complexity 
rating of standard, intermediate, or complex. This classification 
system uses multiple variables to measure surgical complexity arrayed 
along four categories, namely facilities, equipment, workload, and 
staffing load. For example, a hospital assigned a complex rating 
requires special facilities, equipment, and staff for difficult 
operations, such as cardiac surgery and organ transplants. We chose 
three sites with a complex rating and one with an intermediate rating, 
for variation. 

[14] Data from fiscal years 2009 through 2011 were the most recent 
data available at the time we initiated our review. 

[15] Department of Veterans Affairs, Veterans Health Administration, 
Peer Review for Quality Management, VHA Directive 2010-025 
(Washington, D.C.: June 3, 2010). 

[16] VHA Directive 2010-025. VHA refers to the detailed assessment as 
a focused review. 

[17] A peer review trigger typically is a threshold that is determined 
by the number and results of protected peer reviews within a specified 
time frame, usually 12 months, for a particular provider that 
indicates the need for further review of the provider's patient care. 
While VHA's protected peer review policy mentions professional 
activity triggers, it does not provide a definition or illustrative 
example of such triggers. According to a VHA official, a professional 
activity trigger is based on clinical actions or professional clinical 
decisions made by the practitioner in the episode of care under review. 

[18] GAO, VA Administrative Investigations: Improvements Needed in 
Collecting and Sharing Information, [hyperlink, 
http://www.gao.gov/products/GAO-12-483] (Washington, D.C.: Apr. 30, 
2012). 

[19] Department of Veterans Affairs, Administrative Investigations, VA 
Handbook 0700 (Washington, D.C.: undated). 

[20] Since this report covers fiscal years 2009 through 2011, the 
period of time covered by the applicable records retention schedule 
for AIB documentation might have passed by the time we requested the 
documentation. 

[21] A root cause analysis is a process for identifying the basic or 
contributing factors that underlie variations in the performance of 
systems and processes. 

[22] VHA Directive 2010-025. 

[23] The peer reviewer is a provider who has relevant expertise 
necessary to make accurate judgments about the clinical event being 
reviewed, is able to make a fair and credible assessment, and has 
knowledge of relevant standards of care. According to VHA and VAMC 
officials, generally, a provider is peer-reviewed by another provider 
at the same VAMC and, if possible, within the same specialty. 

[24] Language referring to this 45 calendar day requirement was 
changed in the 2010 version of VHA's peer review policy from must to 
should be completed in 45 calendar days. According to the VHA Director 
of Risk Management, despite this change in policy language, VAMCs are 
still expected to complete initial peer reviews around the 45 day time 
frame. VHA's protected peer review policy was updated in June 2010, 
which was during the time period we studied--fiscal years 2009 through 
2011. For the previous version of VHA's protected peer review policy, 
see Department of Veterans Affairs, Veterans Health Administration, 
Peer Review for Quality Management, VHA Directive 2008-004 
(Washington, D.C.: Jan. 28, 2008). 

[25] The peer review committee is a multidisciplinary group that is 
chaired by the Chief of Staff and includes the Nurse Executive, senior 
members of key clinical disciplines, and nonphysician members. 

[26] VHA contracted with Lumetra Healthcare Solutions to conduct 
external protected peer reviews of selected cases and to assess the 
accuracy of peer review conducted by VAMCs. According to VHA 
officials, the implementation of this contract started in fiscal year 
2011. Lumetra Healthcare Solutions has a peer review network of over 
250 physicians and other clinical professionals. 

[27] VAMCs that offer more complex clinical services are required to 
send 15 cases per quarter, and VAMCs that offer less complex clinical 
services are required to send 10 cases per quarter, for this external 
audit. 

[28] VHA Directive 2010-025. 

[29] An FPPE can be initiated for reasons other than when a peer 
review trigger is met or exceeded, such as when a VAMC official has a 
concern about the clinical actions or decisions made by a provider for 
the episode of care under review. 

[30] A privileging action could include reduction, denial, nonrenewal, 
or revocation of privileges. 

[31] These 100 FPPEs include only FPPEs conducted for cause and do not 
include FPPEs conducted for newly hired providers or for providers 
requesting new or additional privileges. 

[32] Clinical competence is the documented demonstration that an 
individual provider has the requisite or adequate abilities or 
qualities to perform up to a defined expectation in providing patient 
care. 

[33] VA Handbook 0700. 

[34] We recommended that VHA establish a process to collect and 
analyze aggregate data from AIB investigations, including the number 
of investigations conducted, the types of matters investigated, 
whether the matters were substantiated, and the systemic deficiencies 
identified. See [hyperlink, http://www.gao.gov/products/GAO-12-483]. 

[35] The VA Office of Inspector General (OIG) found in its report 
focusing on one of the sites we visited that 2 percent of final peer 
reviews exceeded the 120 calendar day time frame and extensions had 
not been requested or approved. This was a repeat finding from the VA 
OIG's previous review. 

[36] This requirement did not change from the 2008 to the 2010 version 
of VHA's peer review policy. 

[37] VHA Directive 2008-004. 

[38] VHA's peer review policy requires that VISNs conduct and complete 
annual and ad hoc site visits, ensure peer review data are collected, 
analyzed, and acted upon, and ensure peer review data are loaded 
quarterly into a shared database. See VHA Directive 2010-025. 

[39] VHA Directive 2010-025. 

[40] The Joint Commission is an independent, not-for-profit 
organization that accredits and certifies more than 20,000 health care 
organizations and programs in the United States, including VAMCs. 
According to VHA's policy on protected peer review, VAMCs' protected 
peer review processes must comply with accreditation requirements of 
The Joint Commission. 

[41] VHA's protected peer review policy requires VHA to conduct 
analysis of peer review data findings submitted by each VISN and to 
disseminate those findings to the Under Secretary for Health, VISNs, 
and other leadership. See VHA Directive 2010-025. 

[42] VHA Directive 2010-025. 

[43] [hyperlink, http://www.gao.gov/products/GAO-01-1008G]. 

[44] VA OIG conducts Combined Assessment Program reviews that provide 
cyclical oversight of VAMCs and are used to review selected clinical 
and administrative operations. 

[45] The medical executive committee is the primary governance 
committee for the medical staff at the VAMC. The medical executive 
committee, with input from the medical staff, makes key leadership 
decisions related to medical staff policies, procedures, and rules, 
with an emphasis on quality control and quality improvement 
initiatives. According to VHA's protected peer review policy, the 
medical executive committee is responsible for utilizing data from the 
peer review committee to determine the need for further action. 

[46] For examples of VA OIG Combined Assessment Program reports for 
the four sites, see Department of Veterans Affairs, Office of 
Inspector General Office of Healthcare Inspections, Combined 
Assessment Program Review of the VA Puget Sound Health Care System, 
Seattle, Washington, Report No. 11-04569-141 (Washington, D.C.: Apr. 
3, 2012); Department of Veterans Affairs, Office of Inspector General 
Office of Healthcare Inspections, Combined Assessment Program Review 
of the Tennessee Valley Healthcare System, Nashville, Tennessee, 
Report No. 12-02185-288 (Washington, D.C.: Sept. 27, 2012); Department 
of Veterans Affairs, Office of Inspector General Office of Healthcare 
Inspections, Combined Assessment Program Review of the VA North Texas 
Health Care System, Dallas, Texas, Report No.12-01875-245 (Washington, 
D.C.: Aug. 13, 2012); and Department of Veterans Affairs, Office of 
Inspector General Office of Healthcare Inspections, Combined 
Assessment Program Review of the VA Maine Healthcare System, Augusta, 
Maine, Report No. 12-03741-61 (Washington, D.C.: Dec. 12, 2012). 

[47] Department of Veterans Affairs, Office of Inspector General, 
Healthcare Inspection: Veterans Integrated Service Network Oversight 
of Peer Review Processes, Report No. 08-00338-115 (Washington, D.C.: 
Apr. 22, 2008). 

[48] VHA Directive 2008-004. 

[49] [hyperlink, http://www.gao.gov/products/GAO-01-1008G]. 

[50] Department of Veterans Affairs, Office of Inspector General, 
Healthcare Inspection: Evaluation of Quality Management in Veterans 
Health Administration Facilities Fiscal Year 2009, Report No. 09-00069-
161 (Washington, D.C.: June 2, 2010). OIG officials confirmed that 
none of the four VAMCs included in our site visits were part of their 
sample of 44 VAMCs in the fiscal year 2009 Combined Assessment Program 
reviews. 

[51] The VA OIG found that 35 of the 44 VAMC peer review committees 
submitted quarterly reports to their medical executive committees as 
required. See VA OIG Report No. 09-00069-161. 

[52] Department of Veterans Affairs, Office of Inspector General, 
Combined Assessment Program Summary Report, Evaluation of Quality 
Management in Veterans Health Administration Facilities Fiscal Year 
2011, Report No. 11-00104-186 (Washington, D.C.: May 17, 2012). See 
also Department of Veterans Affairs, Office of Inspector General, 
Combined Assessment Program Summary Report, Evaluation of Quality 
Management in Veterans Health Administration Facilities, Fiscal Year 
2012, Report No. 12-01480-183 (Washington, D.C.: Apr. 25, 2013). 

[53] Provider profiles consist of practitioner-specific data utilized 
to assist service chiefs and medical staff leadership in the 
privileging process. These profiles contain physician performance 
information collected through VAMC continuous monitoring efforts. 

[54] From three of the four VAMCs, we received documentation for a 
total of 12 FPPEs from fiscal years 2009 through 2011 conducted in 
response to questions that arose regarding a provider's ability to 
provide safe, quality patient care. 

[55] According to VAMC officials, although AIBs can be used to address 
issues of clinical competence, they are not usually used in these 
cases. They said a protected peer review or another nonprotected 
review process, such as an FPPE, would more likely be used to address 
clinical competence issues. 

[56] We identified five AIBs used to investigate potential problems 
with a provider's patient care occurring during fiscal years 2009 
through 2011 from two of the four VAMCs we visited. 

[57] According to VHA's credentialing and privileging policy, the 
DUSHOM's monitoring of credentialing and privileging must continue 
through periodic site visits by The Joint Commission and other reviews 
as applicable. See VHA Directive 1100.19. VHA's Director of 
Credentialing and Privileging told us that VHA policy does not specify 
a process or authority for monitoring FPPEs. However, although the 
policy does not mention monitoring of FPPEs specifically, FPPEs are 
part of the overall credentialing and privileging process. 

[58] VA policy affords due process to providers prior to any 
privileging action. Providers are given the right to reply to a 
proposed privileging action prior to a final action. Once the VAMC 
director makes a final determination on a privileging action, the 
provider is entitled to a fair hearing by a panel of peers. If a 
privileging action is sustained following a fair hearing, providers 
have the right to appeal the decision to the VISN director, who may 
ultimately sustain or reverse the privileging action. 

[59] According to a VA OIG official, these policy elements include 
whether there is evidence that FPPEs for new providers or new 
privileges are initiated, completed, and reported to the medical 
executive committee. 

[60] [hyperlink, http://www.gao.gov/products/GAO-01-1008G]. 

[61] [hyperlink, http://www.gao.gov/products/GAO-12-483]. 

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]