Skip to main content

Veterans Health Administration: Past Performance System Recommendations Have Not Been Implemented

GAO-19-350 Published: Apr 30, 2019. Publicly Released: Apr 30, 2019.
Jump To:

Fast Facts

The Veterans Health Administration ranks medical centers on overall quality from 1 star to 5 stars. We found that these rankings are based on a performance measurement system that may not effectively measure quality of care.

Two previous reviews of VHA's system—one internal VHA review and one review from a private consulting organization—recommended that VHA make significant changes to it, but it has neither assessed those recommendations nor implemented them. In this report, we recommended that VHA use those previous reviews to improve their system.

VA health care has been on our High Risk List since 2015.

Number of VA Medical Centers at Each Star Rating, Fiscal Year 2018

Pictograph of VA medical centers' star ratings in FY2018. 1 star=9; 2 stars=35; 3 stars=56; 4 stars=28; 5 stars=18.

Pictograph of VA medical centers' star ratings in FY2018. 1 star=9; 2 stars=35; 3 stars=56; 4 stars=28; 5 stars=18.

Skip to Highlights

Highlights

What GAO Found

Department of Veterans Affairs' (VA) Veterans Health Administration (VHA) officials told GAO they primarily use the Strategic Analytics for Improvement and Learning (SAIL) system to assess VA medical center performance. SAIL includes 27 quality measures in areas such as acute care mortality and access to care. VHA officials use SAIL to calculate and assign each medical center an annual star rating of 1 (lowest) to 5 (highest) stars as an assessment of overall quality. For the 146 medical centers that received star ratings in fiscal year 2018, the distribution of star ratings was as follows: 6 percent, 1 star; 24 percent, 2 stars; 38 percent, 3 stars; 19 percent, 4 stars; and 12 percent, 5 stars. Although the specific medical centers within each star-rating category could change from year to year, GAO found that the fiscal year 2018 star ratings for 110 of the 127 medical centers (87 percent) that received star ratings in fiscal year 2013 did not differ by more than 1 star from their fiscal year 2013 rating.

Changes in VHA Strategic Analytics for Improvement and Learning Star Ratings, Fiscal Year 2013 Compared to Fiscal Year 2018

U:\Work in Process\Teams\FY19 Reports\HC\102360_350\Master Graphic Files\TIFF files\HL_5-v5_102360.tif

GAO found that VHA's appraisal process for assessing medical center director performance relies heavily on medical center performance information, including SAIL. For example, the most heavily weighted appraisal element (40 percent of the overall rating) is made up entirely of medical center performance information.

SAIL was evaluated in 2014 and 2015, but VHA has not assessed the recommendations from those evaluations, or taken action on them. The evaluations, which found issues related to the validity and reliability of SAIL and its star ratings for measuring performance and fostering accountability, together included more than 40 recommendations for improving SAIL. The findings are similar to concerns expressed by officials GAO interviewed from VHA, networks, and medical centers about SAIL's effectiveness and how it is currently being used to assess medical center performance. VHA officials told GAO the findings and recommendations of the previous SAIL evaluations were not assessed because the evaluation reports were not widely distributed within VHA due to leadership turnover, as well as attention that was diverted to other concerns such as extensive wait times for medical appointments. Without ensuring that the recommendations resulting from these previous evaluations are assessed and implemented as appropriate, the identified deficiencies may not be adequately resolved, and VHA's ability to hold officials accountable for taking the necessary actions may be diminished.

VHA anticipates that it will provide care to more than 7 million veterans in fiscal year 2019. The majority of veterans using VHA health care services receive care in one or more of the 172 medical centers or at associated outpatient facilities. VHA collects an extensive amount of data that can be used to assess and manage the performance of medical centers. Many measures are publicly reported on VA web pages, allowing veterans the ability to compare medical centers' quality of care.

GAO was asked to assess VHA's management of medical center performance. This report examines (1) the tools VHA uses to assess medical center performance; (2) VHA's use of medical center performance information to assess medical center directors; and (3) the extent to which VHA has evaluated the effectiveness of the SAIL system.

GAO reviewed VHA policies, guidance, and performance information for medical centers and their associated directors. GAO also interviewed officials from VHA as well as from four VA medical centers, selected for variation in performance and geographic location.

Recommendations

GAO recommends that the Under Secretary for Health: (1) assess recommendations from previous evaluations of SAIL for implementation; and (2) implement, as appropriate, recommendations resulting from the assessment. VA concurred with GAO's recommendations and identified actions it is taking to implement them.

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Veterans Affairs The Under Secretary for Health should assess recommendations from two previous evaluations of SAIL. This assessment should include the documentation of decisions about which recommendations to implement and assignment of officials or offices as responsible for implementing them. (Recommendation 1)
Closed – Implemented
VA concurred with this recommendation, and in January 2020, provided an update on its actions to implement it. Specifically, VA created a SAIL Governance Group, which first met in May 2019 and reports to the Board of Governors (formerly the National Leadership Council). The Governance Group is responsible for overseeing changes to SAIL. The Governance Group discussed the previous evaluations of SAIL and their recommendations and determined which were still relevant to implement in June 2019. As a result, we consider this recommendation implemented.
Department of Veterans Affairs The Under Secretary for Health should implement, as appropriate, recommendations resulting from the assessment of the two previous SAIL evaluations. (Recommendation 2)
Closed – Implemented
VA concurred with this recommendation, and in January 2020, provided an update on its actions to implement it. After the SAIL Governance Group determined, in June 2019, which recommendations from the previous evaluations of SAIL were still relevant, VA took action to implement changes to how SAIL is used. For example, in December 2019, VA Secretary Wilkie announced that the SAIL star-ratings had been eliminated. Also, in January 2020, VA provided documentation showing that it has reduced the weight of SAIL measures for the fiscal year 2020 VA medical center directors and network directors' performance plans. These actions address recommendations from the previous SAIL evaluations, and as a result, we consider this recommendation implemented.

Full Report

Office of Public Affairs

Topics

Agency evaluationsHealth care administrationHealth care centersHealth care standardsMedical facilitiesPast performancePatient carePerformance appraisalsPerformance managementPerformance measurementQuality of careVeteransVeterans affairsVeterans health careVeterans hospitalsVeterans medical centers