The Veterans Health Administration ranks medical centers on overall quality from 1 star to 5 stars. We found that these rankings are based on a performance measurement system that may not effectively measure quality of care.
Two previous reviews of VHA's system—one internal VHA review and one review from a private consulting organization—recommended that VHA make significant changes to it, but it has neither assessed those recommendations nor implemented them. In this report, we recommended that VHA use those previous reviews to improve their system.
VA health care has been on our High Risk List since 2015.
Number of VA Medical Centers at Each Star Rating, Fiscal Year 2018
Pictograph of VA medical centers' star ratings in FY2018. 1 star=9; 2 stars=35; 3 stars=56; 4 stars=28; 5 stars=18.
What GAO Found
Department of Veterans Affairs' (VA) Veterans Health Administration (VHA) officials told GAO they primarily use the Strategic Analytics for Improvement and Learning (SAIL) system to assess VA medical center performance. SAIL includes 27 quality measures in areas such as acute care mortality and access to care. VHA officials use SAIL to calculate and assign each medical center an annual star rating of 1 (lowest) to 5 (highest) stars as an assessment of overall quality. For the 146 medical centers that received star ratings in fiscal year 2018, the distribution of star ratings was as follows: 6 percent, 1 star; 24 percent, 2 stars; 38 percent, 3 stars; 19 percent, 4 stars; and 12 percent, 5 stars. Although the specific medical centers within each star-rating category could change from year to year, GAO found that the fiscal year 2018 star ratings for 110 of the 127 medical centers (87 percent) that received star ratings in fiscal year 2013 did not differ by more than 1 star from their fiscal year 2013 rating.
Changes in VHA Strategic Analytics for Improvement and Learning Star Ratings, Fiscal Year 2013 Compared to Fiscal Year 2018
GAO found that VHA's appraisal process for assessing medical center director performance relies heavily on medical center performance information, including SAIL. For example, the most heavily weighted appraisal element (40 percent of the overall rating) is made up entirely of medical center performance information.
SAIL was evaluated in 2014 and 2015, but VHA has not assessed the recommendations from those evaluations, or taken action on them. The evaluations, which found issues related to the validity and reliability of SAIL and its star ratings for measuring performance and fostering accountability, together included more than 40 recommendations for improving SAIL. The findings are similar to concerns expressed by officials GAO interviewed from VHA, networks, and medical centers about SAIL's effectiveness and how it is currently being used to assess medical center performance. VHA officials told GAO the findings and recommendations of the previous SAIL evaluations were not assessed because the evaluation reports were not widely distributed within VHA due to leadership turnover, as well as attention that was diverted to other concerns such as extensive wait times for medical appointments. Without ensuring that the recommendations resulting from these previous evaluations are assessed and implemented as appropriate, the identified deficiencies may not be adequately resolved, and VHA's ability to hold officials accountable for taking the necessary actions may be diminished.
Why GAO Did This Study
VHA anticipates that it will provide care to more than 7 million veterans in fiscal year 2019. The majority of veterans using VHA health care services receive care in one or more of the 172 medical centers or at associated outpatient facilities. VHA collects an extensive amount of data that can be used to assess and manage the performance of medical centers. Many measures are publicly reported on VA web pages, allowing veterans the ability to compare medical centers' quality of care.
GAO was asked to assess VHA's management of medical center performance. This report examines (1) the tools VHA uses to assess medical center performance; (2) VHA's use of medical center performance information to assess medical center directors; and (3) the extent to which VHA has evaluated the effectiveness of the SAIL system.
GAO reviewed VHA policies, guidance, and performance information for medical centers and their associated directors. GAO also interviewed officials from VHA as well as from four VA medical centers, selected for variation in performance and geographic location.
GAO recommends that the Under Secretary for Health: (1) assess recommendations from previous evaluations of SAIL for implementation; and (2) implement, as appropriate, recommendations resulting from the assessment. VA concurred with GAO's recommendations and identified actions it is taking to implement them.
Recommendations for Executive Action
|Department of Veterans Affairs||The Under Secretary for Health should assess recommendations from two previous evaluations of SAIL. This assessment should include the documentation of decisions about which recommendations to implement and assignment of officials or offices as responsible for implementing them. (Recommendation 1)|
|Department of Veterans Affairs||The Under Secretary for Health should implement, as appropriate, recommendations resulting from the assessment of the two previous SAIL evaluations. (Recommendation 2)|