Skip to main content

VA Dialysis Pilot: Increased Attention to Planning, Implementation, and Performance Measurement Needed to Help Achieve Goals

GAO-12-584 Published: May 23, 2012. Publicly Released: May 23, 2012.
Jump To:
Skip to Highlights

Highlights

What GAO Found

GAO found a number of weaknesses in the Department of Veterans Affairs’ (VA) execution of the planning and early implementation phases of the Dialysis Pilot. These weaknesses involved pilot location selection, cost estimation practices, and cost savings calculations that could collectively limit the achievement of the pilot’s goals. Specifically, VA did not do the following:

  • Appropriately document its pilot location selection process. VA did not maintain a clear and transparent pilot location selection process; it did not document how its criteria for pilot location selection were applied to all 153 VA medical centers (VAMC) or why substitutions in pilot locations were made. However, VA officials reported that several criteria, including dialysis patient prevalence and average treatment costs, were used to select the pilot locations in Durham and Fayetteville, North Carolina; Philadelphia, Pennsylvania; and Cleveland, Ohio.

  • Produce consistent and comparable cost estimates for pilot locations. VA did not complete consistent and comparable cost estimates for the four pilot locations. Specifically, GAO found several cases where pilot locations did not complete reliable cost estimates because they made changes to formulas and assumptions of the Dialysis Pilot cost estimation model, which was developed by VA systems redesign engineers.

  • Provide clear and timely guidance on start-up fund repayment and cost savings calculations. VA did not provide Veterans Integrated Service Network and VAMC officials with clear and timely written guidance or instructions on how to pay back start-up funds, or how to calculate cost savings generated by the pilot locations.

VA Central Office has not yet determined how it will achieve its goals for the Dialysis Pilot or created clear performance measures for the pilot locations. Previously, GAO found that leading public sector organizations take three steps to improve their performance and help their organizations become more effective: (1) define a clear mission and goals, (2) measure performance to gauge progress toward achieving goals, and (3) use performance information as a basis for decision making. While VA has defined a clear mission and goals for the Dialysis Pilot, it has only made limited progress in the remaining two steps. In March 2012, VA reported that it was in the early stages of establishing an agreement with a leading university research center to conduct an evaluation of the Dialysis Pilot; however, no target dates were provided for when this evaluation would begin or what aspects of the Dialysis Pilot it would evaluate. Because VA has not yet developed an evaluation plan or defined performance measures for pilot locations, it is not collecting consistent and reliable information on the performance of the pilot locations and thus may not have this information available when it is time to either make midcourse corrections to the Dialysis Pilot or decide whether and how to open additional VA-operated outpatient dialysis clinics. VA officials also told GAO they have developed a limited plan for expanding the Dialysis Pilot despite not having access to performance information on the existing four pilot locations.

Why GAO Did This Study

Veterans diagnosed with end-stage renal disease—a condition of permanent kidney failure—represent one of the most resource-intensive patient populations at VA. These veterans are often prescribed dialysis, which is a life-saving and relatively expensive medical procedure that removes excess fluids and toxins from the bloodstream. VA began developing its Dialysis Pilot in 2009 with four goals: (1) improved quality of care, (2) increased veteran access, (3) additional medical research opportunities, and (4) cost savings. Through this pilot, VA will establish four VA-operated outpatient dialysis clinics in communities surrounding select VA medical centers by the end of fiscal year 2012 using start-up funding provided by VA Central Office. Pilot locations are expected to achieve cost savings and to repay their start-up funding. GAO examined VA’s planning and early implementation efforts for the Dialysis Pilot, and how VA plans to evaluate the pilot.

GAO reviewed relevant VA documents, including those related to pilot location selection and cost estimation, and spoke with VA officials responsible for overseeing the Dialysis Pilot and representatives from all pilot locations.

Recommendations

Among other actions, GAO recommends that VA improve its Dialysis Pilot by providing guidance for start-up fund repayment, as well as developing an evaluation plan that includes performance measures for the pilot locations. VA concurred with GAO’s recommendations and provided an action plan to address them.

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Veterans Affairs In order to increase VA's attention to planning, implementation and performance measurement of the Dialysis Pilot, and to improve VA's communication related to the Dialysis Pilot, the Secretary of Veterans Affairs should direct the Under Secretary for Health to ensure that key decisions made regarding pilot location selection and efforts to continue or expand the Dialysis Pilot are clearly documented.
Closed – Implemented
In February 2013, VA developed a written plan entitled "Site Selection Methodology: VA Free-Standing Dialysis Clinics" describing the methodology VA will use to select potential future free-standing dialysis clinic locations. This document states that the methodology relies on the VA Geospatial Dialysis Tool, which identified and prioritizes candidate sites for a dialysis clinic based on various factors including the catchment size, mean commute distance, growth rate in fee basis dialysis expenditure at the parent VAMC, and availability of non-VA dialysis vendors. In developing this methodology document, VA cited GAO's recommendation to ensure that decisions regarding selection of dialysis clinic locations are clearly documented and transparent.
Department of Veterans Affairs In order to increase VA's attention to planning, implementation and performance measurement of the Dialysis Pilot, and to ensure that reasonable cost estimates are created for the Dialysis Pilot and other similar programs, the Secretary of Veterans Affairs should direct the Under Secretary for Health to restrict or evaluate changes made to cost estimation models at the Veterans Integrated Service Network (VISN) and VAMC levels that affect pilot development and analysis.
Closed – Implemented
In September 2012, VA awarded a 3-year contract to an evaluation contractor to provide an independent evaluation of its Dialysis Pilot including VA's make-buy model. As of June 2014, this contractor produced three reports evaluating changes to VA's make-buy model. Furthermore, in its make-buy model user's guide, in response to GAO's recommendation, VA also established a change control process requiring VA officials to complete a change control request form describing any proposed change to the make-buy model for approval by relevant entities in VA.
Department of Veterans Affairs In order to increase VA's attention to planning, implementation and performance measurement of the Dialysis Pilot, and to ensure that start-up funds are repaid and cost savings are accurately calculated for the Dialysis Pilot, the Secretary of Veterans Affairs should direct the Under Secretary for Health to develop written guidance about expectations for the repayment of start-up funds and how the cost savings generated by the four pilot locations should be calculated.
Closed – Implemented
As of July 2012, as part of its business rules for the Dialysis Pilot, VHA provided written guidance defining expectations regarding repayment of start-up funds and the methodology used to compute cost savings generated by the pilots. Based on documents received from VA in May 2014, all four dialysis pilots have repaid the fiscal year (FY) 2010 seed funding of $2.5 million each pilot received in start-up funds in FY 2012 and FY 2014 by returning funds to VA Central Office. This written confirmation was also followed by a conference call between VA's Office of Finance and GAO to describe the mechanism by which start-up funds were repaid.
Department of Veterans Affairs In order to increase VA's attention to planning, implementation and performance measurement of the Dialysis Pilot, and to ensure that VA Central Office effectively evaluates the Dialysis Pilot, the Secretary of Veterans Affairs should direct the Under Secretary for Health to develop an evaluation plan that outlines how the Dialysis Pilot will be assessed and provides target dates for the completion of this assessment.
Closed – Implemented
By February 2013, VA had developed an evaluation plan for its Dialysis Pilot, which laid out the framework for assessing the performance of the dialysis pilot clinics and evaluate whether the pilot goals were accomplished. Also, in September 2012 VA awarded a three year contract to the University of Michigan to provide annual and quarterly reports and evaluations on the various aspects of the Dialysis Pilot sites through September 2015.
Department of Veterans Affairs In order to increase VA's attention to planning, implementation and performance measurement of the Dialysis Pilot, and to ensure that VA Central Office effectively evaluates the Dialysis Pilot, the Secretary of Veterans Affairs should direct the Under Secretary for Health to develop clear measures for assessing the performance of the four Dialysis Pilot locations in key areas-including quality, access, and cost.
Closed – Implemented
By February 2013, VA had developed an evaluation plan for its Dialysis Pilot, which identified performance measures for clinical quality, patient satisfaction, access, and cost. Specifically, the Evaluation Plan identified and described fifteen dialysis clinic performance measures eleven clinical quality metrics (three outcome, eight process), one patient satisfaction measure, two access measures, and one cost measure and characterized them by the metric goal, definition, performance standard, evaluation period, type of metric, level of analysis, measure source, and data source. VA also completed reports assessing each pilot location's performance in across these measures for calendar years 2012, 2013, and 2014. Also, in September 2012 VA awarded a three year contract to the University of Michigan to provide annual and quarterly reports and evaluations on the various aspects of the Dialysis Pilot sites through September 2015.

Full Report

GAO Contacts

Office of Public Affairs

Topics

Veterans medical centersDialysisVeteransCost savingsVeterans affairsMedicareCost estimatesPerformance measurementEngineersPatient care