This is the accessible text file for GAO report number GAO-14-646 entitled 'VA Dialysis Pilot: Documentation of Plans for Concluding the Pilot Needed to Improve Transparency and Accountability' which was released on September 2, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to the Ranking Member, Committee on Veterans' Affairs, United States Senate: September 2014: VA Dialysis Pilot: Documentation of Plans for Concluding the Pilot Needed to Improve Transparency and Accountability: GAO-14-646: GAO Highlights: Highlights of GAO-14-646, a report to the Ranking Member, Committee on Veterans' Affairs, United States Senate. Why GAO Did This Study: Veterans with end-stage renal disease-—a condition of permanent kidney failure-—are one of the most resource-intensive patient populations served by the VA and are generally prescribed a life-saving medical procedure called dialysis. In 2009, VA began developing a pilot program at four VAMCs to provide dialysis to veterans in VA-operated, free-standing dialysis clinics largely in an effort to stem rising costs for providing such care in the private sector through the Non-VA Medical Care Program. In May 2012, GAO issued a report identifying several weaknesses in VA's execution of the planning and early implementation phases of the Dialysis Pilot. GAO was asked to continue its evaluation of the Dialysis Pilot. GAO examined the extent to which VA documented plans for concluding the Dialysis Pilot and the status of data on the quality of care and treatment costs for the four pilot locations. GAO reviewed relevant documents from VA and the evaluation contractor selected by VA to perform an independent analysis of the pilot locations. GAO also spoke with VA officials responsible for managing the Dialysis Pilot, representatives from all four pilot locations, and evaluation contractor officials responsible for reviewing the performance of the four pilot locations. What GAO Found: Five years into the Dialysis Pilot, the Department of Veterans Affairs (VA) Central Office still has not set a timetable for completing the pilot or documented how it will determine the success of the pilot locations. GAO previously identified best practices that state that a project timeline is critical for managing and measuring an entity's performance on projects and that choosing and documenting well- regarded criteria that are used to make comparisons can lead to strong, defensible conclusions. Initially, VA planned to conclude the Dialysis Pilot after the pilot locations were all open for 5 years. However, in March 2014, VA officials told GAO they are no longer operating under this timeline but instead plan to conclude the pilot once the pilot locations achieve (1) the creation of a model for a VA- operated, free-standing dialysis clinic that can be replicated by other VA medical centers (VAMC) and (2) the confirmation of the time necessary for a pilot location to reach a “breakeven point.” VA considers that a pilot location has achieved a breakeven point when it repays its start-up funding and the VAMC realizes a cost savings because its treatment cost for dialysis at the pilot location is lower than purchasing care from non-VA dialysis providers. However, VA has not formally documented these pilot location achievements as criteria for concluding the Dialysis Pilot. By not doing so, the transparency of VA's management decisions on pilot location outcomes is compromised and the Department lacks accountability for ensuring the success of the Dialysis Pilot. The Dialysis Pilot has been under evaluation for 2 years by VA and the contractor it selected to conduct an independent analysis of pilot location quality of care and treatment costs. However, neither has concluded its evaluation. Specifically, VA noted that the delayed openings and initial operational issues of two pilot locations— Philadelphia, Pennsylvania, and Cleveland, Ohio—led to limited data availability, and it recommended another 12 months of data be collected on these two pilot locations before drawing conclusions. Table: Data Available from the Department of Veterans Affairs (VA) and Evaluation Contractor on Quality of Care and Treatment Costs for Dialysis Pilot Locations, Calendar Years 2012 and 2013: Calendar year: 2012: Review type: Quality of care; Evaluator: VA; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were not included in reviews; Cleveland, OH: Data were not included in reviews. Review type: Quality of care; Evaluator: Contractor; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were not included in reviews; Cleveland, OH: Data were not included in reviews. Review type: Treatment cost Evaluator: VA; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were not included in reviews; Cleveland, OH: Data were not included in reviews. Review type: Treatment cost Evaluator: Contractor; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were not included in reviews; Cleveland, OH: Data were not included in reviews. Calendar year: 2013: Review type: Quality of care; Evaluator: VA; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were included in reviews; Cleveland, OH: Data were included in reviews. Review type: Quality of care; Evaluator: Contractor; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were included in reviews; Cleveland, OH: Data were not included in reviews. Review type: Treatment cost Evaluator: VA; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were included in reviews; Cleveland, OH: Data were included in reviews. Review type: Treatment cost Evaluator: Contractor; Pilot location: Raleigh, NC: Reviews were not available at the time of GAO's analysis; Fayetteville, NC: Reviews were not available at the time of GAO's analysis; Philadelphia, PA: Reviews were not available at the time of GAO's analysis; Cleveland, OH: Reviews were not available at the time of GAO's analysis. Source: GAO analysis of VA and evaluation contractor data. GAO-14-646. [End of table] What GAO Recommends: GAO recommends that VA document its plans for concluding the Dialysis Pilot. VA concurred with GAO's recommendation but did not clearly delineate its plans for pilot conclusion. View [hyperlink, http://www.gao.gov/products/GAO-14-646]. For more information, contact Randall B. Williamson at (202) 512-7114 or willamsonr@gao.gov. [End of section] Contents: Letter: Background: VA Has Not Documented Criteria or Plans for Concluding the Dialysis Pilot: Current Data on Quality of Care and Treatment Costs Are Limited Due to Ongoing Evaluations: Conclusions: Recommendation: Agency Comments and Our Evaluation: Appendix I: Results of Clinical Quality Review by the Department of Veterans Affairs: Appendix II: Results of Patient Satisfaction Review by the Department of Veterans Affairs: Appendix III: Results of Access to Care Review by the Department of Veterans Affairs: Appendix IV: Results of Clinical Quality Review by the Evaluation Contractor: Appendix V: Results of Treatment Cost Reviews by the Department of Veterans Affairs and the Evaluation Contractor: Appendix VI: Comments from the Department of Veterans Affairs: Appendix VII: GAO Contact and Staff Acknowledgments: Tables: Table 1: Data Available from the Department of Veterans Affairs (VA) and Evaluation Contractor on Quality of Care and Treatment Costs for Dialysis Pilot Locations, Calendar Years 2012 and 2013: Table 2: Department of Veterans Affairs' (VA) Results of Centers for Medicare & Medicaid Services (CMS) Clinical Quality Measures for Dialysis Pilot Locations, Calendar Years 2012 and 2013: Table 3: Department of Veterans Affairs' (VA) Results of Patient Satisfaction Surveys for the Raleigh, North Carolina, Dialysis Pilot Location, June 2011 through September 2013: Table 4: Department of Veterans Affairs' (VA) Results of Patient Satisfaction Surveys for the Fayetteville, North Carolina, Dialysis Pilot Location, June 2011 through September 2013: Table 5: Department of Veterans Affairs' (VA) Results of Veterans' Access to Care Review for the Raleigh and Fayetteville, North Carolina, Dialysis Pilot Locations, Calendar Year 2012: Table 6: Department of Veterans Affairs' (VA) Results of Veterans' Access to Care Review for the Dialysis Pilot Locations, Calendar Year 2013: Table 7: Evaluation Contractor's 2013 Dialysis Facility Report Clinical Quality Measures for the Raleigh and Fayetteville Department of Veterans Affairs (VA) Dialysis Pilot Locations Compared to North Carolina and U.S. Averages, Calendar Years 2009 to 2012: Table 8: Evaluation Contractor's 2013 Dialysis Facility Report Clinical Quality Measures for the Philadelphia Department of Veterans Affairs (VA) Dialysis Pilot Location Compared to Pennsylvania and U.S. Averages, Calendar Years 2009 to 2012: Table 9: Average Cost Per Treatment for Department of Veterans Affairs (VA) Dialysis Pilot Locations and Non-VA Dialysis Providers as Calculated by VA and the Evaluation Contractor, Calendar Years 2012 and 2013: Abbreviations: CMS: Centers for Medicare & Medicaid Services: DFR: Dialysis Facility Report: ESRD: end-stage renal disease: VA: Department of Veterans Affairs: VAMC: Veterans Affairs medical center: VHA: Veterans Health Administration: VISN: Veterans Integrated Service Network: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: September 2, 2014: The Honorable Richard Burr: Ranking Member: Committee on Veterans' Affairs: United States Senate: Dear Mr. Burr: Veterans diagnosed with end-stage renal disease (ESRD)--a condition of permanent kidney failure--are one of the most resource-intensive patient populations served by the Department of Veterans Affairs (VA) Veterans Health Administration (VHA).[Footnote 1] Veterans with ESRD are generally prescribed dialysis, which is a life-saving medical procedure that removes excess fluids and toxins from the bloodstream. [Footnote 2] Dialysis treatments are time intensive. Specifically, veterans receiving dialysis treatments typically must receive three outpatient treatments per week with each treatment lasting about four hours. Veterans usually remain on dialysis for the rest of their lives unless they receive a kidney transplant. Due to VA's limited dialysis capacity in its own facilities, VA most commonly provides dialysis to veterans by referring them to non-VA dialysis providers in their local communities through the Non-VA Medical Care Program (non-VA medical care).[Footnote 3] In fiscal year 2013, VA referred about 14,000 veterans to non-VA dialysis providers at a cost of about $425 million. [Footnote 4] In 2009, VA began developing a pilot program at several VA medical centers (VAMC) to provide dialysis to veterans in VA-operated dialysis clinics in response to several issues--including an increasing number of veterans needing dialysis, rising costs of providing dialysis through non-VA medical care, and unsuccessful efforts to lower these costs.[Footnote 5] The resulting Dialysis Pilot was approved by the Secretary of VA in September 2010. Ultimately, VA believes the Dialysis Pilot will lead to the development of a business model with a mission of providing more veterans' dialysis treatments in VA facilities and less through non-VA medical care. Through this pilot, VA established four VA-operated dialysis clinics located near sponsoring VAMCs in Durham and Fayetteville, North Carolina; Cleveland, Ohio; and Philadelphia, Pennsylvania.[Footnote 6] The Raleigh and Fayetteville pilot locations opened in June 2011, the Philadelphia pilot location opened in October 2012, and the Cleveland pilot location opened in July 2013. In May 2012, we issued a report on VA's Dialysis Pilot that identified several weaknesses in VA's execution of the planning and early implementation phases of the pilot.[Footnote 7] Specifically, we found VA did not appropriately document its pilot location selection process, produce consistent and comparable cost estimates for pilot locations, provide clear and timely written guidance on how to repay start-up funding to VA Central Office, and provide guidance on how to calculate cost savings generated by the pilot locations.[Footnote 8] Due to continuing congressional concern about issues raised in our prior report, you asked us to continue our evaluation of VA's Dialysis Pilot. In this report, we examine (1) the extent to which VA has documented plans for concluding the Dialysis Pilot and (2) the status of data on the quality of care and treatment costs for the four pilot locations. To examine the extent to which VA has documented plans for concluding the Dialysis Pilot, we interviewed VA Central Office officials and VAMC officials responsible for managing pilot locations regarding VA's plans for the completion of the Dialysis Pilot and reviewed relevant VA documentation. These documents included a VA business analysis for the Dialysis Pilot and VA's contract with a leading university research center (evaluation contractor) to review the Dialysis Pilot locations' quality of care and treatment costs.[Footnote 9] To examine the status of data on the quality of care and treatment costs for the four pilot locations, we interviewed VA Central Office officials and officials from the evaluation contractor regarding the status and initial findings of their evaluations. We analyzed documents that VA and the evaluation contractor produced regarding the quality of care--including clinical quality, patient satisfaction, and access to care--and treatment costs at the pilot locations compared with non-VA dialysis providers. We conducted this performance audit from January 2014 to July 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: VA set four goals for the Dialysis Pilot: (1) improve the quality of dialysis care veterans receive, (2) increase veterans' access to dialysis care, (3) provide additional dialysis research opportunities, and (4) achieve cost savings for VA-funded dialysis treatments. VA Dialysis Pilot Evaluation: VA is performing an internal evaluation of the performance of the four pilot locations by assessing the quality of care--including clinical quality, patient satisfaction, and access to care--and treatment costs at each pilot location. Results of this evaluation are reported in an annual performance review produced by VA each calendar year. To date, VA has completed reviews for calendar years 2012 and 2013. * Clinical quality: To evaluate the clinical quality at the pilot locations, VA selected 11 clinical quality measures that were derived from measures and associated performance standards established by the Centers for Medicare & Medicaid Services (CMS)--the federal entity with primary responsibility for evaluating the quality of dialysis care in the United States--and endorsed by the National Quality Forum, a leader in evaluating clinical performance measurement for a variety of chronic clinical conditions.[Footnote 10] * Patient satisfaction: To evaluate veterans' satisfaction with the quality of care provided at the pilot locations, VA analyzed the CMS standard dialysis facility survey--the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems--for the veterans treated at pilot locations.[Footnote 11] * Access to care: To evaluate veterans' access to the pilot locations, VA is measuring the percentage of veterans whose actual travel distances are no more than 30 miles from their residence to the pilot location where they receive care.[Footnote 12] * Treatment cost: To evaluate the cost of care, VA is comparing the pilot locations' average cost per treatment and the average cost per treatment for non-VA dialysis providers in each pilot location's Veterans Integrated Service Network (VISN).[Footnote 13] To calculate each pilot location's average cost per treatment, VA divided each pilot location's total health care costs by its total number of treatments.[Footnote 14] To calculate non-VA dialysis providers' average cost per treatment, VA created a composite figure using the average per treatment contract rate from the two largest non-VA dialysis providers located in each pilot location's VISN, a per treatment VA claims processing fee, a per treatment VA physician cost for nephrology oversight, and a per treatment VA financial administrative cost.[Footnote 15] Dialysis Pilot Evaluation Contractor Review: In September 2012, VA awarded a 3-year contract to an evaluation contractor to provide an independent evaluation of its Dialysis Pilot-- including a comparison of the clinical quality provided by the four pilot locations with non-VA dialysis providers and a dialysis per treatment cost comparison between the pilot locations and the Non-VA Medical Care Program.[Footnote 16] * Clinical quality: To evaluate the clinical quality of care provided by the pilot locations, the evaluation contractor is assessing the results of several measures--such as mortality, hospitalization, and infection rates--for the pilot locations and comparing them to relevant state and national results. These results are presented for each pilot location in quarterly quality reviews and an annual dialysis facility report (DFR), a standard report produced by the evaluation contractor for CMS for each of the more than 6,000 private sector dialysis facilities nationwide. * Treatment cost: To evaluate the treatment costs for each pilot location, the evaluation contractor is comparing the pilot locations' average cost per treatment to that of private sector dialysis providers, including certain contracted non-VA dialysis providers. [Footnote 17] Results are presented for each pilot location in quarterly cost reviews.[Footnote 18] The evaluation contractor used a similar methodology as VA to determine each pilot location's average cost per treatment.[Footnote 19] To determine the non-VA dialysis provider average cost per treatment in calendar year 2012, the evaluation contractor created a composite figure that included (1) the contract per treatment average price for the three non-VA dialysis providers with the largest market share in the VISN responsible for the Raleigh and Fayetteville pilot locations, (2) VA's per treatment travel-related costs, and (3) VA's per treatment medical oversight costs.[Footnote 20] VA Has Not Documented Criteria or Plans for Concluding the Dialysis Pilot: Even though it is 5 years into the Dialysis Pilot, VA Central Office has not yet set a timetable for completing the pilot or documented how it will determine the success of the four pilot locations. We have previously outlined best practices for project scheduling that specify project timelines are critical for managing and measuring an agency's performance on projects and that it is necessary for the agency to have a document that details the rationale used in developing a project timeline.[Footnote 21] In addition, We have outlined best practices for designing evaluations that state that choosing and documenting well-regarded criteria that are used to make comparisons can lead to strong, defensible conclusions.[Footnote 22] According to VA's business analysis of the Dialysis Pilot conducted prior to opening the pilot locations, VA planned to conclude the Dialysis Pilot 5 years after the pilot locations opened and assumed all four pilot locations would begin operations at generally the same time.[Footnote 23] Due to delays in opening the pilot locations, the completion of this 5-year period would be in fiscal year 2018--5 years after the opening of the fourth pilot location.[Footnote 24] However, in March 2014, VA officials told us they are no longer operating under the 5-year timeline outlined in this plan, although they have not updated the timeline included in the Dialysis Pilot business analysis. VA officials told us that instead of ending the Dialysis Pilot based on the timeline outlined in VA's business analysis, they believe the Dialysis Pilot will be successful and should conclude once the pilot locations have achieved the following: * Creation of a replicable model. According to VA officials, the first achievement that indicates success is the creation of a model for a VA- operated, free-standing dialysis clinic that can be replicated by other VAMCs. Such a model would include clear documentation of the necessary staff, equipment, time, and resources required to establish a new clinic and serve as a guide to other VAMCs seeking to establish similar clinics. VA officials did not specify a timeline for the creation of this model; however, they did state that they believe the pilot locations have already achieved this milestone. * Pilot location breakeven points. VA officials stated that the second achievement that indicates success is the confirmation of the time necessary for a pilot location to reach its breakeven point--which VA has defined as the point when a pilot location achieves cost savings in the Non-VA Medical Care Program for dialysis services for its sponsoring VAMC and repays its start-up funding to VA Central Office. Through the Dialysis Pilot, sponsoring VAMCs for the pilot locations were each provided about $2.5 million in start-up funding to establish a VA-operated, free-standing dialysis clinic and were to finish repaying this start-up funding to VA Central Office in fiscal year 2014. According to VA Central Office officials, all four pilot locations have repaid their start-up funding as of the beginning of fiscal year 2014 and two pilot locations--Raleigh and Fayetteville-- have demonstrated that their cost per treatment is lower than comparable per treatment costs of non-VA dialysis providers in their areas. Achieving this breakeven point indicates that the pilot locations are no longer incurring additional costs for VA and are fulfilling VA's goal of providing dialysis treatments to veterans at a lower cost than similar care provided through non-VA dialysis providers. While VA officials indicated they could conclude the Dialysis Pilot once the pilot locations had created a replicable model and realized their breakeven points, VA has not formally communicated these achievements in writing as criteria for concluding the pilot. Moreover, these criteria were not included in the performance work statement for the evaluation contractor, and according to a VA official, have not been formally recorded in any official document. By not clearly communicating these milestones as criteria for concluding the Dialysis Pilot, the transparency of VA's management decisions on pilot location outcomes is compromised and the Department lacks accountability for ensuring the success of the Dialysis Pilot. Current Data on Quality of Care and Treatment Costs Are Limited Due to Ongoing Evaluations: VA and the evaluation contractor have been evaluating the Dialysis Pilot for 2 years; however, neither has concluded its evaluation of the Dialysis Pilot and data on two of the pilot locations-- Philadelphia and Cleveland--is limited due to their delayed opening. Table 1 shows the data available for each pilot location as of June 2014. Table 1: Data Available from the Department of Veterans Affairs (VA) and Evaluation Contractor on Quality of Care and Treatment Costs for Dialysis Pilot Locations, Calendar Years 2012 and 2013: Calendar year: 2012: Review type: Quality of care; Evaluator: VA; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were not included in reviews; Cleveland, OH: Data were not included in reviews. Review type: Quality of care; Evaluator: Contractor; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were not included in reviews; Cleveland, OH: Data were not included in reviews. Review type: Treatment cost Evaluator: VA; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were not included in reviews; Cleveland, OH: Data were not included in reviews. Review type: Treatment cost Evaluator: Contractor; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were not included in reviews; Cleveland, OH: Data were not included in reviews. Calendar year: 2013: Review type: Quality of care; Evaluator: VA; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were included in reviews; Cleveland, OH: Data were included in reviews. Review type: Quality of care; Evaluator: Contractor; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were included in reviews; Cleveland, OH: Data were not included in reviews. Review type: Treatment cost Evaluator: VA; Pilot location: Raleigh, NC: Data were included in reviews; Fayetteville, NC: Data were included in reviews; Philadelphia, PA: Data were included in reviews; Cleveland, OH: Data were included in reviews. Review type: Treatment cost Evaluator: Contractor; Pilot location: Raleigh, NC: Reviews were not available at the time of GAO's analysis; Fayetteville, NC: Reviews were not available at the time of GAO's analysis; Philadelphia, PA: Reviews were not available at the time of GAO's analysis; Cleveland, OH: Reviews were not available at the time of GAO's analysis. Source: GAO analysis of VA and evaluation contractor data. GAO-14-646. Note: The VA review includes information on quality of care--including clinical quality, patient satisfaction, and access to care--and treatment costs at pilot locations compared to non-VA dialysis providers. VA contracted with a leading university research center to evaluate the Dialysis Pilot. This evaluation contractor's review includes information on clinical quality provided by the pilot locations and private sector free-standing dialysis providers and a comparison of the treatment costs for dialysis care through the pilot locations and the Non-VA Medical Care Program. The evaluation contractor's quality of care data included in this table are from the Dialysis Facility Reports, which are standard reports produced by the evaluation contractor for the Centers for Medicare & Medicaid Services for each of the more than 6,000 private sector dialysis facilities nationwide. [End of table] To date, VA has produced two annual performance reviews on quality of care and treatment costs for the Dialysis Pilot--the first for calendar year 2012 and the second for calendar year 2013--that include the results of pilot locations' clinical quality, patient satisfaction, access to care, and treatment cost performance. The VA calendar year 2012 annual performance review included results for the Raleigh and Fayetteville pilot locations only. VA included results for all four pilot locations in its calendar year 2013 annual performance review; however, this review does not include a full year of data for the Cleveland pilot location and VA noted operational challenges that affected the data for the Philadelphia pilot location. VA noted several contributing factors to the limited data available on the Cleveland and Philadelphia pilot locations, including the following: * The Cleveland pilot location experienced issues with laboratory data in veterans' VA medical records that limited the results on one of the clinical quality measures in 2013. * For both the Cleveland and Philadelphia pilot locations, VA reported that the dialysis machines did not operate properly, which resulted in a lower than expected number of veterans being treated at the Philadelphia pilot location between January and June 2013 and delayed the opening of the Cleveland pilot location until July 2013. As a result, VA recommended that another full 12 months of data from the Philadelphia and Cleveland pilot locations be analyzed before making conclusive comparisons of the treatment costs of these pilot locations with non-VA dialysis providers. In addition, the evaluation contractor has reviewed the Dialysis Pilot for 2 years and has one year remaining on its contract. At the time of our analysis, the evaluation contractor had completed 2 years of evaluations on the quality of care provided by the four pilot locations and 1 year of treatment cost evaluations for two of the pilot locations. For information on VA's clinical quality review of pilot locations, see appendix I; VA's patient satisfaction review for pilot locations, see appendix II; VA's access to care review for pilot locations, see appendix III; the evaluation contractor's clinical quality review of pilot locations, see appendix IV; and VA's and the evaluation contractor's reviews of pilot location average cost per treatment, see appendix V. Conclusions: VA has not documented its plan for the conclusion of the Dialysis Pilot, despite beginning pilot development 5 years ago. Without a formally established project timeline, VA cannot effectively monitor performance against specified timeframes, validate and defend the timeline of the Dialysis Pilot to VA decisionmakers and other stakeholders, or ultimately hold VA decisionmakers accountable for future resource investment decisions. In addition, the transparency of the Dialysis Pilot is jeopardized by VA not clearly documenting the achievements stated by VA officials--the creation of a replicable model for additional VA-operated free-standing dialysis clinics and demonstrating a breakeven point--as criteria for concluding the Dialysis Pilot. Recommendation: To improve the transparency and accountability of the Dialysis Pilot, we recommend that the Secretary of Veterans Affairs direct the Under Secretary for Health to document plans for concluding the Dialysis Pilot, including establishing an end date or documenting criteria for deciding what constitutes the successful completion of the pilot. Agency Comments and Our Evaluation: VA provided written comments on a draft of this report, which we have reprinted in appendix VI. In its comments, VA generally agreed with our conclusions and concurred with our recommendation to improve the transparency and accountability of the Dialysis Pilot. VA also stated that in July 2015--2 years after the Cleveland pilot location became operational--it plans to review the outcomes from the four pilot locations and make recommendations for the Dialysis Pilot. VA did not provide any technical comments. In its general comments, VA noted that the criteria for deciding what constitutes the successful completion of its Dialysis Pilot is included in the department's document entitled, "Evaluation Plan: VA Free-Standing Dialysis Centers," which VA enclosed with its comments. While this document describes the measures VA is using to evaluate the pilot, it does not delineate the amount of data or length of review that is needed in order to conclude the pilot. In addition, VA did not clearly state when recommendations will be made and whether the pilot will end at that time. We continue to believe that VA should clearly document its plan for concluding the Dialysis Pilot by clearly specifying an end date for the pilot or specific outcomes that must be met. We are sending copies of this report to the Secretary of Veterans Affairs, appropriate congressional committees, and other interested parties. In addition, the report is available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or williamsonr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VII. Sincerely yours, Signed by: Randall B. Williamson: Director, Health Care: [End of section] Appendix I: Results of Clinical Quality Review by the Department of Veterans Affairs: This appendix provides information on the Department of Veterans Affairs' (VA) results from its review of Dialysis Pilot location clinical quality. To assess clinical quality, VA selected and evaluated Dialysis Pilot locations using 11 performance measures established by the Centers for Medicare & Medicaid Services (CMS). For calendar years 2012 and 2013, CMS defined the lowest achievement threshold for facility performance as the 15th percentile, the average performance standard as the 50th percentile, and the highest benchmark as the 90th percentile. Specifically, table 2 provides results of VA's review of clinical quality at the four Dialysis Pilot locations-- Raleigh and Fayetteville, North Carolina; Philadelphia, Pennsylvania; and Cleveland, Ohio. Table 2: Department of Veterans Affairs' (VA) Results of Centers for Medicare & Medicaid Services (CMS) Clinical Quality Measures for Dialysis Pilot Locations, Calendar Years 2012 and 2013: Dialysis adequacy: Clinical quality measures: Minimum delivered hemodialysis dose[D]; Higher quality of care indicator: higher percentage; Dialysis pilot location: Raleigh, NC: 2012: 99.0; Raleigh, NC: 2013: 99.0; Fayetteville, NC: 2012: 97.5; Fayetteville, NC: 2013: 95.0; Philadelphia, PA[A]: 2013: 96.0; Cleveland, OH[B]: 2013: 81.0; CMS-established performance standards [lowest-average-highest][C]: 2012: 94-97-100; 2013: 86-93-97. Bone and mineral metabolism: Clinical quality measures: Measurement of phosphorous concentration[E]; Higher quality of care indicator: higher percentage; Dialysis pilot location: Raleigh, NC: 2012: 99.6; Raleigh, NC: 2013: 100; Fayetteville, NC: 2012: 100; Fayetteville, NC: 2013: 100; Philadelphia, PA[A]: 2013: 100; Cleveland, OH[B]: 2013: 97.0; CMS-established performance standards [lowest-average-highest][C]: 2012: N/A; 2013: N/A. Clinical quality measures: Measurement of calcium concentration[F]; Higher quality of care indicator: higher percentage; Dialysis pilot location: Raleigh, NC: 2012: 100; Raleigh, NC: 2013: 100; Fayetteville, NC: 2012: 100; Fayetteville, NC: 2013: 100; Philadelphia, PA[A]: 2013: 100; Cleveland, OH[B]: 2013: 97.0; CMS-established performance standards [lowest-average-highest][C]: 2012: N/A; 2013: N/A. Clinical quality measures: Proportion of patients with hypercalcemia[G]; Higher quality of care indicator: lower percentage; Dialysis pilot location: Raleigh, NC: 2012: 0.0; Raleigh, NC: 2013: 0.0; Fayetteville, NC: 2012: 0.0; Fayetteville, NC: 2013: 0.0; Philadelphia, PA[A]: 2013: 0.0; Cleveland, OH[B]: 2013: 2.0; CMS-established performance standards [lowest-average-highest][C]: 2012: N/A; 2013: N/A. Anemia management: Clinical quality measures: Proportion of patients who exceed hemoglobin target[H]; Higher quality of care indicator: lower percentage; Dialysis pilot location: Raleigh, NC: 2012: 0.0; Raleigh, NC: 2013: 0.0; Fayetteville, NC: 2012: 0.0; Fayetteville, NC: 2013: 1.0; Philadelphia, PA[A]: 2013: 0.0; Cleveland, OH[B]: 2013: 2.0; CMS-established performance standards [lowest-average-highest][C]: 2012: 10-4-0; 2013: 5-1-0. Clinical quality measures: Monthly measurement of hemoglobin concentration[I]; Higher quality of care indicator: higher percentage; Dialysis pilot location: Raleigh, NC: 2012: 100; Raleigh, NC: 2013: 100; Fayetteville, NC: 2012: 100; Fayetteville, NC: 2013: 100; Philadelphia, PA[A]: 2013: 100; Cleveland, OH[B]: 2013: 99.0; CMS-established performance standards [lowest-average-highest][C]: 2012: N/A; 2013: N/A. Clinical quality measures: Proportion of patients whose iron levels were assessed[J]; Higher quality of care indicator: higher percentage; Dialysis pilot location: Raleigh, NC: 2012: 100; Raleigh, NC: 2013: 100; Fayetteville, NC: 2012: 100; Fayetteville, NC: 2013: 100; Philadelphia, PA[A]: 2013: 100; Cleveland, OH[B]: 2013: 98.0; CMS-established performance standards [lowest-average-highest][C]: 2012: N/A; 2013: N/A. Vascular access type: Clinical quality measures: Proportion of patients with catheters[K]; Higher quality of care indicator: lower percentage; Dialysis pilot location: Raleigh, NC: 2012: 3.2; Raleigh, NC: 2013: 5.0; Fayetteville, NC: 2012: 3.6; Fayetteville, NC: 2013: 6.0; Philadelphia, PA[A]: 2013: 11.0; Cleveland, OH[B]: 2013: 6.0; CMS-established performance standards [lowest-average-highest][C]: 2012: 24-14-5; 2013: 22-13-5. Clinical quality measures: Proportion of patients with arterial venous fistula[L]; Higher quality of care indicator: higher percentage; Dialysis pilot location: Raleigh, NC: 2012: 49.4; Raleigh, NC: 2013: 58.0; Fayetteville, NC: 2012: 68.4; Fayetteville, NC: 2013: 72.0; Philadelphia, PA[A]: 2013: 53.0; Cleveland, OH[B]: 2013: 76.0; CMS-established performance standards [lowest-average-highest][C]: 2012: 46-58-74; 2013: 47-60-75. Infection and immunization: Clinical quality measures: Proportion of patients with bloodstream infections[M]; Higher quality of care indicator: lower percentage; Dialysis pilot location: Raleigh, NC: 2012: 1.2; Raleigh, NC: 2013: 0.7; Fayetteville, NC: 2012: 0.7; Fayetteville, NC: 2013: 0.4; Philadelphia, PA[A]: 2013: 0.0; Cleveland, OH[B]: 2013: 0.8; CMS-established performance standards [lowest-average-highest][C]: 2012: N/A; 2013: N/A. Clinical quality measures: Proportion of patients receiving influenza immunization[N]; Higher quality of care indicator: higher percentage; Dialysis pilot location: Raleigh, NC: 2012: 100; Raleigh, NC: 2013: 100; Fayetteville, NC: 2012: 100; Fayetteville, NC: 2013: 97.0; Philadelphia, PA[A]: 2013: 100; Cleveland, OH[B]: 2013: 100; CMS-established performance standards [lowest-average-highest][C]: 2012: N/A; 2013: N/A. Source: VA. GAO-14-646. Legend: Higher percentage indicates higher quality of care; Lower percentage indicates higher quality of care; N/A = no CMS-established performance standard available for this calendar year. Note: Measures in this table are either current or former CMS End- Stage Renal Disease (ESRD) Quality Incentive Program measures. The measures are also currently or were previously endorsed by the National Quality Forum. [A] VA did not assess clinical performance of the Philadelphia pilot location in 2012 because it opened in October 2012. [B] VA did not assess clinical performance of the Cleveland pilot location in 2012 because it opened in July 2013. [C] In 2009, CMS established the ESRD Quality Incentive Program as a way to promote high-quality services in outpatient dialysis facilities treating patients with ESRD. While the principles guiding the ESRD Quality Incentive Program remain the same over time, the program's specific quality measures and standards change from year to year. For calendar years 2012 and 2013, CMS defined the lowest achievement threshold for facility performance as the 15th percentile, the average performance standard as the 50th percentile, and the highest benchmark as the 90th percentile. [D] This measure calculates the proportion of dialysis patients that received the minimum recommended amount of dialysis. [E] This measure calculates the percentage of months in which the blood phosphorous levels--an indicator of potential bone disease--were measured for at least 97 percent of patients. [F] This measure calculates the percentage of months in which the blood calcium levels--an indicator of potential bone disease--were measured for at least 97 percent of patients. [G] This measure calculates the proportion of patients whose blood calcium levels--an indicator of potential bone disease--were on average above the recommended levels. [H] This measure calculates the proportion of patients whose hemoglobin levels were above the recommended levels. [I] This measure calculates the percentage of months in which the hemoglobin levels were measured for at least 99 percent of patients. [J] This measure calculates the proportion of patients whose iron levels were measured at least once every three months. [K] This measure calculates the proportion of patients that received dialysis via a catheter for 90 days or longer, which is associated with poorer patient outcomes. [L] This measure calculates the proportion of patients that received dialysis via an arterial venous fistula, which is associated with the most favorable health outcomes. [M] This measure calculates the proportion of patients with blood infections. [N] This measure calculates the proportion of patients who received influenza immunizations. [End of table] [End of section] Appendix II: Results of Patient Satisfaction Review by the Department of Veterans Affairs: This appendix provides results from the Department of Veterans Affairs' (VA) review of patient satisfaction at its two more established Dialysis Pilot locations--Raleigh and Fayetteville, North Carolina. VA did not analyze patient satisfaction for the newer pilot locations--Philadelphia, Pennsylvania, and Cleveland, Ohio--because these pilot locations were either not open or not operating at full capacity during the periods of VA's analysis. * Table 3 provides results for VA's review of the Raleigh, North Carolina, Dialysis Pilot location's patient satisfaction. * Table 4 provides results for VA's review of the Fayetteville, North Carolina, Dialysis Pilot location's patient satisfaction. Table 3: Department of Veterans Affairs' (VA) Results of Patient Satisfaction Surveys for the Raleigh, North Carolina, Dialysis Pilot Location, June 2011 through September 2013: Measure[A]: Nephrologists' communication and caring; June 2011 to October 2012[B]: Patient satisfaction scores for Raleigh pilot location[D]: 3.85; Patient satisfaction scores for non-VA dialysis providers[E]: 3.62; October 2012 to September 2013[C]: Patient satisfaction scores for Raleigh pilot location: 3.69; Patient satisfaction scores for non-VA dialysis providers[E]: 3.08. Measure[A]: Quality of dialysis center care and operations; June 2011 to October 2012[B]: Patient satisfaction scores for Raleigh pilot location[D]: 3.85: 3.63; Patient satisfaction scores for non-VA dialysis providers[E]: 3.52; October 2012 to September 2013[C]: Patient satisfaction scores for Raleigh pilot location: 3.59; Patient satisfaction scores for non-VA dialysis providers[E]: 3.16. Measure[A]: Providing information to patients; June 2011 to October 2012[B]: Patient satisfaction scores for Raleigh pilot location[D]: 0.79; Patient satisfaction scores for non-VA dialysis providers[E]: 0.81; October 2012 to September 2013[C]: Patient satisfaction scores for Raleigh pilot location: 0.77; Patient satisfaction scores for non-VA dialysis providers[E]: 0.72. Measure[A]: Rating of nephrologists; June 2011 to October 2012[B]: Patient satisfaction scores for Raleigh pilot location[D]: 2.83; Patient satisfaction scores for non-VA dialysis providers[E]: 2.77; October 2012 to September 2013[C]: Patient satisfaction scores for Raleigh pilot location: 2.79; Patient satisfaction scores for non-VA dialysis providers[E]: 2.50. Measure[A]: Rating of dialysis center staff; June 2011 to October 2012[B]: Patient satisfaction scores for Raleigh pilot location[D]: 2.67; Patient satisfaction scores for non-VA dialysis providers[E]: 2.69; October 2012 to September 2013[C]: Patient satisfaction scores for Raleigh pilot location: 2.79; Patient satisfaction scores for non-VA dialysis providers[E]: 2.50. Measure[A]: Rating of dialysis center; June 2011 to October 2012[B]: Patient satisfaction scores for Raleigh pilot location[D]: 2.64; Patient satisfaction scores for non-VA dialysis providers[E]: 2.62; October 2012 to September 2013[C]: Patient satisfaction scores for Raleigh pilot location: 2.68; Patient satisfaction scores for non-VA dialysis providers[E]: 3.00. Measure[A]: Number of completed surveys; June 2011 to October 2012[B]: Patient satisfaction scores for Raleigh pilot location[D]: 12; Patient satisfaction scores for non-VA dialysis providers[E]: 30; October 2012 to September 2013[C]: Patient satisfaction scores for Raleigh pilot location: 28; October 2012 to September 2013[C]: 2. Source: VA. GAO-14-646. [A] The score range for the providing information to patients measure is between 0 and 1, with 1 being the best possible score. The score range for all other measures is between 1 and 4, with 4 being the best possible score. [B] According to VA, patient satisfaction scores for this period did not include any adjustments for patient characteristics due to small sample sizes. [C] According to VA, patient satisfaction scores for this period did not include any adjustments for patient characteristics and should be interpreted with caution because sample sizes were too small to make significant comparisons. VA did not test for statistical significance for any measure during this time period. [D] The differences between the patient satisfaction scores for the Raleigh pilot location and veterans' previous non-VA dialysis providers were not statistically significant for any of the six measures. [E] The non-VA patient satisfaction scores reflect how veterans rated the previous non-VA dialysis provider prior to joining the pilot location. [End of table] Table 4: Department of Veterans Affairs' (VA) Results of Patient Satisfaction Surveys for the Fayetteville, North Carolina, Dialysis Pilot Location, June 2011 through September 2013: Measure[A]: Nephrologists' communication and caring; June 2011 to October 2012[B]: Patient satisfaction scores for Fayetteville pilot location: 3.62[E]; Patient satisfaction scores for non-VA dialysis provider[D]: 3.27[E]; October 2012 to September 2013[C]: Patient satisfaction scores for Fayetteville pilot location: 3.59; Patient satisfaction scores for non-VA dialysis provider[D]: NR. Measure[A]: Quality of dialysis center care and operations; June 2011 to October 2012[B]: Patient satisfaction scores for Fayetteville pilot location: 3.43; Patient satisfaction scores for non-VA dialysis provider[D]: 3.31; October 2012 to September 2013[C]: Patient satisfaction scores for Fayetteville pilot location: 3.38; Patient satisfaction scores for non-VA dialysis provider[D]: NR. Measure[A]: Providing information to patients; June 2011 to October 2012[B]: Patient satisfaction scores for Fayetteville pilot location: 0.70; Patient satisfaction scores for non-VA dialysis provider[D]: 0.74; October 2012 to September 2013[C]: Patient satisfaction scores for Fayetteville pilot location: 0.78; Patient satisfaction scores for non-VA dialysis provider[D]: NR. Measure[A]: Rating of nephrologists; June 2011 to October 2012[B]: Patient satisfaction scores for Fayetteville pilot location: 2.83[E]; Patient satisfaction scores for non-VA dialysis provider[D]: 2.52[E]; October 2012 to September 2013[C]: Patient satisfaction scores for Fayetteville pilot location: 2.82; Patient satisfaction scores for non-VA dialysis provider[D]: NR. Measure[A]: Rating of dialysis center staff; June 2011 to October 2012[B]: Patient satisfaction scores for Fayetteville pilot location: 2.63; Patient satisfaction scores for non-VA dialysis provider[D]: 2.42; October 2012 to September 2013[C]: Patient satisfaction scores for Fayetteville pilot location: 2.82; Patient satisfaction scores for non-VA dialysis provider[D]: NR. Measure[A]: Rating of dialysis center; June 2011 to October 2012[B]: Patient satisfaction scores for Fayetteville pilot location: 2.67; Patient satisfaction scores for non-VA dialysis provider[D]: 2.52; October 2012 to September 2013[C]: Patient satisfaction scores for Fayetteville pilot location: 2.80; Patient satisfaction scores for non-VA dialysis provider[D]: NR. Measure[A]: Number of completed surveys; June 2011 to October 2012[B]: Patient satisfaction scores for Fayetteville pilot location: 24; Patient satisfaction scores for non-VA dialysis provider[D]: 38; October 2012 to September 2013[C]: Patient satisfaction scores for Fayetteville pilot location: 15; Patient satisfaction scores for non-VA dialysis provider[D]: 1. Source: VA. GAO-14-646. Legend: NR = Survey response was not recorded to protect the anonymity of the one veteran who completed it. [A] The score range for the providing information to patients measure is between 0 and 1, with 1 being the best possible score. The score range for all other measures is between 1 and 4, with 4 being the best possible score. [B] According to VA, patient satisfaction scores for this period did not include any adjustments for patient characteristics due to small sample sizes. [C] According to VA, patient satisfaction scores for this period did not include any adjustments for patient characteristics and should be interpreted with caution because sample sizes were too small to make significant comparisons. VA did not test for statistical significance for any measure during this time period. [D] The non-VA patient satisfaction scores reflect how veterans rated the previous non-VA dialysis provider prior to joining the pilot location. [E] VA's analysis found that the difference between the Fayetteville pilot location and non-VA dialysis provider scores were statistically significant. [End of table] [End of section] Appendix III: Results of Access to Care Review by the Department of Veterans Affairs: This appendix provides results from the Department of Veterans Affairs' (VA) review of veterans' access to care at its two more established Dialysis Pilot locations--Raleigh and Fayetteville, North Carolina. In calendar year 2012, VA did not analyze veterans' access to care for the newer pilot locations--Philadelphia, Pennsylvania, and Cleveland, Ohio--because these pilot locations were either not open or not operating at full capacity during the periods of VA's analysis. * Table 5 provides results for VA's review of veterans' access to care at the Raleigh and Fayetteville pilot locations for calendar year 2012. * Table 6 provides results for VA's review of veterans' access to care at all four pilot locations for calendar year 2013. Table 5: Department of Veterans Affairs' (VA) Results of Veterans' Access to Care Review for the Raleigh and Fayetteville, North Carolina, Dialysis Pilot Locations, Calendar Year 2012: Performance measure: Average travel distances for veterans to pilot location; Pilot location: Raleigh, NC: 17.7 miles; Fayetteville, NC: 7.5 miles; VA performance standard: 30 miles. Source: VA. GAO-14-646. [End of table] Table 6: Department of Veterans Affairs' (VA) Results of Veterans' Access to Care Review for the Dialysis Pilot Locations, Calendar Year 2013: Performance measure: Percentage of veterans who traveled 30 miles or less from their residence to the pilot location where they received care; Pilot location: Raleigh, NC: 87%; Fayetteville, NC: 98%; Philadelphia, PA: 100%; Cleveland, OH: 100%; VA performance standard: 100%. Source: VA. GAO-14-646. [End of table] [End of section] Appendix IV: Results of Clinical Quality Review by the Evaluation Contractor: This appendix provides results from the evaluation contractor's review of clinical quality for three of the Department of Veterans Affairs' (VA) Dialysis Pilot locations in Raleigh and Fayetteville, North Carolina, and Philadelphia, Pennsylvania. VA contracted with a leading university research center to evaluate the Dialysis Pilot. The evaluation contractor's review of the Philadelphia Dialysis Pilot location was limited due to operational difficulties at the pilot location in calendar year 2012. The evaluation contractor did not complete a review of the Cleveland, Ohio, Dialysis Pilot location due to insufficient data resulting from the delayed opening of this pilot location. * Table 7 provides results for the evaluation contractor's review of the Raleigh and Fayetteville, North Carolina, Dialysis Pilot locations' clinical quality. * Table 8 provides results for the evaluation contractor's review of the Philadelphia, Pennsylvania, Dialysis Pilot location's clinical quality. Table 7: Evaluation Contractor's 2013 Dialysis Facility Report Clinical Quality Measures for the Raleigh and Fayetteville Department of Veterans Affairs (VA) Dialysis Pilot Locations Compared to North Carolina and U.S. Averages, Calendar Years 2009 to 2012: Clinical quality measure: Standardized mortality ratio[A]; Dialysis pilot location: Raleigh: 0.24[B]; Fayetteville: 0.83[C]; North Carolina: 1.02; U.S.: 1.00. Clinical quality measure: Standardized hospitalization ratio for days[D]; Dialysis pilot location: Raleigh: 0.52[C]; Fayetteville: 1.16[B]; North Carolina: 0.87; U.S.: 1.00. Clinical quality measure: Standardized hospitalization ratio for admissions[D]; Dialysis pilot location: Raleigh: 0.47[C]; Fayetteville: 0.54[C]; North Carolina: 0.91; U.S.: 1.00. Clinical quality measure: Standardized hospitalization ratio for emergency department visits[D]; Dialysis pilot location: Raleigh: 0.75[C]; Fayetteville: 0.85[C]; North Carolina: 1.04; U.S.: 1.00. Clinical quality measure: Hospitalized with septicemia[E]; Dialysis pilot location: Raleigh: 12.50%; Fayetteville: 12.60%; North Carolina: 10.20%; U.S.: 10.60%. Clinical quality measure: Transplant waitlist[F]; Dialysis pilot location: Raleigh: 19.40%[C]; Fayetteville: 11.40%[B]; North Carolina: 20.10%; U.S.: 24.30%. Clinical quality measure: Influenza vaccination[G]; Dialysis pilot location: Raleigh: 93.60%; Fayetteville: 85.20%; North Carolina: 75.50%; U.S.: 69.70%. Clinical quality measure: Urea reduction ratio >65%[H]; Dialysis pilot location: Raleigh: 95.80%; Fayetteville: 89.80%; North Carolina: 98.40%; U.S.: 98.40%. Clinical quality measure: Arterial venous fistula[I]; Dialysis pilot location: Raleigh: 54.10%; Fayetteville: 76.00%; North Carolina: 63.20%; U.S.: 65.30%. Clinical quality measure: Catheter[J]; Dialysis pilot location: Raleigh: 3.50%; Fayetteville: 13.50%; North Carolina: 6.40%; U.S.: 7.60%. Clinical quality measure: Average hemoglobin[K]; Dialysis pilot location: Raleigh: 10.10g/dL; Fayetteville: 10.50g/dL; North Carolina: 10.60 g/dL; U.S.: 10.60 g/dL. Source: Evaluation contractor. GAO-14-646. Legend: g/dL = grams per deciliter. Notes: VA contracted with a leading university research center to evaluate the Dialysis Pilot. This table includes results from the Dialysis Facility Reports (DFR) produced by the evaluation contractor for the Raleigh and Fayetteville pilot locations in calendar year 2013. DFRs are standard reports produced by the evaluation contractor for CMS for each of the more than 6,000 private sector dialysis facilities nationwide. These DFRs include information on several clinical quality measures. Each clinical quality measure included in the DFRs is analyzed for a unique reporting period--the most common of which was from when the pilot location opened through calendar year 2012. The DFRs only specify whether the difference between the results of VA pilot location and non-VA providers is statistically significant for some of the quality of care measures. [A] The standardized mortality ratio compares the observed death rate in the facility to the death rate that was expected based on national death rates during that year for patients with the same characteristics as those in the facility. The standardized mortality ratio accounts for factors such as patient age, race, ethnicity, sex, and duration of end-stage renal disease. For this measure, a lower result indicates higher quality of care. [B] According to the DFR produced by the evaluation contractor, the difference between this figure and the national figure was statistically significant. [C] According to the DFR produced by the evaluation contractor, the difference between this figure and the national figure was not statistically significant. [D] This measure compares the facility's observed number of days hospitalized, hospital admissions, or emergency department visits to the expected number, based on national averages. For this measure, a lower result indicates higher quality of care. [E] This measure represents the percentage of dialysis patients whose hospitalization diagnoses included septicemia--a serious, life- threatening infection. For this measure, a lower percentage indicates higher quality of care. [F] The percentage of all dialysis patients under age 70 that were being treated on December 31 in a facility that were on the transplant waitlist. [G] The percentage of dialysis patients being treated on December 31 that received the influenza vaccination. For this measure, a higher percentage indicates higher quality of care. [H] This measure represents the percentage of patients with a urea reduction ratio that was 65 percent or more. The urea reduction ratio is one measure of how effectively a dialysis treatment removed waste products from the body and is commonly expressed as a percentage. For this measure, a higher percentage indicates higher quality of care. [I] Arterial venous fistula is considered the best long-term access method for hemodialysis because it has a lower complication rate than other types of access. This measure calculates the percentage of patients that used an arterial venous fistula for their last treatment of the month. For this measure, a higher percentage indicates higher quality of care. [J] Catheters are alternative access methods for hemodialysis patients that are not ideal for permanent use because they increase the risk of infection. This measure reports the percentage of patients in which a catheter was used for their last treatment of the month, was the only means of vascular access, and was in place for at least 90 days prior to treatment. For this measure, a lower percentage indicates higher quality of care. [K] Anemia--a disease among people whose blood is low in red blood cells--is common in people with kidney disease. One of the common causes of anemia includes blood loss from hemodialysis and low levels of iron. Iron helps red blood cells make hemoglobin. Studies have shown that raising the hemoglobin above 12 g/dL in people who have kidney disease increases the risk of heart attack, heart failure, and stroke. [End of table] Table 8: Evaluation Contractor's 2013 Dialysis Facility Report Clinical Quality Measures for the Philadelphia Department of Veterans Affairs (VA) Dialysis Pilot Location Compared to Pennsylvania and U.S. Averages, Calendar Years 2009 to 2012: Clinical quality measure: Hospitalized with septicemia[A]; Philadelphia pilot location: 0.00%; Pennsylvania: 11.90%; U.S.: 10.60%. Clinical quality measure: Influenza vaccination[B]; Philadelphia pilot location: 66.70%; Pennsylvania: 66.90%; U.S.: 69.70%. Clinical quality measure: Urea reduction ratio >65%[C]; Philadelphia pilot location: 100%; Pennsylvania: 98.70%; U.S.: 98.40%. Clinical quality measure: Average hemoglobin[D]; Philadelphia pilot location: 9.80g/dL; Pennsylvania: 10.60g/dL; U.S.: 10.60g/dL. Legend: g/dL = grams per deciliter. Source: Evaluation contractor. GAO-14-646. Note: VA contracted with a leading university research center to evaluate the Dialysis Pilot. This table includes results from the Dialysis Facility Reports (DFR) produced by the evaluation contractor for the Philadelphia pilot location in calendar year 2013. DFRs are standard reports produced by the evaluation contractor for CMS for each of the more than 6,000 private sector dialysis facilities nationwide. These DFRs include information on several clinical quality measures. Each clinical quality measure included in the DFR is analyzed for a unique reporting period--the most common of which was from when the pilot location opened through calendar year 2012. [A] This measure represents the percentage of dialysis patients whose hospitalization diagnoses included septicemia--a serious, life- threatening infection. For this measure, a lower percentage indicates higher quality of care. [B] This measure represents the percentage of dialysis patients being treated on December 31 that received the influenza vaccination. For this measure, a higher percentage indicates higher quality of care. [C] This measure represents the percentage of patients with a urea reduction ratio that was 65 percent or more. The urea reduction ratio is one measure of how effectively a dialysis treatment removed waste products from the body and is commonly expressed as a percentage. For this measure, a higher percentage indicates higher quality of care. [D] Anemia--a disease among people whose blood is low in red blood cells--is common in people with kidney disease. One of the common causes of anemia includes blood loss from hemodialysis and low levels of iron. Iron helps red blood cells make hemoglobin. Studies have shown that raising the hemoglobin above 12 g/dL in people who have kidney disease increases the risk of heart attack, heart failure, and stroke. [End of table] [End of section] Appendix V: Results of Treatment Cost Reviews by the Department of Veterans Affairs and the Evaluation Contractor: Table 9 provides results from the Department of Veterans Affairs' (VA) and the evaluation contractor's reviews of the average cost per treatment for Dialysis Pilot locations in Raleigh and Fayetteville, North Carolina, Philadelphia, Pennsylvania, and Cleveland, Ohio, and non-VA dialysis providers for calendar year 2012 and calendar year 2013. VA contracted with a leading university research center to evaluate the Dialysis Pilot. Table 9: Average Cost Per Treatment for Department of Veterans Affairs (VA) Dialysis Pilot Locations and Non-VA Dialysis Providers as Calculated by VA and the Evaluation Contractor, Calendar Years 2012 and 2013: Data source: VA; Pilot location: Raleigh, NC; 2012: Pilot location average cost per treatment: $287[A]; Non-VA dialysis provider average cost per treatment: $318[B]; 2013: Pilot location average cost per treatment: $266[C]; Non-VA dialysis provider average cost per treatment: $350[B,D]. Pilot location: Fayetteville, NC; 2012: Pilot location average cost per treatment: $295[A]; Non-VA dialysis provider average cost per treatment: $318[B]; 2013: Pilot location average cost per treatment: $296[C]; Non-VA dialysis provider average cost per treatment: $350[B,D]. Pilot location: Philadelphia, PA[E]; 2012: Pilot location average cost per treatment: No data; Non-VA dialysis provider average cost per treatment: No data; 2013: Pilot location average cost per treatment: $766[C,F]; Non-VA dialysis provider average cost per treatment: $348[B,D]. Pilot location: Evaluation contractor: Cleveland, OH[G]; 2012: Pilot location average cost per treatment: No data; Non-VA dialysis provider average cost per treatment: No data; 2013: Pilot location average cost per treatment: $636[C,F]; Non-VA dialysis provider average cost per treatment: $362[B,D]. Data source: Evaluation contractor; Pilot location: Raleigh, NC; 2012: Pilot location average cost per treatment: $291[H]; Non-VA dialysis provider average cost per treatment: $324[I]; 2013: Pilot location average cost per treatment: N/A; Non-VA dialysis provider average cost per treatment: N/A. Pilot location: Fayetteville, NC; 2012: Pilot location average cost per treatment: $310[H]; Non-VA dialysis provider average cost per treatment: $324[I]; 2013: Pilot location average cost per treatment: N/A; Non-VA dialysis provider average cost per treatment: N/A. Pilot location: Philadelphia, PA[E]; 2012: Pilot location average cost per treatment: No data; Non-VA dialysis provider average cost per treatment: No data; 2013: Pilot location average cost per treatment: N/A; Non-VA dialysis provider average cost per treatment: N/A. Pilot location: Pilot location: Cleveland, OH[G]; 2012: Pilot location average cost per treatment: No data; Non-VA dialysis provider average cost per treatment: No data; 2013: Pilot location average cost per treatment: N/A; Non-VA dialysis provider average cost per treatment: N/A. Source: VA and evaluation contractor. GAO-14-646. Legend: No data: No data existed because the pilot location was not open for all or a majority of 2012; N/A = No data were available at the time we issued our report. Note: VA contracted with a leading university research center to evaluate the Dialysis Pilot. Both VA and the evaluation contractor calculated the average cost per treatment for each pilot location by dividing the pilot location's total health care costs by the total number of treatments. Total health care costs included, at a minimum, the following costs from Centers for Medicare & Medicaid Services (CMS) standard dialysis facility reports: capital-related costs on buildings and fixtures, operation and maintenance costs, housekeeping costs, machine capital-related or rental and maintenance costs, salaries and benefits for direct patient care, supply costs, laboratory costs, administrative and general costs, drug costs, and medical record costs. Additional costs specific to certain figures are detailed below. [A] For calendar year 2012, VA excluded costs from services furnished to individual patients--known as physician professional services--from its nephrologist costs, as these costs are billed separately and therefore excluded from CMS's facility cost methodology. Additionally, administrative overhead costs from Veterans Integrated Service Network (VISN) and VA Central Office staff were not included in calendar year 2012. [B] To calculate non-VA dialysis providers' average cost per treatment, VA created a composite figure using the average per treatment contract rate from the two largest non-VA dialysis providers in each pilot location's VISN, a per treatment VA local claims processing fee, a per treatment VA physician cost, and a per treatment VA financial administrative cost. [C] For calendar year 2013, VA included costs from services furnished to individual patients--known as physician professional services--in its nephrologist costs. VA also included in calendar year 2013 per treatment costs for VISN and VA Central Office overhead and travel benefits paid to veterans being treated at the pilot locations. These changes were made in order to more closely align VA's calculations of pilot location and non-VA dialysis provider costs. [D] This figure includes the per treatment cost for VA physician cost for nephrology oversight. [E] There were no data for the Philadelphia pilot location for calendar year 2012 because it did not open until October 2012. [F] According to VA, the average cost per treatment for dialysis at both the Philadelphia and Cleveland pilot locations was affected by these pilot locations not operating at or near full capacity for a portion of calendar year 2013. [G] There were no data for the Cleveland pilot location for calendar year 2012 because it did not open until July 2013. [H] This figure includes per treatment costs for nephrology oversight and travel benefits paid to veterans being treated at the pilot locations. [I] To determine the non-VA dialysis provider average cost per treatment in calendar year 2012, the evaluation contractor created a composite figure that included the weighted average of the contract per treatment prices for the three non-VA dialysis providers with the largest market share in the VISN where the pilots are located, the VA per treatment cost of travel benefits paid to veterans being treated at the pilot locations, and the VA per treatment nephrology oversight costs. [End of table] [End of section] Appendix VI: Comments from the Department of Veterans Affairs: Department of Veterans Affairs: Washington DC 20420: August 15, 2014: Mr. Randall B. Williamson: Director, Health Care: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Williamson: The Department of Veterans Affairs (VA) has reviewed the Government Accountability Office's (GAO) draft report, "VA Dialysis Pilot: Documentation of Plans for Concluding the Pilot Needed to Improve Transparency and Accountability" (GAO-14-646). VA generally agrees with GAO's conclusions and concurs with GAO's recommendation to the Department. The enclosure specifically addresses GAO's recommendation in the draft report. VA appreciates the opportunity to comment on your draft report. Sincerely, Signed by: Jose D. Riojas: Chief of Staff: Enclosure: Department of Veterans Affairs (VA) Response to Government Accountability Office (GAO) Draft Report "VA Dialysis Pilot: Documentation of Plans for Concluding the Pilot Needed to Improve Transparency and Accountability" (GA0-14-646): GAO Recommendation: To improve the transparency and accountability of the Dialysis Pilot, GAO recommends that the Secretary of Veterans Affairs direct the Under Secretary for Health to document plans for concluding the Dialysis Pilot, including establishing an end date or documenting criteria for deciding what constitutes the successful completion of the pilot. VA Comments: Concur. In July 2015, which is 2 years after the final center became operational, the Veterans Health Administration (VHA) will review the outcomes from all pilot programs and make recommendations for the Dialysis Pilot. VHA anticipates the data analysis to be completed by September 30, 2015. The criteria for deciding what constitutes the successful completion of the pilots is included in the document entitled, "Evaluation Plan: VA Free-Standing Dialysis Centers" (Attachment A), and was provided to Senator Burr of the Senate Veterans' Affairs Committee in November 2013. Target Completion Date: October 1, 2015. [End of section] Appendix VII: GAO Contact and Staff Acknowledgments: GAO Contact: Randall B. Williamson, (202) 512-7114 or williamsonr@gao.gov: Staff Acknowledgments: In addition to the contact named above, Marcia A. Mann, Assistant Director; Cathleen Hamann; Katherine Nicole Laubacher; Said Sariolghalam; and Teresa Tam made key contributions to this report. Jacquelyn Hamilton provided legal support. [End of section] Footnotes: [1] VHA oversees VA's health care system. The VA health care system includes 151 VA medical centers (VAMC) organized into 21 Veterans Integrated Service Networks (VISN). Each of VA's 21 VISNs is responsible for managing and overseeing VAMCs within a defined geographic area. [2] In this report, we use the term dialysis to describe hemodialysis- -the most common form of dialysis treatment provided in the United States. VA provides dialysis as part of the medical benefits package-- a full range of hospital and outpatient services, prescription drugs, and noninstitutional long-term care services--to all veterans enrolled in its health care system. See 38 C.F.R. § 17.38. [3] The Non-VA Medical Care Program--previously known as the Fee Basis Care Program--includes dialysis services and provides care to eligible veterans outside of VA when VA medical facilities are not feasibly available. VAMCs have criteria to determine whether non-VA medical care may be used. If a veteran is eligible for certain medical care, the VAMC should provide it as the first option. If the VAMC cannot provide the services--due to a lack of available specialists, long wait times, or extraordinary distances from the veteran's home--it may consider providing care through the Non-VA Medical Care Program in the veteran's community. [4] Veterans may elect to have their dialysis treatments through VA or Medicare but cannot receive dialysis benefits from both simultaneously. Medicare covers dialysis treatments for most individuals with ESRD regardless of age. Medicare coverage generally begins on the fourth month after they start dialysis. Medicare reimburses dialysis providers 80 percent of a specified rate and beneficiaries or private health insurance companies are responsible for the remaining 20 percent. Veterans who elect to have their dialysis treatments paid for by Medicare are responsible for paying the remaining 20 percent of their treatment costs because VA is not authorized to pay these out-of-pocket expenses incurred by veterans covered by Medicare. Veterans who elect to have their dialysis treatments provided through VA--either in VAMCs or through the Non-VA Medical Care Program--may not incur any out-of-pocket expenses. [5] VA-operated dialysis clinics are freestanding clinics that are typically located in convenient areas and vary in the number of patients served. The VHA Dialysis Steering Committee and its Dialysis Center Activation Subcommittee are responsible for overseeing the Dialysis Pilot. The VHA Dialysis Steering Committee includes VA clinicians and officials who assess and assist in the management of the delivery of dialysis services for veterans enrolled at VA. The VHA Dialysis Center Activation Subcommittee was created to implement the Dialysis Pilot. [6] The dialysis clinic for the Durham VAMC is located in Raleigh, North Carolina. [7] GAO, VA Dialysis Pilot: Increased Attention to Planning, Implementation, and Performance Measurement Needed to Help Achieve Goals, [hyperlink, http://www.gao.gov/products/GAO-12-584] (Washington, D.C.: May 23, 2012). [8] Through the Dialysis Pilot, sponsoring VAMCs for the pilot locations were each provided about $2.5 million in start-up funding to establish a VA-operated free-standing dialysis clinic and were to finish repaying this start-up funding to VA Central Office in fiscal year 2014. According to VA Office of Finance officials, each pilot location was provided $2.5 million from VHA's fiscal year 2010 Medical Services appropriation with the requirement that the pilot site would repay this amount in two scheduled payments of $1.25 million occurring in fiscal years 2012 and 2014. The pilot sites made the scheduled 2012 and 2014 payments using available balances from their respective fiscal years (2012 and 2014). VA added back the same amount to the VHA Medical Services appropriation account for the same respective fiscal year. [9] The contractor selected by VA to evaluate the Dialysis Pilot is a leading university research center that carries out a wide variety of epidemiological, clinical, medical outcomes, public policy, and economic research relating to ESRD, chronic kidney disease, and organ transplantation. Its projects are funded by multiple government and private sources, including the Centers for Medicare & Medicaid Services, Centers for Disease Control and Prevention, National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases, National Institute of Allergy and Infectious Diseases, Renal Research Institute, VA, and the American Society of Transplant Surgeons. [10] The measures selected to assess the clinical quality of VA pilot locations are: (1) the proportion of dialysis patients that received the minimum recommended amount of dialysis; (2) the percentage of months in which blood phosphorous levels were measured for at least 97 percent of the patients--blood phosphorous levels are an indicator of potential bone disease; (3) the percentage of months in which blood calcium levels were measured for at least 97 percent of the patients-- blood calcium levels are an indicator of potential bone disease; (4) the proportion of patients whose blood calcium levels were, on average, above the recommended levels; (5) the percentage of months in which hemoglobin levels were measured for at least 99 percent of the patients--hemoglobin levels are an indicator of anemia; (6) the proportion of patients whose hemoglobin levels were above the recommended levels; (7) the proportion of patients whose iron levels were measured at least once every three months--iron levels are an indicator of anemia; (8) the proportion of patients that received dialysis via a catheter, which is associated with poorer patient outcomes; (9) the proportion of patients that received dialysis via an arterial venous fistula, which is associated with the most favorable health outcomes; (10) the proportion of patients with blood infections; and (11) the proportion of patients who received influenza immunizations. [11] CMS requires that all dialysis facilities treating 30 or more patients in the United States administer this survey to all patients treated within their facilities. [12] The 30 mile access measure is based upon the recommendation of VA clinicians. [13] These data are extracted from VA's Managerial Cost Accounting Office's adjusted cost reports. The Managerial Cost Accounting Office manages VA's designated cost accounting system. Adjusted cost reports include start-up costs for the pilot locations which are amortized over a 10-year period. [14] VA uses data from its own Managerial Cost Accounting Office to assess pilot location costs and categorized these costs according to CMS cost reporting requirements. CMS requires dialysis facilities to follow a certain methodology when completing the CMS Independent Renal Dialysis Facility Cost Report (CMS Form 265-11) each calendar year. This methodology requires facilities to report capital-related costs on buildings and fixtures, operation and maintenance costs, housekeeping costs, machine capital-related or rental and maintenance costs, salaries and benefits for direct patient care, supply costs, laboratory costs, administrative and general costs, drug costs, medical record costs, and nephrologist costs. A nephrologist is a medical doctor who specializes in kidney care and treating diseases of the kidneys. [15] Nephrology is the branch of medicine that deals with diseases of the kidneys. [16] This contract includes a mandatory first year of work, referred to as a base year, and gives VA the option of continuing the contract for two additional years, referred to as option years. VA and the evaluation contractor are currently in the first option year of this contract and VA officials reported they expect to exercise the second option year for fiscal year 2015. [17] The data used for the analysis of pilot locations' costs conducted by the evaluation contractor are provided by VA's Managerial Cost Accounting Office. [18] The evaluation contractor is also providing VA with an annual dialysis facility cost review for the four pilot locations; however, these do not include a cost comparison with non-VA dialysis providers. [19] In calendar year 2012, the evaluation contractor did not include administrative overhead costs for VISN and VA Central Office staff because the evaluation contractor had not yet determined how to accurately incorporate them. The evaluation contractor recommended that VA revise its methodology for calculating pilot locations' costs to include these overhead costs. The evaluation contractor also included veteran travel-related and medical oversight costs to ensure that the methodology used to calculate the average cost per treatment for the pilot locations and non-VA dialysis providers more closely aligned. [20] A non-VA dialysis provider's market share was determined by the percentage of dialysis facilities it owned out of the total number of dialysis facilities located in that VISN's geographic area. [21] GAO, GAO Schedule Assessment Guide: Best Practices for Project Schedules, [hyperlink, http://www.gao.gov/products/GAO-12-120G] (Washington, D.C.: May 30, 2012). [22] GAO, Designing Evaluations: 2012 Revision, [hyperlink, http://www.gao.gov/products/GAO-12-208G] (Washington, D.C.: Jan. 31, 2012). [23] In March 2010, VA completed a business analysis that demonstrated the rationale for conducting the Dialysis Pilot. This document communicates the results of VA's analysis of internal and external options for providing outpatient dialysis treatments to veterans. The implementation timeline in this document establishes a 5-year timeline for implementing the Dialysis Pilot beginning in April 2010 and concluding in fiscal year 2015 with the evaluation of the fifth year of pilot location operations. [24] VA's business analysis estimated that all four pilot locations would open in October 2010; however, the opening of the pilot locations was delayed. The first two pilot locations, Raleigh and Fayetteville, opened in June 2011, the Philadelphia pilot location opened in October 2012, and the Cleveland pilot location opened in July 2013. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]