VA Disability Benefits:
Improved Planning Practices Would Better Ensure Successful Appeals Reform
GAO-18-352: Published: Mar 22, 2018. Publicly Released: Mar 22, 2018.
What GAO Found
The Department of Veterans Affairs' (VA) plan for implementing a new disability appeals process while attending to appeals in the current process addresses most, but not all, elements required by the Veterans Appeals Improvement and Modernization Act of 2017 (Act). VA's appeals plan addresses 17 of 22 required elements, partially addresses 4, and does not address 1. For example, not addressed is the required element to include the resources needed by the Veterans Benefits Administration (VBA) and the Board of Veterans' Appeals (Board) to implement the new appeals process and address legacy appeals under the current process. VA needs this information to certify, as specified under the Act, that it has sufficient resources to implement appeals reform and make timely appeals decisions under the new and legacy processes.
VA's appeals plan reflects certain sound planning practices, but it could benefit from including important details in several key planning areas:
Performance measurement: VA's plan reflects steps taken to track performance, but could articulate a more complete and balanced set of goals and measures for monitoring and assessing performance on a range of dimensions of success. Specifically, the plan reports that VA is developing a process to track timeliness of the new and legacy processes. However, contrary to sound planning practices, the plan does not include timeliness goals for all five appeals options available to veterans, does not include goals or measures for additional aspects of performance (such as accuracy or cost), and does not explain how VA will monitor or assess the new process compared to the legacy process. Unless VA clearly articulates a complete and balanced set of goals and measures, it could inadvertently incentivize staff to focus on certain aspects of appeals performance over others or fail to improve overall service to veterans.
Project management: VA's plan includes a master schedule for implementing the new appeals plan. However, this schedule falls short of other sound practices for guiding implementation and establishing accountability, such as articulating interim goals and needed resources for, and interdependencies among, activities. Unless VA augments its master schedule to include all key activities and reflect sound practices, VA may be unable to provide reasonable assurance that it has the essential program management information needed for this complex and important effort.
Risk assessment: VA has taken steps to assess and mitigate some risks related to appeals reform by, for example, pilot testing two of the five appeals options through its Rapid Appeals Modernization Program (RAMP). However, as designed, RAMP does not include key features of a well-developed and documented pilot test. For example, VA has not articulated how it will assess RAMP before proceeding with full implementation. In addition, RAMP is not pilot testing three options and, as a result, VA will not have data on the extent to which veterans will appeal directly to the Board when given the option. Unless VA identifies and mitigates key risks associated with implementing a new process, VA is taking a chance that untested aspects will not perform as desired.
Why GAO Did This Study
VA's disability compensation program pays cash benefits to veterans with disabilities connected to their military service. In recent years, the number of appeals of VA's benefit decisions has been rising. For decisions made on appeal in fiscal year 2017, veterans waited an average of 3 years for resolution by either VBA or the Board, and 7 years for resolution by the Board. The Veterans Appeals Improvement and Modernization Act of 2017 makes changes to VA's current (legacy) appeals process, giving veterans new options to have their claims further reviewed by VBA or appeal directly to the Board. The Act requires VA to submit to Congress and GAO a plan for implementing a new appeals process, and includes a provision for GAO to assess VA's plan.
This report examines the extent to which VA's plan (1) addresses the required elements in the Act, and (2) reflects sound planning practices identified in prior GAO work. GAO reviewed and assessed VA's appeals plan and related documents against sound planning practices, and solicited VA's views on its assessments.
What GAO Recommends
GAO recommends that VA (1) fully address all legally required elements in its appeals plan, (2) articulate how it will monitor and assess the new appeals process as compared to the legacy process, (3) augment its master schedule for implementation, and (4) address risk more fully. VA agreed with GAO's recommendations and outlined its planned actions to address them.
For more information, contact Elizabeth Curda at (202) 512-7215 or firstname.lastname@example.org.
Recommendations for Executive Action
Comments: VA concurred with our recommendation. As of August 2018, VA has taken some action on the five elements that were not fully addressed. For example, VA's updated plan added details related to projecting staff productivity, identifying total resources, as well as about personnel requirements and projections for processing legacy appeals. For identifying total resources, VA added FTE information for other offices that help implement the appeals process and prepared a model to project resource needs. However, although VA's August 2018 updated plan now addresses the element related to projecting productivity, it continues to only partially address 3 elements related to monitoring implementation and workforce planning, and 1 element related to delineating the total resources. Regarding monitoring implementation, for example, VA's updated plans do include broad indicators to help determine VA's readiness for full implementation and describes testing of new appeals processes, but do not include well-defined, measureable criteria for assessing these tests and determining the circumstances in which VA would make adjustments prior to fully implementing reform. In addition, as of September 2018, VA had not established complete metrics and interim goals for all appeals options. For total resources, Section 3(b)(1), VA's updated plan projects through 2024 the number of needed VBA and Board personnel, which is likely the largest share of resources required for processing appeals in the legacy and new processes. However, the updated plan does not delineate the total resources required by VBA and the Board-such as personnel that may be required beyond 2024 until the agency is no longer processing legacy claims, as well as resources related to information technology, training, and onboarding new staff-and indicates the agency plans no further action on this aspect of the recommendation. We acknowledge that in some cases delineating the total resources could prove challenging, such as delineating information technology resources for the legacy and new processes. Nevertheless, we continue to believe VA's plan does not address this required element. We will consider closing this recommendation when VA provides complete information on all required elements.
Recommendation: The Secretary of Veterans Affairs should address all of the required elements in the Act in VA's appeals plan to Congress--including delineating resources required for all VBA and Board appeals options--using sensitivity analyses and RAMP results, where appropriate and needed. (Recommendation 1)
Agency Affected: Department of Veterans Affairs
Comments: VA concurred with our recommendation. In VA's three subsequent updates to its November 2017 comprehensive plan for implementing appeals reform (February, May, and August 2018), VA has addressed some but not all aspects of this recommendation. Regarding establishing timeliness goals and balanced measures, VA states it is collecting data to inform its development of a complete and balanced set of measures for all new appeals options (e.g., timely and accurate processing of appeals while ensuring veteran satisfaction). However, VA's plan indicates that the agency will not establish timeliness goals for all Board options until after fully implementing the new appeals process, thereby delaying VA's vision for what successful implementation would look like, and hindering awareness of resources required to achieve that vision. Regarding monitoring, VA's plans indicate the agency will monitor and manage the allocation of staff for concurrent workloads in its legacy and new appeals processes using sensitivity and other analyses-analyses that could better position VA to manage the two parallel processes. Regarding assessing the new and legacy processes, VA officials told us they are working to produce and report metrics required under the Act. Further, the August updated plan articulates VA's intention to use customer satisfaction surveys to compare veterans' experience in the legacy and new processes as well as projected VBA productivity for legacy versus new appeal reviews from fiscal year 2019 through 2024. However, while VA's August 2018 plan indicates taking steps toward developing complete and balanced metrics and interim goals for the new process, it does not fully explain how the agency will use the Act's metrics to assess the relative performance of the new and legacy appeals processes on issues like overall timeliness or accuracy. Without this assessment, VA cannot determine the extent to which the new process, which also allows for multiple appeal opportunities, will achieve final resolution of veterans' appeals sooner, on average, than the legacy process. We will consider closing this recommendation when VA establishes timeliness goals and balanced measures as well as articulates a system that assesses the new and legacy processes to determine how well the new process is performing relative to the legacy process.
Recommendation: The Secretary of Veterans Affairs should clearly articulate in VA's appeals plan how VA will monitor and assess the new appeals process compared to the legacy process, including specifying a balanced set of goals and measures--such as timeliness goals for all VBA appeals options and Board dockets, and measures of accuracy, veteran satisfaction, and cost--and related baseline data. (Recommendation 2)
Agency Affected: Department of Veterans Affairs
Comments: VA concurred with our recommendation. As of August 2018, VA took steps to develop interim goals and milestones for monitoring implementation, among other positive actions, but the integrated master schedule (IMS) included gaps in sound practices for project management. For example, VA has taken steps to monitor implementation, such as adding broad indicators to monitor the status of implementing regulations and information technology, stating it will consider any lessons learned from piloting aspects of the new process. In addition, VA has updated the IMS to include several major activities that were missing in the original November 2017 plan and February and May 2018 updates, such as its small-scale pilot of the new Board options. However, the IMS overall does not include activities related to implementing appeals reform beyond the February 2019 implementation date. Moreover, the plan does not clearly reflect interdependencies among activities or other sound practices for establishing accountability and guiding implementation that we have detailed in our prior work (e.g., GAO-16-89G). VA is in the process of providing us with additional information about the IMS. When we review and confirm what actions VA has taken, we will update the status of this recommendation.
Recommendation: The Secretary of Veterans Affairs should augment the master schedule for VA's appeals plan to reflect all activities--such as modifications to information technology (IT) systems--as well as assigned responsibilities, interdependencies, start and end dates for key activities for each workgroup, and resources, to establish accountability and reduce overall risk of implementation failures. (Recommendation 3)
Agency Affected: Department of Veterans Affairs
Comments: VA concurred with our recommendation. As of August 2018, VA's updated plans reflect the continuous identification and updates of risks and related mitigation strategies; however, opportunities exist to better identify and assess risks associated with implementing appeals reform and managing appeals workloads in the legacy process. Regarding steps to mitigate risks, since November 2017 VA has been testing the two options available within VBA (i.e., review of a claim by a higher-level official based on the same evidence and review of a supplemental claim with additional evidence) through its Rapid Appeals Modernization Program (RAMP). Furthermore, the agency's May and August 2018 updated plans indicated steps VA is taking to conduct small-scale pilot testing of the three new Board appeals options, testing that was missing from VA's original plan. For example, VA has reported using the results of the Board's Early Applicability of Appeals Modernization (BEAAM) pilot to update training, test new procedures and forms, and take other actions intended to enhance veterans' appeals experience. VA also identified additional risks for the timeliness of Board options involving new evidence or a hearing given the new process allows a veteran to bypass VBA and appeal directly to the Board, which may have implications for timeliness of decisions and resource involved. Moreover, since May 2018, VA officials reported progress with developing and using new sensitivity analyses that allow the agency to project potential budget needs and staffing requirements and more accurately predict resolution of legacy appeals given certain assumptions. By taking these and other steps, VA should be better positioned to assess implementation risks. However, additional steps are needed to clarify risk mitigation strategies associated with implementing the new options, particularly those at the Board. For example, VA's August 2018 updated plan does not describe potential implications and related risk mitigation strategies for all of the Board's workloads, attendant resources, and/or timeliness of decisions. In addition, VA's plans lack well-defined, measurable criteria for assessing lessons learned from its RAMP and BEAAM pilots and determining the circumstances that would cause VA to consider making course corrections before moving to full implementation. Taking these additional steps would improve VA's ability to assess and mitigate risks as it implements its reforms. In addition, VA's August 2018 updated plan has not established a complete and balanced set of goals and measures for the new appeals options, which are a necessary pre-condition to effectively assessing risk. Lacking a complete set of goals and measures, VA may not have comprehensively reflected key risks in its updated plan.
Recommendation: The Secretary of Veterans Affairs should ensure that the appeals plan more fully addresses risk associated with appeals reform--for example, by assessing risks against a balanced set of goals and measures, articulating success criteria and an assessment plan for RAMP, and testing or conducting sensitivity analyses of all appeals options--prior to fully implementing the new appeals process. (Recommendation 4)
Agency Affected: Department of Veterans Affairs