VA Disability Benefits:
Improved Planning Practices Would Better Ensure Successful Appeals Reform
GAO-18-352: Published: Mar 22, 2018. Publicly Released: Mar 22, 2018.
- Highlights Page:
- Full Report:
- Accessible Version:
What GAO Found
The Department of Veterans Affairs' (VA) plan for implementing a new disability appeals process while attending to appeals in the current process addresses most, but not all, elements required by the Veterans Appeals Improvement and Modernization Act of 2017 (Act). VA's appeals plan addresses 17 of 22 required elements, partially addresses 4, and does not address 1. For example, not addressed is the required element to include the resources needed by the Veterans Benefits Administration (VBA) and the Board of Veterans' Appeals (Board) to implement the new appeals process and address legacy appeals under the current process. VA needs this information to certify, as specified under the Act, that it has sufficient resources to implement appeals reform and make timely appeals decisions under the new and legacy processes.
VA's appeals plan reflects certain sound planning practices, but it could benefit from including important details in several key planning areas:
Performance measurement: VA's plan reflects steps taken to track performance, but could articulate a more complete and balanced set of goals and measures for monitoring and assessing performance on a range of dimensions of success. Specifically, the plan reports that VA is developing a process to track timeliness of the new and legacy processes. However, contrary to sound planning practices, the plan does not include timeliness goals for all five appeals options available to veterans, does not include goals or measures for additional aspects of performance (such as accuracy or cost), and does not explain how VA will monitor or assess the new process compared to the legacy process. Unless VA clearly articulates a complete and balanced set of goals and measures, it could inadvertently incentivize staff to focus on certain aspects of appeals performance over others or fail to improve overall service to veterans.
Project management: VA's plan includes a master schedule for implementing the new appeals plan. However, this schedule falls short of other sound practices for guiding implementation and establishing accountability, such as articulating interim goals and needed resources for, and interdependencies among, activities. Unless VA augments its master schedule to include all key activities and reflect sound practices, VA may be unable to provide reasonable assurance that it has the essential program management information needed for this complex and important effort.
Risk assessment: VA has taken steps to assess and mitigate some risks related to appeals reform by, for example, pilot testing two of the five appeals options through its Rapid Appeals Modernization Program (RAMP). However, as designed, RAMP does not include key features of a well-developed and documented pilot test. For example, VA has not articulated how it will assess RAMP before proceeding with full implementation. In addition, RAMP is not pilot testing three options and, as a result, VA will not have data on the extent to which veterans will appeal directly to the Board when given the option. Unless VA identifies and mitigates key risks associated with implementing a new process, VA is taking a chance that untested aspects will not perform as desired.
Why GAO Did This Study
VA's disability compensation program pays cash benefits to veterans with disabilities connected to their military service. In recent years, the number of appeals of VA's benefit decisions has been rising. For decisions made on appeal in fiscal year 2017, veterans waited an average of 3 years for resolution by either VBA or the Board, and 7 years for resolution by the Board. The Veterans Appeals Improvement and Modernization Act of 2017 makes changes to VA's current (legacy) appeals process, giving veterans new options to have their claims further reviewed by VBA or appeal directly to the Board. The Act requires VA to submit to Congress and GAO a plan for implementing a new appeals process, and includes a provision for GAO to assess VA's plan.
This report examines the extent to which VA's plan (1) addresses the required elements in the Act, and (2) reflects sound planning practices identified in prior GAO work. GAO reviewed and assessed VA's appeals plan and related documents against sound planning practices, and solicited VA's views on its assessments.
What GAO Recommends
GAO recommends that VA (1) fully address all legally required elements in its appeals plan, (2) articulate how it will monitor and assess the new appeals process as compared to the legacy process, (3) augment its master schedule for implementation, and (4) address risk more fully. VA agreed with GAO's recommendations and outlined its planned actions to address them.
For more information, contact Elizabeth Curda at (202) 512-7215 or email@example.com.
Recommendations for Executive Action
Status: Closed - Not Implemented
Comments: VA took some steps to address the required elements but not enough to fully address it before implementing the Appeals Modernization Act in February 2019. VA's updated plan added information related to the five required elements. And for two of the elements - monitoring the processing of legacy appeals, and projecting staff productivity - it described using sensitivity models and other analyses to monitor and forecast future VBA and Board workloads, production, and staffing requirements. However, as of February 2019 when the agency implemented reform, the agency only partially addressed three elements: 1) monitoring progress toward implementation of the new process, 2) delineating total resources, and 3) identifying personnel requirements. Regarding element 1, VA set various goals or metrics for appeal options but did not establish well-defined, measurable criteria (such as for the testing of the new appeals processes) against which it could have assessed readiness for implementing appeals reform. For element 2, VA's plan projected the number of needed VBA and Board personnel and added FTE information for other offices that help implement the appeals process and used a model to monitor resource needs. As of February 2019, however, the plan did not delineate the total resources required by VBA and the Board, such as resources related to fully implementing information technology, training, and onboarding new Board staff in FY20 and beyond. Regarding element 3, VBA and the Board used modeling to project personnel needs and identified solutions to scale back resources when it is no longer processing legacy appeals. However, as of February 2019, the Board had not provided similar details about how it would address future personnel needs; and neither VBA nor the Board provide projected staffing levels when VA is no longer processing any legacy appeals. Overall, we continue to believe that VA was missing information required by the Act prior to implementing appeals reform.
Recommendation: The Secretary of Veterans Affairs should address all of the required elements in the Act in VA's appeals plan to Congress--including delineating resources required for all VBA and Board appeals options--using sensitivity analyses and RAMP results, where appropriate and needed. (Recommendation 1)
Agency Affected: Department of Veterans Affairs
Comments: VA concurred with this recommendation and, as of March 2020, addressed some but not all aspects of it. Regarding monitoring, VA has made progress in monitoring and addressing workload changes in its new and legacy appeals processes. For example, VA has developed sensitivity models and other analyses to monitor and forecast future VBA and Board workloads, production, and staffing requirements to help VA manage the new and legacy appeals processes. Further, the Board is aiming to complete non-remanded legacy appeals by December 2022 by first addressing hearing-requested appeals. However, VBA and the Board have yet to specify a complete set of balanced goals for monitoring the new and legacy appeals processes (e.g., timely and accurate processing of appeals while ensuring veteran satisfaction). For example, the Board will not establish timeliness goals for all new Board options until after collecting additional data during implementation of the new appeals process, thus delaying VA's vision for what successful implementation should look like and hindering awareness of resources required to achieve that vision. Regarding comparing the performance of the new and legacy appeals processes, VA officials have previously reported that they intend to use timeliness and productivity metrics from section 5 of the Appeals Modernization Act. Further, VBA and Board officials have articulated steps they are taking to collect, through surveys, comparable information on veterans' satisfaction with the new and legacy appeals processes. However, VA has not fully articulated detailed steps and timeframes for assessing the relative performance of the new and legacy appeals processes. For example, VA has not indicated how it will assess whether or the extent to which the new process, which also allows for multiple appeal opportunities, will achieve final resolution of veterans' appeals sooner, on average, than the legacy process, as we recommended in GAO-17-234. We will consider closing this recommendation when VA provides documentation indicating how it will assess the new appeals process against the legacy process vis a vis balanced measures.
Recommendation: The Secretary of Veterans Affairs should clearly articulate in VA's appeals plan how VA will monitor and assess the new appeals process compared to the legacy process, including specifying a balanced set of goals and measures--such as timeliness goals for all VBA appeals options and Board dockets, and measures of accuracy, veteran satisfaction, and cost--and related baseline data. (Recommendation 2)
Agency Affected: Department of Veterans Affairs
Comments: VA concurred with this recommendation. Since our report was issued, VA took significant action on its project plan and in February 2019, implemented the Veterans Appeals Improvement and Modernization Act of 2017. VA provided increasingly more details in updates to its plan about sub-activities related to processing legacy appeals, monitoring implementation, drafting Board policies, training, and testing of the new process. For example, VA added activities related to the Rapid Appeals Modernization Program (RAMP) test of the VBA-only options as well as the test of the new Board options. In spring and summer 2018, according to VA officials, VA set a baseline schedule for implementing appeals reform in response to the potential February 2019 implementation date established in the Act. In October 2018, VA provided us with lower-level schedules and other related information that allowed us to conduct a more detailed assessment of VA's IMS against applicable best practices criteria. However, as of December 2018, VA's schedule did not fully align with best practices. For example, VA's schedule did not contain a work breakdown structure that defines the work, activities, and resources necessary to accomplish implementation-details that would inform resources and time needed for the project. (For details on this assessment, see GAO-19-272T.) While VA fully implemented appeals reform in February 2019, incorporating such lessons learned into future project planning could help VA improve its project scheduling capabilities. Specifically, in February 2020 VA reported that Caseflow-VA's replacement system intended to support appeals reform-had "minimal functionality," with many functionalities yet to be implemented. We will consider closing this recommendation when VA has produced a more complete plan for-and that establishes accountability and reduces risk of failure related to-developing, implementing, and integrating remaining key functionality envisioned under Caseflow. We will also continue tracking whether the Board addresses long-term Caseflow planning under recommendation 2 in GAO-17-234.
Recommendation: The Secretary of Veterans Affairs should augment the master schedule for VA's appeals plan to reflect all activities--such as modifications to information technology (IT) systems--as well as assigned responsibilities, interdependencies, start and end dates for key activities for each workgroup, and resources, to establish accountability and reduce overall risk of implementation failures. (Recommendation 3)
Agency Affected: Department of Veterans Affairs
Comments: The Department of Veterans Affairs (VA) concurred with our recommendation to more fully address risks associated with implementing a new appeals process. As of March 2020, VA has taken many steps to address our recommendation, although key steps are remaining for VA to better assess risks associated with implementing appeals reform and managing appeals workloads in the legacy process. Specifically, since May 2018, VBA and Board officials reported developing and using new sensitivity analyses that would allow the agency to project potential budget needs and staffing requirements. Further, VBA is using these analyses to more accurately predict resolution of legacy appeals given certain assumptions. Furthermore, in addition to testing two appeal options available within VBA through its Rapid Appeals Modernization Program (RAMP), VA conducted small-scale testing of the three new Board appeals options through the Board's Early Applicability of Appeals Modernization (BEAAM) pilot, testing that was missing from VA's original November 2017 plan. VA also reported using lessons learned from testing all appeals options to update the implementation process. However, as designed, RAMP and BEAAM lacked well-defined, measurable criteria for assessing lessons learned from its testing, and the agency did not fully test all aspects of the appeals options before moving to full implementation in February 2019. Even though VA has fully implemented the new disability appeals process without addressing these aspects of our recommendation, many of the other principles of sound planning practices that informed our recommendation remain relevant, even after implementation, to ensure the new process meets veterans' needs. Specifically, VA has not developed mitigation strategies for all identified risks, such as veterans appealing to the Board at higher rates than expected or choosing more resource-intensive Board options, such as those involving new evidence or a hearing. As of January 2020, the more resource-intensive hearing option accounted for over 50 percent of new appeals inventory. At the same time, the Board reported it plans to prioritize resources on reducing the legacy appeals inventory and on the least resource intensive appeal option for which it established a timeliness goal. Although the Board has increased productivity, addressing legacy hearings will take several years as veterans may continue choosing the new hearing option at high rates and the Board prioritizes other workloads. This circumstance could subject veterans to longer wait times and increasing backlogs under the hearing option. As of March 2020, VA also has not established a complete and balanced set of goals and measures for the new appeals options, which are a necessary pre-condition to effectively assessing risk. Lacking a complete set of goals and measures, VA may not have comprehensively identified key risks.
Recommendation: The Secretary of Veterans Affairs should ensure that the appeals plan more fully addresses risk associated with appeals reform--for example, by assessing risks against a balanced set of goals and measures, articulating success criteria and an assessment plan for RAMP, and testing or conducting sensitivity analyses of all appeals options--prior to fully implementing the new appeals process. (Recommendation 4)
Agency Affected: Department of Veterans Affairs