VA Disability Benefits:
Improved Planning Practices Would Better Ensure Successful Appeals Reform
GAO-18-352: Published: Mar 22, 2018. Publicly Released: Mar 22, 2018.
What GAO Found
The Department of Veterans Affairs' (VA) plan for implementing a new disability appeals process while attending to appeals in the current process addresses most, but not all, elements required by the Veterans Appeals Improvement and Modernization Act of 2017 (Act). VA's appeals plan addresses 17 of 22 required elements, partially addresses 4, and does not address 1. For example, not addressed is the required element to include the resources needed by the Veterans Benefits Administration (VBA) and the Board of Veterans' Appeals (Board) to implement the new appeals process and address legacy appeals under the current process. VA needs this information to certify, as specified under the Act, that it has sufficient resources to implement appeals reform and make timely appeals decisions under the new and legacy processes.
VA's appeals plan reflects certain sound planning practices, but it could benefit from including important details in several key planning areas:
Performance measurement: VA's plan reflects steps taken to track performance, but could articulate a more complete and balanced set of goals and measures for monitoring and assessing performance on a range of dimensions of success. Specifically, the plan reports that VA is developing a process to track timeliness of the new and legacy processes. However, contrary to sound planning practices, the plan does not include timeliness goals for all five appeals options available to veterans, does not include goals or measures for additional aspects of performance (such as accuracy or cost), and does not explain how VA will monitor or assess the new process compared to the legacy process. Unless VA clearly articulates a complete and balanced set of goals and measures, it could inadvertently incentivize staff to focus on certain aspects of appeals performance over others or fail to improve overall service to veterans.
Project management: VA's plan includes a master schedule for implementing the new appeals plan. However, this schedule falls short of other sound practices for guiding implementation and establishing accountability, such as articulating interim goals and needed resources for, and interdependencies among, activities. Unless VA augments its master schedule to include all key activities and reflect sound practices, VA may be unable to provide reasonable assurance that it has the essential program management information needed for this complex and important effort.
Risk assessment: VA has taken steps to assess and mitigate some risks related to appeals reform by, for example, pilot testing two of the five appeals options through its Rapid Appeals Modernization Program (RAMP). However, as designed, RAMP does not include key features of a well-developed and documented pilot test. For example, VA has not articulated how it will assess RAMP before proceeding with full implementation. In addition, RAMP is not pilot testing three options and, as a result, VA will not have data on the extent to which veterans will appeal directly to the Board when given the option. Unless VA identifies and mitigates key risks associated with implementing a new process, VA is taking a chance that untested aspects will not perform as desired.
Why GAO Did This Study
VA's disability compensation program pays cash benefits to veterans with disabilities connected to their military service. In recent years, the number of appeals of VA's benefit decisions has been rising. For decisions made on appeal in fiscal year 2017, veterans waited an average of 3 years for resolution by either VBA or the Board, and 7 years for resolution by the Board. The Veterans Appeals Improvement and Modernization Act of 2017 makes changes to VA's current (legacy) appeals process, giving veterans new options to have their claims further reviewed by VBA or appeal directly to the Board. The Act requires VA to submit to Congress and GAO a plan for implementing a new appeals process, and includes a provision for GAO to assess VA's plan.
This report examines the extent to which VA's plan (1) addresses the required elements in the Act, and (2) reflects sound planning practices identified in prior GAO work. GAO reviewed and assessed VA's appeals plan and related documents against sound planning practices, and solicited VA's views on its assessments.
What GAO Recommends
GAO recommends that VA (1) fully address all legally required elements in its appeals plan, (2) articulate how it will monitor and assess the new appeals process as compared to the legacy process, (3) augment its master schedule for implementation, and (4) address risk more fully. VA agreed with GAO's recommendations and outlined its planned actions to address them.
For more information, contact Elizabeth Curda at (202) 512-7215 or firstname.lastname@example.org.
Recommendations for Executive Action
Comments: VA concurred with our recommendation. As of August 2018, VA has taken some action on the five elements that were not fully addressed. For example, VA's updated plan added details related to projecting staff productivity, identifying total resources, as well as about personnel requirements and projections for processing legacy appeals. For identifying total resources, VA added FTE information for other offices that help implement the appeals process and prepared a model to project resource needs. However, although VA's August 2018 updated plan now addresses the element related to projecting productivity, it continues to only partially address 3 elements related to monitoring implementation and workforce planning, and 1 element related to delineating the total resources. Regarding monitoring implementation, for example, VA's updated plans do include broad indicators to help determine VA's readiness for full implementation and describes testing of new appeals processes, but do not include well-defined, measureable criteria for assessing these tests and determining the circumstances in which VA would make adjustments prior to fully implementing reform. In addition, as of September 2018, VA had not established complete metrics and interim goals for all appeals options. For total resources, Section 3(b)(1), VA's updated plan projects through 2024 the number of needed VBA and Board personnel, which is likely the largest share of resources required for processing appeals in the legacy and new processes. However, the updated plan does not delineate the total resources required by VBA and the Board-such as personnel that may be required beyond 2024 until the agency is no longer processing legacy claims, as well as resources related to information technology, training, and onboarding new staff-and indicates the agency plans no further action on this aspect of the recommendation. We acknowledge that in some cases delineating the total resources could prove challenging, such as delineating information technology resources for the legacy and new processes. Nevertheless, we continue to believe VA's plan does not address this required element. We will consider closing this recommendation when VA provides complete information on all required elements.
Recommendation: The Secretary of Veterans Affairs should address all of the required elements in the Act in VA's appeals plan to Congress--including delineating resources required for all VBA and Board appeals options--using sensitivity analyses and RAMP results, where appropriate and needed. (Recommendation 1)
Agency Affected: Department of Veterans Affairs
Comments: The Department of Veterans Affairs (VA) concurred with our recommendation calling for the agency to clearly articulate how it will monitor and assess the new appeals process compared to the legacy process. As of April 2019, VA has addressed some but not all aspects of this recommendation. Regarding monitoring, VA has made progress in monitoring and addressing workload changes in its new and legacy appeals processes. For example, VA has developed sensitivity models and other analyses to monitor and forecast future VBA and Board workloads, production, and staffing requirements to help VA manage the new and legacy appeals processes. However, as of April 2019, the Board has not used these analyses to develop milestones or goals for reducing the backlog of legacy appeals. Importantly, VBA and the Board have yet to specify a complete set of balanced goals for monitoring the new and legacy appeals processes (e.g., timely and accurate processing of appeals while ensuring veteran satisfaction). For example, the Board will not establish timeliness goals for all Board options until after collecting additional data during implementation of the new appeals process, thus delaying VA's vision for what successful implementation would look like and hindering awareness of resources required to achieve that vision. Regarding comparing the performance of the new and legacy appeals processes, VA officials have previously reported that they intend to use timeliness and productivity metrics from section 5 of the Appeals Modernization Act. Further, VBA and Board officials have articulated steps they are taking to collect, through surveys, comparable information on veterans' satisfaction with the new and legacy appeals processes. However, VA has not fully articulated detailed steps and timeframes for assessing the relative performance of the new and legacy appeals processes. Without this assessment, VA cannot determine the extent to which the new process, which also allows for multiple appeal opportunities, will achieve final resolution of veterans' appeals sooner, on average, than the legacy process. Once VA reports having fully implemented this recommendation, we will assess any documentation provided to determine whether the agency's actions are sufficient to close this recommendation.
Recommendation: The Secretary of Veterans Affairs should clearly articulate in VA's appeals plan how VA will monitor and assess the new appeals process compared to the legacy process, including specifying a balanced set of goals and measures--such as timeliness goals for all VBA appeals options and Board dockets, and measures of accuracy, veteran satisfaction, and cost--and related baseline data. (Recommendation 2)
Agency Affected: Department of Veterans Affairs
Comments: The Department of Veterans Affairs (VA) concurred with our recommendation on project management. As of December 2018, VA took steps to develop interim goals and milestones for monitoring implementation, among other positive actions; however, the integrated master schedule (IMS) included gaps in sound practices for project management. For example, the Board, VBA, and other VA administrations made progress over time with developing and integrating underlying plans into the IMS in spring and summer 2018. According to VA officials, VA set a baseline schedule for implementing appeals reform in response to the potential February 2019 implementation date established in the Act. Since November 2017, VA's plan and progress reports have stated that VA uses an agency-wide governance structure to coordinate implementation, and regularly uses the schedule as a management tool for monitoring progress on appeals reform. In October 2018, VA provided us with lower-level schedules and information that allowed us to conduct a more detailed assessment of VA's IMS against applicable best practices criteria (for best practices see GAO-16-89G). We found that, while VA has made progress with providing more detail, its IMS and underlying schedules only minimally met sound practices for project management. Specifically, as with our March 2018 assessment, we found that the schedule does not contain enough detail to manage the work or provide a realistic representation of the resources and time needed for this project. For example, the schedule did not contain a work breakdown structure that defines the work, activities, and resources necessary to accomplish implementation. In addition, the schedule contains an invalid critical path, meaning that the schedule does not present the amount of time that key activities could be delayed before such delays affect VA's estimated implementation date. Without a valid critical path, management cannot focus on activities that will detrimentally affect the key program milestones and deliveries if they slip. (For more information about our assessment, see GAO-19-272T.) Establishing an overly optimistic schedule can reduce capacity for carrying out a project and potentially create pressure to sacrifice the quality of work activities to meet deadlines. Further, other VA efforts to redesign or update key aspects of VA's disability compensation process-including the Veterans Benefits Management System (VBMS)-were not driven by robust, comprehensive planning and did not achieve their schedule goals. While VA fully implemented appeals reform in February 2019 without adequately addressing our findings, we do not know the extent to which the lack of a robust schedule poses risks to successful and smooth implementation. Nevertheless, incorporating such lessons learned into future project planning could help VA improve its project scheduling capabilities.
Recommendation: The Secretary of Veterans Affairs should augment the master schedule for VA's appeals plan to reflect all activities--such as modifications to information technology (IT) systems--as well as assigned responsibilities, interdependencies, start and end dates for key activities for each workgroup, and resources, to establish accountability and reduce overall risk of implementation failures. (Recommendation 3)
Agency Affected: Department of Veterans Affairs
Comments: The Department of Veterans Affairs (VA) concurred with our recommendation to more fully address risks associated with implementing a new appeals process. As of April 2019, VA has taken many steps to address our recommendation, although key steps are remaining for VA to better assess risks associated with implementing appeals reform and managing appeals workloads in the legacy process. Specifically, since May 2018, VBA and Board officials reported developing and using new sensitivity analyses that would allow the agency to project potential budget needs and staffing requirements. Further, VBA is using these analyses to more accurately predict resolution of legacy appeals given certain assumptions. Furthermore, in addition to testing two appeal options available within VBA through its Rapid Appeals Modernization Program (RAMP), VA conducted small-scale testing of the three new Board appeals options through the Board's Early Applicability of Appeals Modernization (BEAAM) pilot, testing that was missing from VA's original November 2017 plan. VA also reported using lessons learned from testing all appeals options to update the implementation process. However, as designed, RAMP and BEAAM lacked well-defined, measurable criteria for assessing lessons learned from its testing, and the agency did not fully test all aspects of the appeals options before moving to full implementation in February 2019. Even though VA has fully implemented the new disability appeals process without addressing these aspects of our recommendation, many of the other principles of sound planning practices that informed our recommendation remain relevant, even after implementation, to ensure the new process meets veterans' needs. Specifically, VA has not developed mitigation strategies for all identified risks, such as veterans appealing to the Board at higher rates than expected or choosing more resource-intensive Board options, such as those involving new evidence or a hearing. In April 2019 Board officials told us that greater numbers of veterans were choosing the more resource-intensive Board options than expected. At the same time, the Board reported it plans to prioritize resources on reducing the legacy appeals inventory and on the least resource intensive appeal option for which it established a timeliness goal, potentially subjecting veterans to longer wait times and increasing backlogs under the hearing option. As of April 2019, VA also has not established a complete and balanced set of goals and measures for the new appeals options, which are a necessary pre-condition to effectively assessing risk. Lacking a complete set of goals and measures, VA may not have comprehensively identified key risks.
Recommendation: The Secretary of Veterans Affairs should ensure that the appeals plan more fully addresses risk associated with appeals reform--for example, by assessing risks against a balanced set of goals and measures, articulating success criteria and an assessment plan for RAMP, and testing or conducting sensitivity analyses of all appeals options--prior to fully implementing the new appeals process. (Recommendation 4)
Agency Affected: Department of Veterans Affairs