VA Disability Benefits:
Additional Planning Would Enhance Efforts to Improve the Timeliness of Appeals Decisions
GAO-17-234: Published: Mar 23, 2017. Publicly Released: Mar 23, 2017.
What GAO Found
The Department of Veterans Affairs' (VA) is taking steps to improve the timeliness of its benefit compensation appeals process, in which veterans who are dissatisfied with claims decisions by the Veterans Benefits Administration (VBA) can appeal first to VBA, and then to the Board of Veterans' Appeals (the Board). VA has taken actions related to increasing staff, reforming the process, and updating information technology (IT), which are consistent with relevant sound planning practices. However, gaps in planning exist, thereby reducing the agency's ability to ensure that these actions will improve the timeliness of disability appeals decisions.
Increase staff: VA determined that staff resources have not sufficiently kept pace with increased pending appeals, and concluded that additional staff are needed, particularly at the Board, to improve timeliness and reduce its appeals inventory. The Board received approval to hire more staff in fiscal year 2017, and expects to need an additional hiring surge beginning in fiscal year 2018. As of October 2016, officials estimated that if the agency does not take any action, such as increasing staff in 2018, veterans may have to wait an average of 8.5 years by fiscal year 2026 to have their appeals resolved.
Consistent with sound workforce planning practices, VA modeled different options for increasing staff levels to support its conclusion that staff increases in conjunction with process change would reduce the appeals inventory sooner. However, contrary to sound practices, VA often used fixed estimates for key variables in its models—such as staff productivity—rather than a range of estimates (sensitivity analysis) to understand the effect variation in these key variables could have on staffing needs. Also, VA's written workforce plans—which cover recruiting, hiring and training—did not include detailed steps, time frames, and mitigation strategies consistent with sound workforce planning practices. For example, while VA has established a center for excellence in hiring to focus on recruitment and hiring the agency has not finalized training or telework plans or otherwise mitigated space constraints that it encountered for hiring staff in fiscal year 2017. Without a timely, detailed workforce plan, VA risks delays in hiring and preparing staff to help manage workloads as soon as possible.
Reform process: VA determined that new evidence—which a veteran can submit at any point during his or her appeal—inefficiently causes an additional round of reviews, and thus delays appeals decisions, and in response it proposed legislation (not enacted) to streamline the process. Consistent with sound practices for process redesign, VA worked with veterans service organizations (VSO) and other key stakeholders in developing the proposal, and continued to update VSOs about the development of its implementation plans.
VA's proposed reform is promising, but there are several gaps in its implementation plans. In particular, VA plans to fully implement appeals process reform at the Board as well as at VBA regional offices across the country while it concurrently manages the existing appeals inventory, a hiring surge, and planned system changes discussed below. However, VA's plans run counter to sound redesign practices that suggest pilot testing the process changes in a more limited fashion before full implementation, in order to manage risks and help ensure successful implementation of significant institutional change. VA officials told GAO that pilot testing—which would require legislation to implement—will prolong a process that is fundamentally broken and delay urgently needed repairs. However, without pilot testing VA may experience challenges and setbacks on a broader scale, which could undermine planned efficiencies and other intended outcomes. In addition, VA has not sufficiently identified how it will monitor progress, evaluate efficiency and effectiveness, identify trouble spots, and otherwise know whether implementation of its proposed process change is on track and meeting expectations. The absence of a robust monitoring plan with success criteria is inconsistent with sound planning practices for redesign and places the agency at risk of not being able to quickly identify and address setbacks. In addition, the timeliness measures that VA currently plans to report to Congress and the public lack transparency because they focus on individual parts of the agency and pieces of the new process rather than overall appeals resolution time from the veterans' perspective. Without a strategy for assessing the proposed new process that includes comprehensive measures, VA, the public, and Congress cannot know the extent to which the proposed process represents an improvement over the old process.
Update technology: VA determined that the computer system supporting its appeals process is outdated, prone to failures, and does not adequately support electronic claims processing. VA proposed a new IT system to reduce delays in appeals to the Board, and better integrate data from other systems. Consistent with sound practices, VA clearly laid out the scope and purpose of IT upgrades, and identified risks and strategies to mitigate them. However, the agency's plan lacks details for how and when its new system will be implemented, as suggested by sound planning practices for implementing new technology. Without a detailed schedule, VA risks not having new systems aligned with potential changes in the appeals process when they are implemented.
Why GAO Did This Study
VA compensates veterans for disabling conditions incurred in or aggravated by military service. Veterans can appeal VBA's decisions on their compensation claims, first to VBA and then to the Board, a separate agency within VA. In fiscal year 2015, more than 427,000 appeals were pending and veterans waited over 3 years on average for decisions. Of this total, about 81,000 were pending at the Board and the average cumulative time veterans waited for a decision by the Board in 2015 was almost 5 years.
This report examines VA's approaches to address challenges it identified as contributing to lengthy appeals processing times, and the extent to which those approaches are consistent with sound planning practices.
GAO focused mainly on the Board, which experienced an increase in workload of about 20 percent from fiscal year 2014 to 2015. GAO reviewed VA's proposed plans and actions and compared them to sound practices relevant to workforce planning and implementing process redesign and new information technology identified in federal guidance, such as internal control standards, and prior GAO work. GAO also analyzed VA's data for fiscal years 2011-2015 (the most recent available) on appeals decision timeliness and workloads; reviewed relevant federal laws, regulations, and planning documents; and interviewed VA officials and veterans service organizations.
What GAO Recommends
GAO is making five recommendations to VA and one matter for congressional consideration. VA should: apply sensitivity analyses when projecting staff needs, develop a more timely and detailed workforce plan, develop a robust plan for monitoring process reform, develop a strategy for assessing process reform, and create a schedule for IT improvements that takes into account plans for potential process reform. VA concurred in principle with the five recommendations, but believes it has met the intent of those recommendations and does not need to take additional action. GAO disagrees and—while recognizing VA's ongoing efforts—believes further action is needed on all five recommendations to improve VA's ability to successfully implement reforms, as discussed in the report.
VA disagreed with an additional draft recommendation that it incorporate pilot testing of its proposed appeals process into implementation plans and pursue necessary legislative authority. VA cited its perspective that the appeals process is broken and that piloting a new process would result in further delays to veterans appealing their disability decisions. GAO maintains that the benefits of pilot testing—which provides an opportunity to resolve implementation challenges and make refinements to the process on a smaller scale—outweigh the potentially negative consequences of delaying full implementation. Therefore, GAO removed the recommendation and added a matter for congressional consideration stating that Congress should consider requiring that appeals process reform be subject to a pilot test.
For more information, contact Daniel Bertoni at (202) 512-7215 or email@example.com.
Matter for Congressional Consideration
Status: Closed - Implemented
Comments: In August 2017, Congress enacted the Veteran Appeals Improvement and Modernization Act (P.L. 115-55). This legislation grants the Secretary of Veterans Affairs the ability to carry out such programs as the Secretary considers appropriate to test any assumptions relied upon in developing the comprehensive plan required by section 3(a) of the Act and to test the feasibility and advisability of any facet of the new appeals system.
Matter: To improve VA's ability to successfully implement appeals process reform, Congress should consider requiring that reforms of the VA disability appeals process be subject to a pilot test. To aid in the development of such a pilot test, Congress could require the Secretary of Veterans Affairs to propose options that would allow the agency the flexibility to test and refine the new process in a cost-effective and efficient manner, while ensuring pre-established interim goals and success criteria are being met prior to full implementation.
Recommendations for Executive Action
Comments: Since our March 2017 report, Congress passed the Veterans Appeals Improvement and Modernization Act of 2017, which required VA to produce a comprehensive plan for managing appeals under both the legacy and new system and required that the plan address, among other things, hiring, training, office space needs related to appeals and related risks. To date, VA took additional steps to recruit, hire, train, and manage office space for additional FTEs to address appeals workloads (part 1 of this recommendation); however, additional steps are needed to clarify risk mitigation strategies associated with calibrating its workforce dedicated to appeals (part 2). In its plan for implementing appeals reform (November 2017) and three subsequent updates to the plan (February, May, and August 2018), VA reported having established goals and related plans to recruit and hire additional FTEs, including most recently the need to hire 145 Board FTEs in fiscal year 2018, and hiring 605 FTEs in FY 2019 to process appeals at VBA. Regarding Board 2018 hiring, VA officials told us in June 2018 that of the 145 FTEs, 91 are in the hiring pipeline and the Board plans to make 100 additional offers of employment in June 2018. As of August 2018, VA's Integrated Master Schedule indicated that the Board's effort to hire up to and sustain 1,050 FTEs including 160 attorney advisors by September 1, 2018 was still in process. Regarding VBA 2019 hiring, although VA requested 605 FTEs in the FY 2019 President's Budget, VA also reported plans to use current funds to acquire 60 of the 605 FTEs in FY 2018 and use its FY 2019 operating budget to continue expedited hiring of FTEs. With respect to training, VA's May 2018 update includes a description and schedule for training new staff, and for training VA staff with respect to piloting and implementing appeals reform. For example, VA officials confirmed, that virtually all VBA regional office appeals claims staff and all Board staff have received training that provides an overview of the new appeals process, and VA will provide new and existing staff more extensive training on appeals modernization by January 2019 just prior to implementation. Moreover, VA indicated in August 2018 that the agency's testing of the new appeals options is informing the development and refinement of these trainings. Regarding managing office space, VA reported efforts to identify additional space for new Board staff, and provided GAO with its telework plan, and information on how it plans to adjudicate cases and provide ongoing training in a virtual environment. As of August 2018, VA reported having sufficient work stations for 1050 Board FTEs at end of calendar year 2018, and having plans in place to accommodate 605 new FTEs at VBA facilities, including the Appeals Management Office. While the additional steps taken by VA largely address part 1 of this recommendation, questions remain regarding risk mitigation strategies related to part 2. For example, VA's August plan noted that a recent test of the new process (BEAAM) revealed risks that the claims disagreement rate and average appeal by veterans may be higher than previously modeled. However, VA's August plan does not describe potential implications and related risk mitigation strategies for VBA's or the Board's workload inventories and/or timeliness of decisions. VA's plan also indicated that VBA's FTE needs would decrease by half over a 4 year period (from 2,100 in 2020 to 1,050 in 2024), but does not explain the extent to which VA will need to rely on tools or strategies other than attrition (such as employee removal or reassignment) to scale back resources dedicated to appeals if attrition is the same or lower than expected. We will hold this recommendation open until VA has further articulated risk mitigation strategies for ensuring it has appropriate capacity to manage appeals workloads and improve timeliness of appeals decisions.
Recommendation: To further align efforts to address appeals workload and improve timeliness of decisions, and reduce the risk that efforts will not go as planned, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits; the Chairman, Board of Veterans' Appeals; and the Chief Information Officer, as appropriate, to ensure development of a timely, detailed workforce plan for recruiting, hiring and training new hires. In particular, this plan should: (1) include detailed steps and timetables for updating training curriculum (such as preparing decisions in a virtual environment) and ensuring office space (such as telework guidance); and (2) incorporate risk mitigation strategies that consider how the timing of recruitment and training dovetails with uncertain time frames for implementing a new appeals process.
Agency Affected: Department of Veterans Affairs
Comments: VA concurred in principle with this recommendation. Moreover, since our March 2017 report, Congress passed the Veterans Appeals Improvement and Modernization Act of 2017, which required VA to develop a comprehensive appeals plan that included, among other things, descriptions of modifications to, cost estimates of and timelines for information technology that the agency needs to carry out appeals reform. As of May 2018, VA reported minimal Caseflow functionality needed to implement appeals reform at the Board as well a schedule for developing and implementing 6 areas of functionality through the end of calendar year 2018. However, VA's May and August updated plans and integrated management schedule (IMS) do not include specific steps or goals related to overall functionality, integrated testing, or IT training for staff on new functionality still to be implemented in 2018 and beyond. More significantly, while the VA's use of the agile process for IT development can help mitigate risks and avoid cost overruns and delays, we continue to believe that VA should define schedules beyond 6 months. For example, VA's August 2018 updated plan provides very high level plans only through March 2019 with respect to 4 areas of Caseflow functionality, while the August IMS does not include any plans for Caseflow development and implementation after 2018, even though such longer-term planning would help ensure that all potential changes are anticipated in the plans of various VA components. We will consider closing this recommendation when VA has produced a complete plan for developing, implementing and integrating all functionality envisioned under Caseflow. We will continue to track whether VA addresses IT training under recommendation 3 in GAO-18-352.
Recommendation: To further align efforts to address appeals workload and improve timeliness of decisions, and reduce the risk that efforts will not go as planned, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits; the Chairman, Board of Veterans' Appeals; and the Chief Information Officer, as appropriate, to develop a schedule for IT updates that explicitly addresses when and how any process reform will be integrated into new systems and when Caseflow will be ready to support a potential streamlined appeals process at its onset.
Agency Affected: Department of Veterans Affairs
Status: Closed - Implemented
Comments: As of May 2018, VA developed a new forecast model that allows it to change assumptions related to key variables-both individually and in conjunction with one another. Variables include rates that veterans with pending appeals opt out of the legacy process into the new one, refile rates (in cases where veteran agrees with appeal decision under new system), Board remand rates under new and legacy process, annual number of VBA claim decisions and claim disagreement rates (appeals), productivity under new and legacy processes, percent of appeals under various options in the new process, and staff assigned to work in various VBA review options and Board appeal dockets. Among other things, VA anticipates that this model will allow the agency to project potential budget needs and staffing requirements and more accurately predict resolution of legacy appeals given certain assumptions. Further, VA anticipates using the model to determine an efficient distribution of resources, and quickly react to changes in its pending legacy and new appeals processes, as well as changes in claimant behavior and other trends. While VA's actions taken to date address this recommendation, we will continue monitoring VA's application of this forecast model to inform decision making related to implementing appeals reform, pursuant to recommendation 4 in GAO-18-352.
Recommendation: To further align efforts to address appeals workload and improve timeliness of decisions, and reduce the risk that efforts will not go as planned, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits; the Chairman, Board of Veterans' Appeals; and the Chief Information Officer, as appropriate, to conduct additional sensitivity analyses based on the assumptions used in projection models to more accurately estimate future appeals inventories and timeliness. In doing so, consider running additional analyses on how these factors, in conjunction with one another, may affect the timeliness and cost of deciding pending appeals.
Agency Affected: Department of Veterans Affairs
Comments: VA agreed in principle with this recommendation. Since our report, Congress passed the Veterans Appeals Improvement and Modernization Act of 2017, which required VA to produce a comprehensive appeals plan for, among other things, monitoring implementation of the new appeals system consistent with our 2017 recommendation. As of August 2018, completed and reported various actions for monitoring implementation of appeals reform. At the broadest level, VA produced a comprehensive plan for implementing appeals reform (Nov. 2017) and subsequently issued three updates to the plan (February, May and August 2018). As of August 2018, VA's plans reflected a range of efforts consistent with this recommendation, including: (1) continuous updates to its Integrated Master Schedule, which outlines steps and timetables for implementing appeals reform as envisioned by the Appeals Modernization Act; (2) continuous identification of risks and updates to risk mitigation strategies associated with implementing appeals reform; (3) designation of project management leads who meet regularly to coordinate all activities (as of May 2018); (4) testing of appeals reform at VBA (as of November 2017) and the Board (as of May 2018) to collect empirical data on its effectiveness; (5) Board efforts to work with a change management expert (as of May 2018) and VBA's development of a change management plan (as of August 2018); and (6) development of surveys of veterans who have active appeals to assess their experience with the legacy and eventually the new appeals process (as of May 2018). While VA has made great strides monitoring implementation of the new process, as of August 2018, VA has not established complete metrics and interim goals, such as interim goals for 2 of 3 appeal options at the Board level. Further, questions still remain as to whether, how and when VA will use a balanced set of measures, including overall timeliness, accuracy and productivity, to monitor implementation and efficacy of the new appeals process. We will consider closing this recommendation when VA's comprehensive plan reflects a complete and balanced set of metrics to track progress, evaluate efficiency and effectiveness, and identify trouble spots. Closure of this recommendation is related to recommendation 2 in GAO-18-352.
Recommendation: To further align efforts to address appeals workload and improve timeliness of decisions, and reduce the risk that efforts will not go as planned, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits; the Chairman, Board of Veterans' Appeals; and the Chief Information Officer, as appropriate, to develop a more robust plan for closely monitoring implementation of process reform that includes metrics and interim goals to help track progress, evaluate efficiency and effectiveness, and identify trouble spots.
Agency Affected: Department of Veterans Affairs
Comments: VA concurred in principle with this recommendation. Moreover, since our March 2017 report, Congress passed the Veterans Appeals Improvement and Modernization Act of 2017, which required VA to produce a comprehensive appeals plan that required VA, among other things, to periodically publish a range of metrics, including timeliness, related to the processing of appeals under the new and legacy system. As of August 2018, VA reported on its plans to measure decision times by appeal option, as well as use customer satisfaction surveys and sensitivity analyses to measure the success of the new appeals process. In it August plan, VA reported contracting with an outside firm to survey veterans with active appeals and interview veterans who receive decisions, to determine their experience and satisfaction with the legacy and (eventually) new appeals processes. Regarding decision timeliness, while we agree that timeliness metrics for each new appeal option could be valuable for VA, the Congress, and the public, because veterans may pursue more than one option under VBA, the Board or both, we believe that VA's approach does not take into account the veteran's perspective of how long it took for them to receive a final appeal decision. We believe metrics on how long it takes a veteran to receive a final appeal decision would complement, not replace, metrics for VBA, the Board, and each option. In its August 2018 update to its comprehensive plan, VA maintains that it does not have a timeliness goal that measures legacy appeals processing from the date the appeal is filed to when it is finally resolved because the current legacy process has no defined endpoint. However, this circumstance does not preclude VA from treating VBA and Board decisions to fully or partially grant, and Board decisions to not grant as endpoints in baseline measures. The absence of such metrics raises questions as to whether the new process achieves final resolution of veterans' claims quicker than the old process. Further, while overall timeliness is a key metric, it is only one of several key measures of success, including accuracy and productivity. The Appeals Modernization Act requires VA to report on productivity. In August 2018, VA reported VBA productivity for legacy versus non legacy reviews, and indicated it anticipates having the required IT functionality to report required metrics by the end of calendar year 2018, with additional enhancements as of March 2019. Moreover, VBA and the Board anticipate using existing systems and goals to track the accuracy and quality of cases in the legacy and new processes. However, VA's updated plans do not fully explain how VA will use the Act's metrics to assess the relative performance of the new and legacy appeals processes on issues like accuracy, veteran satisfaction, or productivity. For example, in reporting relative VBA productivity, VBA is reflecting only those non-legacy decisions that have been decided most quickly, which is not a fair comparison. We will consider closing this recommendation when VA fully develops or implements its plan for using metrics consistent with this recommendation-such as overall appeals resolution timeliness, accuracy and productivity-with which VA can demonstrate that is has improved veterans' experiences with its disability appeals process. Closure of this recommendation is related to recommendation 2 in GAO-18-352.
Recommendation: To better understand whether appeals process reform, in conjunction with other efforts, has improved timeliness, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits; the Chairman, Board of Veterans' Appeals; and the Chief Information Officer, as appropriate to develop a strategy for assessing process reform--relative to the current process--that ensures transparency in reporting to Congress and the public on the extent to which VA is improving veterans' experiences with its disability appeals process.
Agency Affected: Department of Veterans Affairs