Veterans Benefits Management System:

Ongoing Development and Implementation Can Be Improved; Goals Are Needed to Promote Increased User Satisfaction

GAO-15-582: Published: Sep 1, 2015. Publicly Released: Sep 15, 2015.

Additional Materials:

Contact:

Valerie C. Melvin
(202) 512-6304
MelvinV@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Veterans Benefits Administration (VBA) within the Department of Veterans Affairs (VA) has made progress in developing and implementing the Veterans Benefits Management System (VBMS), with deployment of the system to all of its regional offices as of June 2013. While 95 percent of records related to veterans' disability claims are electronic and reside in the system, additional capabilities have not yet been completed, such as automation of the steps associated with a veteran's request for an increase in benefits. Further, VBA has not yet developed and implemented pension processing capabilities in VBMS, nor has it articulated when the system will support appeals processing. The VBMS program reported receiving funding of about $1 billion from fiscal years 2009 to 2015, at which time system completion was originally planned. Although development of the system is expected to continue beyond 2015, the incremental approach VA is using to develop and implement VBMS has not yet produced a plan that identifies when the system will be completed and can be expected to fully support disability and pension claims processing and appeals. Thus, it will be difficult for VA to hold its managers accountable for meeting its time frame and for demonstrating progress.

As VA continues its efforts to complete development and implementation of the system, three areas could benefit from increased management attention.

Cost estimating: The program office does not have a reliable estimate of the cost for completing the system. Without such an estimate, VA management and the department's stakeholders have a limited view of the system's future resource needs, and the program risks not having sufficient funding to complete development and implementation of the system.

System availability: Although VBA has improved its performance for ensuring the system is available to users, it has not established system response time goals. Without such goals, users do not have an expectation of the system response times they can anticipate and management does not have an indication of how well the system is performing relative to performance goals.

System defects: While the program has actively managed system defects, a recent system release included unresolved defects that impacted system performance and users' experiences. Continuing to deploy releases with large numbers of defects that reduce system functionality could adversely affect users' ability to process disability claims in an efficient manner.

While VBA has employed various methods to obtain VBMS users' feedback, it has neither established goals to define user satisfaction, nor conducted a survey of claims processing employees to obtain a more comprehensive picture of overall customer satisfaction. GAO's survey of VBMS users estimated that a majority report satisfaction with the system, but that one group of users who are responsible for examining claims decisions was considerably less satisfied. Although the results of GAO's survey provide VBA with useful data about users' satisfaction with the system, the absence of user satisfaction goals limits the utility of survey results. Specifically, without having established goals to define user satisfaction, VBA does not have a basis for gauging the success of its efforts to promote satisfaction with the system.

Why GAO Did This Study

VBA pays disability benefits for disabling conditions incurred or aggravated while in military service, while pension benefits are for low-income veterans who are either elderly or have disabilities unrelated to military service. In fiscal year 2014, the department paid about $58 billion in disability compensation and about $5 billion in pension claims. The disability claims process has been the subject of attention by Congress and others, due in part, to long waits for processing claims and a large backlog of claims. To process disability and pension claims more efficiently, VA began implementation of an electronic, paperless system in 2009. GAO was asked to study VBMS. Specifically, GAO (1) assessed VA's progress toward completing the development and implementation of VBMS and (2) determined to what extent users report satisfaction with the system. To do so, GAO reviewed relevant program documentation, administered a survey to a stratified random sample of about 3,500 users, and interviewed appropriate VA officials.

What GAO Recommends

GAO recommends that VA develop a plan for completing VBMS, establish goals for system response time, minimize the incidence of high and medium priority system defects for future VBMS releases, assess user satisfaction, and establish satisfaction goals to promote improvement. VA concurred with the recommendations and described actions it is planning to take in response, except for the first recommendation. GAO continues to believe development of a plan for completing the system is important.

For more information, contact Valerie C. Melvin at (202) 512-6304 or MelvinV@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Priority recommendation

    Comments: The Department of Veterans Affairs (VA) concurred with our recommendation and as of January 2020, is continuing to develop requirements for VBMS in order to develop functionality to replace legacy information systems. In addition, the department subsequently provided us with expected completion dates for implementation of claims and appeals processing, but has not provided a schedule for the implementation of pension claims processing. To fully implement this recommendation, the department needs to provide the expected completion date for pension claims processing and an estimate of the cost to complete remaining development and implementation of VBMS.

    Recommendation: To improve VA's efforts to effectively complete the development and implementation of VBMS, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits and the Chief Information Officer to develop an updated plan for VBMS that includes (1) a schedule for when VBA intends to complete development and implementation of the system, including capabilities that fully support disability claims, pension claims, and appeals processing and (2) the estimated cost to complete development and implementation of the system.

    Agency Affected: Department of Veterans Affairs

  2. Status: Closed - Implemented

    Comments: The Department of Veterans Affairs (VA) concurred with this recommendation and reported that the Veterans Benefits Management System (VBMS) program office has developed draft metrics for performance of the system. Specifically, VA stated that the office has established key performance indicators as a basis for monitoring the response times of the most commonly executed user transactions (or work events) within VBMS. According to the department, these indicators have been incorporated into the application's continuous monitoring tools for all service level agreements and these agreements are enforced by the VA Service Level Management Board. In April 2018, the department provided the goals for system response time and the reported VBMS response times that are tracked on a monthly basis. As a result, VA is better equipped to track the performance of VBMS against established goals for the system's performance.

    Recommendation: To improve VA's efforts to effectively complete the development and implementation of VBMS, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits and the Chief Information Officer to establish goals for system response time and use the goals as a basis for periodically reporting actual system performance.

    Agency Affected: Department of Veterans Affairs

  3. Status: Open

    Comments: The Department of Veterans Affairs (VA) concurred with this recommendation and reiterated its plans and procedures for decreasing the incidences of defects in each system release. However, while the most recent VBMS release (i.e., May 2019) showed a decrease in the number of high- and medium-priority level defects, the release in February 2019 showed an increase in the number of high- and medium-priority defects. In addition, both the February 2019 and May 2019 releases showed the presence of the highest severity defects--critical--which have extensive user impact and workarounds do not exist. We will continue to monitor VA's actions and progress in response to this recommendation.

    Recommendation: To improve VA's efforts to effectively complete the development and implementation of VBMS, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits and the Chief Information Officer to reduce the incidence of high- and medium-priority level defects that are present at the time of future VBMS releases.

    Agency Affected: Department of Veterans Affairs

  4. Status: Closed - Not Implemented

    Comments: The Department of Veterans Affairs (VA) concurred with this recommendation and in January 2017, conducted a survey of VBMS users that was sent to over 16,000 claims processors at each of its 56 regional offices. Although 52 percent of respondents indicated that they were very satisfied or satisfied with VBMS, the department received only about 2500 responses to the survey for a 15 percent response rate. This low response rate raises concern about whether the survey results were statistically valid. We asked VA about actions the department has taken to ensure the statistical validity of its survey results and, per its response in July 2019, VA did not take any actions to ensure the statistical validity of the survey and does not have any plans to take any further action.

    Recommendation: To improve VA's efforts to effectively complete the development and implementation of VBMS, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits and the Chief Information Officer to develop and administer a statistically valid survey of VBMS users to determine the effectiveness of steps taken to make improvements in users' satisfaction.

    Agency Affected: Department of Veterans Affairs

  5. Status: Closed - Not Implemented

    Comments: The Department of Veterans Affairs (VA) concurred with this recommendation and conducted a survey of VBMS users in January 2017. However, per its response in July 2019, the department had yet to develop customer satisfaction goals for VBMS that would provide users with an expectation of the system response times they should anticipate, and management with an indication of how well the system is performing relative to performance goals. Additionally, the department indicated that it does not plan to conduct another survey or develop customer satisfaction goals for VBMS.

    Recommendation: To improve VA's efforts to effectively complete the development and implementation of VBMS, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits and the Chief Information Officer to establish goals that define customer satisfaction with VBMS and report on actual performance toward achieving the goals based on the results of GAO's survey of VBMS users and any future surveys VA conducts.

    Agency Affected: Department of Veterans Affairs

 

Explore the full database of GAO's Open Recommendations »

Mar 3, 2020

Feb 19, 2020

Feb 6, 2020

Jan 23, 2020

Jan 15, 2020

Jan 2, 2020

Dec 11, 2019

Dec 10, 2019

Dec 2, 2019

Looking for more? Browse all our products here