Veterans Benefits Administration:

Progress Made in Long-Term Effort to Replace Benefits Payment System, but Challenges Persist

GAO-07-614: Published: Apr 27, 2007. Publicly Released: Apr 27, 2007.

Additional Materials:

Contact:

Valerie C. Melvin
(202) 512-6304
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Since 1996, the Veterans Benefit Administration (VBA) has been working on an initiative to replace its aging system for paying compensation and pension benefits. In 2005, concerned about the slow pace of development, VBA contracted with the Software Engineering Institute (SEI) for an independent evaluation of the project, known as the Veterans Service Network (VETSNET). SEI advised VBA to continue working on the project at a reduced pace while addressing management and organization weaknesses that it determined had hampered the project's progress. GAO was requested to determine to what extent the VETSNET project has followed the course of action recommended by SEI and describe the project's current status. To perform its review, GAO analyzed project documentation, conducted site visits, and interviewed key program officials.

VBA is generally following the course of action recommended by SEI by continuing to work on the replacement initiative at a reduced pace, while taking action to address identified weaknesses in overall management and software development processes. For example, VBA established a new governance structure for the initiative that included senior management and involved all stakeholders, and it incorporated all critical areas of system development in an integrated master schedule. However, not all of SEI's management concerns have been addressed. For example, SEI advised VBA to ensure that stakeholders take ownership responsibility for the project, including the total system and process operating costs; however, although VBA is tracking costs incurred by contractors, it is not yet tracking and reporting in-house costs incurred by the project. Further, although the project has improved its management processes, such as establishing a process to manage and stabilize system requirements, it has not yet developed processes for capacity planning and management. This will be important for ensuring that further VETSNET development does not lead to delays and slowdowns in processing of benefits. In addition, although the project has established certain performance measures, it has not yet established results-oriented measures for productivity and user satisfaction, both of which will be important for measuring progress. Finally, the process improvements that VBA has incorporated in the replacement initiative remain in draft and have not been established through documented policies and procedures. If VBA does not institutionalize these improvements, it increases the risk that they may not be maintained through the life of the project or be available for application to other development initiatives. After more than 10 years of effort, including the recent management, organizational, and process improvements, VBA has developed critical functionalities needed to process and pay certain original compensation claims using the replacement system, but it remains far from completing the project. According to VBA officials, all five of the major software applications that make up the new system are now being used to establish and process new compensation claims for veterans. In total, the replacement system is currently providing monthly compensation payments to almost 50,000 veterans (out of about 3 million veterans who receive such payments); the system was used to process about 83 percent of all new compensation claims completed in March 2007. Nonetheless, the system requires further development before it can be used to process claims for the full range of compensation and pension benefits available to veterans and their dependents.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In June 2009, VA's CIO established the Program Management Accountability System (PMAS) as a standardized approach to improve the department's processes, including risk and requirements management, for projects such as VETSNET. PMAS includes a "red flag escalation process" for raising risks and issues that jeopardize successful project completion. In addition, PMAS addresses requirements management and specifies that business owners document and validate requirements for initial incremental deliverables before work can begin. Defect management, while not dictated by PMAS, is the responsibility of individual projects; and accordingly, the VETSNET project performs defect management activities.

    Recommendation: To enhance the likelihood that the replacement system will be successfully completed and implemented, the Secretary of Veterans Affairs should direct the CIO to document and incorporate the improved processes for managing risks, requirements, and defects into specific policy and guidance for the replacement initiative and for future use throughout VBA.

    Agency Affected: Department of Veterans Affairs

  2. Status: Closed - Implemented

    Comments: VA undertook a strategy for implementing VETSNET that mitigated the need for return of records to the Benefits Delivery Network (BDN). This strategy was designed with a 95 percent confidence level and included automated processes and reports to ensure records are converted successfully. Further, the department developed validation plans and designated validation team members for record conversions at its regional offices. In September 2010, VA had converted over 4.3 million of the approximately 5.2 million records from the BDN into its corporate database. As of August 2011, 24,879 (less than one percent) of all compensation and pension records remained on the BDN for conversion. The department intends to complete conversion of these records by October 2012. Also, no records have been returned from VETSNET to BDN.

    Recommendation: To enhance the likelihood that the replacement system will be successfully completed and implemented, the Secretary of Veterans Affairs should direct VBA to (1) develop a strategy to address the risk that large numbers of records may need to be returned to the Benefits Delivery Network (BDN); (2) determine whether a greater confidence level for accuracy should be required in the conversion process; and (3) develop a detailed validation plan that includes the scheduled conversions for each regional office and the validation team members needed for that specific conversion.

    Agency Affected: Department of Veterans Affairs

  3. Status: Closed - Implemented

    Comments: VA has taken actions that demonstrated consistent monitoring of the VETNET replacement system's capacity over time. Since 2007, the department has been engaged in and made resources available for various capacity planning activities for VETSNET. For example, the department developed an operational support document that outlined capacity requirements for VETSNET. Further, since 2008, VETSNET capacity has been monitored on a quarterly basis by a capacity planning workgroup within VA's information technology organization. In addition, the VETSNET executive team meets weekly to discuss critical system issues and remediation plans specifically related to system performance metrics and storage capacity.

    Recommendation: To enhance the likelihood that the replacement system will be successfully completed and implemented, the Secretary of Veterans Affairs should direct the VETSNET project to include activities for capacity planning in the VETSNET Integrated Master Schedule and ensure that resources are available for these activities.

    Agency Affected: Department of Veterans Affairs

  4. Status: Closed - Implemented

    Comments: VA has taken measures that address this recommendation. Specifically, in 2007, the department began to track costs associated with business line and information technology employees involved in the VETSNET project, including travel expenses. In addition, in fiscal years 2010 and 2011, the department provided information to the Office of Management and Budget that reflected VA personnel costs associated with the VETSNET project.

    Recommendation: To enhance the likelihood that the replacement system will be successfully completed and implemented, the Secretary of Veterans Affairs should direct the Chief Information Officer (CIO) to institute measures to track in-house expenditures for the project.

    Agency Affected: Department of Veterans Affairs

  5. Status: Closed - Implemented

    Comments: VA has developed performance metrics for VETSNET that include measures to track the volume of claims processed using the VETSNET suite of applications, the dollar amount of benefits paid, the number of beneficiaries being paid, and the number of claims converted from the legacy system to VETSNET. In addition, the VETSNET team continues to have regular interaction with the user community and tracks "help desk" requests to gauge user satisfaction with the system. VETSNET performance metrics, as well as information on system defects and system outages are reported to VBA's senior level portfolio governance board on a regular basis.

    Recommendation: To enhance the likelihood that the replacement system will be successfully completed and implemented, the Secretary of Veterans Affairs should direct the replacement project to develop effective results-oriented performance measures that show changes in efficiency, economy, or improvements in mission performance, as well as measures of user satisfaction, and to monitor and report on the progress of the initiative according to these measures.

    Agency Affected: Department of Veterans Affairs

 

Explore the full database of GAO's Open Recommendations »

Nov 19, 2014

Nov 7, 2014

Oct 30, 2014

Sep 17, 2014

Sep 10, 2014

Sep 9, 2014

Sep 2, 2014

Jul 14, 2014

Jun 23, 2014

Jun 18, 2014

Looking for more? Browse all our products here