Information Technology:

Veterans Affairs Can Further Improve Its Development Process for Its New Education Benefits System

GAO-11-115: Published: Dec 1, 2010. Publicly Released: Dec 1, 2010.

Additional Materials:

Contact:

Valerie C. Melvin
(202) 512-6304
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Post-9/11 GI Bill was signed into law in June 2008 and provides educational assistance for veterans and members of the armed forces who served on or after September 11, 2001. The Department of Veterans Affairs (VA) is responsible for processing claims for these new education benefits. VA concluded that its legacy systems and manual processes were insufficient to support the new benefits and, therefore, began an initiative to modernize its benefits processing capabilities. The long-term solution was to provide a fully automated end-to-end information technology (IT) system to support the delivery of benefits by December 2010. VA chose an incremental development approach, called Agile software development, which is intended to deliver functionality in short increments before the system is fully deployed. GAO was asked to (1) determine the status of VA's development and implementation of its IT system to support the implementation of education benefits identified in the Post-9/11 GI Bill and (2) evaluate the department's effectiveness in managing its IT project for this initiative.

VA has made important progress in delivering key automated capabilities to process the new education benefits. Specifically, it deployed the first two of four releases of its long-term system solution by its planned dates, thereby providing regional processing offices with key automated capabilities to prepare original and amended benefit claims. In addition, the Agile process allowed the department the flexibility to accommodate legislative changes and provide functionality according to business priorities. While progress has been made, VA did not ensure that certain critical tasks were completed that were initially expected to be included in the second release by June 30, 2010. For example, the conversion of data from systems in the interim solution to systems developed for the long-term solution was not completed until August 23, 2010. Because of the delay, VA planned to reprioritize the functionality that was to be included in the third release. Further, while VA plans to include full self-service capabilities to veterans, it will not do so in the fourth release as scheduled; instead it intends to provide this capability after the release or in a separate initiative. VA reported obligations and expenditures for these releases, through July 2010, to be approximately $84.6 million, with additional planned obligations of $122.5 million through fiscal year 2011. VA has taken important steps by demonstrating a key Agile practice essential to effectively managing its system development--establishing a cross-functional team that involves senior management, governance boards, key stakeholders, and distinct Agile roles. In addition, VA made progress toward demonstrating three other Agile practices--focusing on business priorities, delivering functionality in short increments, and inspecting and adapting the project as appropriate. Specifically, to ensure business priorities are a focus, VA established a vision that captures the project purpose and goals and established a plan to maintain requirements traceability. To aid in delivering functionality, the department established an incremental testing approach. It also used an oversight tool, which was intended to allow the project to be inspected and adapted by management. However, VA could make further improvements to these practices. In this regard, it did not (1) establish metrics for the goals or prioritize project constraints; (2) always maintain traceability between legislation, policy, business rules, and test cases; (3) establish criteria for work that was considered "done" at all levels of the project; (4) provide for quality unit and functional testing during the second release, as GAO found that 10 of the 20 segments of system functionality were inadequate; and (5) implement an oversight tool that depicted the rate of the work completed and the changes to project scope over time. Until VA improves these areas, management will lack the visibility it needs to clearly communicate progress and unresolved issues in its development processes may not allow VA to maximize the benefits of the system. To help guide the full development and implementation of the long-term solution, GAO is recommending that VA take five actions to improve its development process for its new education benefits system. VA concurred with three of GAO's five recommendations and provided details on planned actions, but did not concur with the remaining two.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: VA took actions that addressed this recommendation. For example, the department provided information showing it had completed in November 2011 a dashboard that included performance measures which align with goals and objectives of the IT project. The information also identified project constraints and described their impact on the vision and expectations for the project. As a result, VA has taken steps to identify and prioritize constraints that would impact how decisions affecting the project should be made; it also has developed the means to compare the projected performance and actual results of its goals, fostering management's ability to reduce the risks of unresolved issues in its development processes. These actions should better position VA to fully maximize the system for processing Post-9/11 GI Bill education benefits.

    Recommendation: To help guide the full development and implementation of the Chapter 33 long-term solution, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits to establish performance measures for goals and identify constraints to provide better clarity in the vision and expectations of the project.

    Agency Affected: Department of Veterans Affairs

  2. Status: Closed - Implemented

    Comments: VA took steps that addressed this recommendation. In particular, in November 2011, the department developed a special report in the Rational Team Concert tool that provides links from the Chapter 33 GI Bill regulations to the system requirements documents, as well as links back to the regulations in another tool called the Rational Requirements Composer. VA provided us a copy of the Regulation to Requirements report that is produced from the bi-directional Rational Team Concert tool. The department also provided evidence that, going forward in the systems modernization efforts, it intends to use these tools to achieve bi-directional traceability for the requirements developed to support Public Law 111-377 legislation that will be included in the department's release 5.0. As a result of these actions, VA now has the capability to validate that the Chapter 33 legislative and regulatory requirements have been implemented. Further, by establishing bi-directional traceability of system requirements to legislation, policies, and business rules, the department has increased its assurance that the requirements will be fully met.

    Recommendation: To help guide the full development and implementation of the Chapter 33 long-term solution, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits to establish bidirectional traceability between requirements and legislation, policies, and business rules to provide assurance that the system will be developed as expected.

    Agency Affected: Department of Veterans Affairs

  3. Status: Closed - Implemented

    Comments: VA took action to address this recommendation. In particular, the Agile development teams developing the Post-9/11 GI Bill system formally defined the definition of "done" and provided evidence of this definition. In this regard, at an October 2012 sprint review, department officials showed us the department's definition that is located on the VA development teams' wiki page. Department officials also provided us a copy of the documented definition. By providing a mutually agreed-upon definition of "done" across VA teams, the department has established standards to help assess the quality of work that is considered to be complete.

    Recommendation: To help guide the full development and implementation of the Chapter 33 long-term solution, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits to define the conditions that must be present to consider work "done" in adherence with agency policy and guidance.

    Agency Affected: Department of Veterans Affairs

  4. Status: Closed - Not Implemented

    Comments: The agency disagreed with, and did not take action to address, this recommendation.

    Recommendation: To help guide the full development and implementation of the Chapter 33 long-term solution, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits to implement an oversight tool to clearly communicate velocity and the changes to project scope over time.

    Agency Affected: Department of Veterans Affairs

  5. Status: Closed - Not Implemented

    Comments: The agency disagreed with, and did not take action to address, this recommendation.

    Recommendation: To help guide the full development and implementation of the Chapter 33 long-term solution, the Secretary of Veterans Affairs should direct the Under Secretary for Benefits to improve the adequacy of the unit and functional testing processes to reduce the amount of system rework.

    Agency Affected: Department of Veterans Affairs

 

Explore the full database of GAO's Open Recommendations »

Sep 25, 2014

Sep 23, 2014

Jun 10, 2014

May 22, 2014

May 12, 2014

May 8, 2014

May 7, 2014

Apr 2, 2014

Looking for more? Browse all our products here