This is the accessible text file for GAO report number GAO-07-1045 entitled 'Veterans Affairs: Improved Planning Needed to Guide Development and Implementation of Education Benefits System' which was released on July 31, 2007. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to the Subcommittee on Economic Opportunity, Committee on Veterans' Affairs, House of Representatives: United States Government Accountability Office: GAO: July 2007: Veterans Affairs: Improved Planning Needed to Guide Development and Implementation of Education Benefits System: Veterans' Education Benefits System: GAO-07-1045: GAO Highlights: Highlights of GAO-07-1045, a report to the Subcommitee on Economic Opportunity, Committee on Veterans’ Affairs, House of Representatives Why GAO Did This Study: The Department of Veterans Affairs (VA) administers programs that provide educational resources to veterans, service members, reservists, and eligible family members of veterans. Under an initiative that the Veterans Benefits Administration (VBA) began in 1999, called The Education Expert System (TEES), VA is moving its education benefits claims processing systems to new technology platforms and replacing the current systems with a new system that is to add more automated processes and eliminate most human intervention. GAO was requested to (1) report on the results of TEES development to date and identify the remaining work to complete the project, (2) identify VBA’s plans for completing and implementing TEES, and (3) assess the feasibility of developing and deploying TEES before the completion of VBA’s Finance and Accounting System. To accomplish these objectives, GAO analyzed relevant documentation, interviewed program and project officials, and observed processing systems at VA regional offices. What GAO Found: To date, as part of the TEES project, VBA has enhanced education benefits claims processing by developing certain capabilities that allow information to be captured in an electronic format. However, VBA has not taken action on various other initiatives that it identified as necessary to complete the project. These initiatives include (1) moving the functionality provided by its aging, stand-alone system used to process flight school, on-the-job training, and apprenticeship claims to a platform compliant with the department’s enterprise architecture; (2) moving the processing and payment functionality used for many of the education claims from its current antiquated system to new technology; and (3) creating a rules-based claims processing system that will process all education benefits and types of training. VBA does not have an integrated project management plan for the TEES initiative. According to agency officials, a project management plan originally developed in 2001 has not been updated since 2004. Because VBA does not have an integrated project management plan, it lacks critical elements needed to effectively guide the project to completion, such as a full description of its scope; identification of stakeholders’ responsibilities, documented risks, and performance metrics; and an overall approach for coordinating its various education claims improvement initiatives. Moreover, without an integrated project management plan, VBA risks initiating work that could be duplicative and overlapping. VBA’s Finance and Accounting System (FAS) is substantially complete and can currently pay new compensation claims; all functionality for compensation and pension payments is scheduled to be completed by August 2008. Thus, since much of the functionality for FAS has been developed, VBA could implement a rules-based system that integrates with FAS. However, while the current strategy calls for using FAS as the payment system for TEES, the existing functional requirements for an interface between the two systems, which were developed in 2003, have not been assessed and updated. Subject matter experts needed to define business requirements for interfacing the systems have not been assigned by a key stakeholder. As a result, it is uncertain when VBA will validate and update functional requirements for the FAS interface with TEES. What GAO Recommends: To ensure the successful implementation of TEES, GAO is recommending that the Secretary of Veterans Affairs, among other actions, require the development of a comprehensive, integrated project plan to coordinate and manage the initiative. Commenting on a draft of this report, VA concurred with GAO’s recommendations and described actions planned to address them. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-1045]. To view the full product, including the scope and methodology, click on the link above. For more information, contact Valerie Melvin at (202) 512-6304 or melvinv@gao.gov. [End of section] Contents: Letter: Conclusions: Recommendations: Agency Comments and Our Evaluation: Appendix I: Veterans Affairs: Improved Planning Needed to Guide Implementation of The Education Expert System (TEES): Appendix II: Comments from the Department of Veterans Affairs: Appendix III: GAO Contact and Staff Acknowledgments: Abbreviations: FAS: Finance and Accounting System: OMB: Office of Management and Budget: TEES: The Education Expert System: VA: Department of Veterans Affairs: VBA: Veterans Benefits Administration: United States Government Accountability Office: Washington, DC 20548: July 31, 2007: The Honorable Stephanie Herseth Sandlin: Chair: The Honorable John Boozman: Ranking Member: Subcommittee on Economic Opportunity: Committee on Veterans' Affairs: House of Representatives: Under an initiative called The Education Expert System (TEES), the Department of Veterans Affairs (VA) is taking steps intended to improve its education benefits delivery services by moving its education claims processing systems to new technology platforms and a new architecture. The effort, which the Veterans Benefits Administration (VBA) initiated in December 1999, is expected to enable faster and more accurate processing of education claims by replacing the existing education benefits systems within VBA's antiquated Benefits Delivery Network with a new rules-based system that will add more automated capabilities and eliminate most human intervention. When it began the initiative, VBA had planned to complete the new system by September 2005; however, in 2004, the department refocused and rebaselined the system's development effort. VA currently estimates that the TEES initiative will be completed by 2011. At your request, we examined VA's actions on the TEES initiative. Specifically, our objectives were to: * report the results of VBA's development of TEES to date, and identify the remaining work needed to complete the system; * identify VBA's plans for completing and implementing TEES; and: * assess the feasibility of developing and deploying TEES application processing functions before the completion of VBA's Finance and Accounting System (FAS). To address these objectives, we obtained and analyzed documentation describing the planned and ongoing activities for TEES and FAS, including functional requirements documents, system development and code conversion contracts for the initiative, regional office processing metrics, and cost information. We supplemented our analysis with interviews of education program officials, project managers, and other officials responsible for developing the new system. In addition, we observed education claims processing and the systems currently used to support claims processing at two of VA's four regional offices-- Atlanta, Georgia and Muskogee, Oklahoma.[Footnote 1] We performed our work at VA's headquarters in Washington, D.C., and at the Atlanta and Muskogee Regional Offices from October 2006 through May 2007, in accordance with generally accepted government auditing standards. On May 25, 2007, we provided your offices with briefing slides (included as appendix I) that outlined the results of our study. Subsequently, we met with your staff on June 15, 2007, to further discuss our findings and recommendations. The purpose of this report is to provide the published briefing slides to you and to officially transmit our recommendations to the Secretary of Veterans Affairs. In summary, our study highlighted the following: * VBA has, to date, enhanced education benefits claims processing by developing certain functionalities to allow information to be captured in an electronic format. For example, it has developed automated systems that allow (1) education institutions to provide online enrollment certifications, (2) students to provide online and telephonic verification of enrollment, and (3) the public to inquire about approved academic programs, licensing and certification programs, and national exams. However, VBA has not taken action on various other initiatives that it identified as necessary to complete the new system and eliminate most human intervention. These initiatives include (1) moving the functionality provided by its aging, stand-alone system used to process flight school, on-the-job training, and apprenticeships claims to a platform compliant with the department's enterprise architecture; (2) moving the processing and payment functionality used for many of the education claims from its current antiquated system to new technology; and (3) creating a rules-based claims processing system that will process all education benefits and types of training. * VBA does not have an integrated project plan for the TEES initiative. According to agency officials, the plan that had been developed in 2001 has not been updated since 2004, when program goals were modified. Agency officials told us the most current detailed information on planned TEES-related efforts is the fiscal year 2008 TEES Exhibit 300, submitted to the Office of Management and Budget (OMB), which includes an initiative that could duplicate the Benefits Delivery Network code conversion effort. Because VBA does not have an integrated project management plan, it lacks: * critical elements needed to effectively guide the initiative to completion--such as a full description of the scope of the system development efforts, identified stakeholders' responsibilities, documented risks, and performance metrics; and: * an overall approach for coordinating its various education claims initiatives, such as its Benefits Delivery Network code conversion effort and the development of a rules-based system to improve education claims processing. * Since much of the functionality for FAS has been developed, VBA could implement a rules-based system that is integrated with FAS. However, while the current strategy calls for using FAS as the payment system for TEES, the existing functional requirements for an interface between these two systems, which were developed in 2003, have not yet been assessed and updated. Subject matter experts needed to define business requirements for interfacing the systems have not been assigned by a key stakeholder. Thus, it is uncertain when VBA will validate and update functional requirements for this interface. Conclusions: While VBA has completed several new initiatives to provide more automated assistance for processing education claims, the absence of an integrated project plan for TEES could place the department at risk of wasting millions of dollars on education claims processing initiatives that may overlap or be duplicative. Further, without updated requirements, VA's success in interfacing the new system with FAS, and ultimately processing education benefits payments more efficiently, could be jeopardized. Recommendations: To ensure the successful implementation of a system that provides consistent and effective delivery of education benefits payments, we are recommending that the Secretary of Veterans Affairs, before approving any new funding for TEES and related systems, direct VA's Chief Information Officer to take the following three actions: * Develop a comprehensive, integrated TEES project plan that: 1. defines critical elements, including but not limited to project scope, budget, schedule, risk, planning for data management, needed knowledge and skills, and stakeholder involvement; and: 2. addresses redundancy between the requests for developing a TEES rules-based system, the department-level code conversion effort, and integration of the TEES development with FAS. * Once an integrated plan among related TEES initiatives has been completed and documented, revise the OMB Exhibit 300s for TEES and related efforts to ensure that duplication in efforts and budgets is eliminated and that funding is consistent with timelines. * Obtain stakeholders' commitment of resources to ensure that functional requirements are revalidated and updated as necessary to provide an interface between FAS and the future VBA education systems. Agency Comments and Our Evaluation: We received written comments on a draft of this report from the Deputy Secretary of Veterans Affairs. (The department's comments are reproduced in appendix II.) In the comments, VA generally agreed with our findings and concurred with our recommendations. The department acknowledged that there is still much to be done to ensure that TEES is properly planned and developed. The comments described actions planned that respond to our recommendations, such as obtaining independent contractor support to conduct a TEES program assessment and validation of consolidated functional requirements in conjunction with defining and implementing a program-level governance organization. In addition, the department stated that its Education Project Management Service would work with the responsible project manager to ensure TEES timelines, requirements, risks, and budget requirements are accurately reflected in VA's FY09 OMB Exhibit 300 submission. Further, it stated that Education Project Management Service would coordinate with related program offices, such as Benefits Delivery Network Rehosting and the Finance and Accounting System, to ensure that intersections and dependencies have been identified and accounted for in all schedules, and eliminate the possibility of redundant efforts. If the planned actions are properly implemented and monitored by agency leadership, they should help to ensure that TEES will be successfully completed. The department also provided technical comments on the draft report, which we incorporated as appropriate. We are sending copies of this report to the Chairman and Ranking Member of the Committee on Veterans' Affairs. We are also sending copies to the Secretary of Veterans Affairs and other appropriate congressional committees. We will make copies available to other interested parties upon request. Copies of this report will also be made available at no charge on GAO's Web site at [hyperlink, http://www.gao.gov]. Should you or your staff have any questions about this report, please contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. Signed by: Valerie C. Melvin: Director, Human Capital and Management: Information Systems Issues: [End of section] Appendix I: Veterans Affairs: Improved Planning Needed to Guide Implementation of The Education Expert System (TEES): Briefing to the Staff of the Subcommittee on Economic Opportunity House Committee on Veterans' Affairs: May 25, 2007: Table of Contents: Introduction: Objectives: Scope and Methodology: Results in Brief: Background: Development Results to Date and Remaining Work: Plans for Completing TEES: Feasibility of Deploying TEES before FAS Conclusions: Recommendations: Agency Comments and Our Evaluation: Introduction: The Department of Veterans Affairs (VA) provides valuable benefits and services to veterans and their families, including educational resources to compensate for opportunities missed because of military service. The timely provision of such services is central to the department’s mission. According to the Veterans Benefits Administration (VBA),1: the average number of days to complete original (first-time) education claims to payment increased from 33 to 40 from fiscal year 2005 to 2006 (VBA’s 2006 goal was 27 days) and: the average number of days to complete supplemental education claims2 increased from 19 to 20 from 2005 to 2006 (VBA’s 2006 goal was 13 days). 1: As reported in the department’s Fiscal Year 2005 and Fiscal Year 2006 Performance and Accountability Reports. 2: Supplemental claims are used for subsequent school enrollment after an original claim award or for changes in information such as address change. Under an initiative called The Education Expert System (TEES), VA is taking steps to improve its education services by moving its education claims processing systems to new technology platforms and a new architecture. This effort, begun in December 1999, is intended to provide faster and more accurate processing of education claims by replacing existing education benefit systems with a new system that will add more automated processes, eliminate most human intervention, and work effectively within the evolving VA corporate environment.3: VA currently estimates that the TEES development effort will cost $77.4 million and be completed by 2011.4: 3: TEES is a follow-on to an April 1998 prototype that successfully processes certain education claims automatically. 4: This estimate is shown in VA's 2008 OMB Exhibit 300. Objectives: We were asked to: report the results of VBA's development of TEES to date, and identify the remaining work to complete the system, identify VBA’s plans for completing and implementing TEES, and: assess the feasibility of developing and deploying TEES application processing functions before the completion of VBA’s Finance and: Accounting System (FAS). 5: 5: FAS is used to develop a payment record, generate various accounting reports, and support generation and audit of benefit payments. Scope and Methodology: To respond to this request, we: researched laws related to education benefits; obtained and analyzed documentation describing the planned and ongoing activities for TEES and FAS, including functional requirements documents, system development and code conversion contracts for the TEES initiative, regional office processing metrics, and cost information; interviewed education program officials at VBA headquarters and interviewed project managers and other officials responsible for TEES in VA’s Office of Information and Technology about the various actions taken to develop the system and costs incurred; and: observed education claims processing and the systems currently used to support claims processing at two of the four regional offices— Atlanta, Georgia, and Muskogee, Oklahoma—and discussed the systems with personnel in those offices. 6: Our work was performed primarily at VA headquarters in Washington, D.C., from October 2006 to May 2007 in accordance with generally accepted government auditing standards. 6: Together these offices processed about 58 percent of total education claims received in fiscal year 2006. Muskogee was selected because VBA has identified Muskogee as proficient in education claims processing. Results in Brief: To date, VBA has enhanced education benefits claims processing by developing certain functionalities to allow documents to be captured in an electronic format. These enhancements include: an automated system that allows education institutions to provide online enrollment certification; an automated system to allow students to provide online verification of enrollment; an application to allow students to phone in their monthly verifications; and an automated system to enable public inquiry about approved academic programs, licensing and certification programs, and national examinations. However, VBA has not taken action on various other initiatives that it identified as necessary to move its education claims processing to new technology platforms and a new architecture and eliminate most human intervention. These initiatives include: moving the functionality provided by its aging, stand-alone system used to process flight school, on the job training, and apprenticeships claims to a platform compliant with the department’s enterprise architecture; moving the processing and payment functionality used for many of the education claims from its current antiquated system to new technology; and: creating a rules-based claims processing system: 7 that will process all education benefits and training types. 7: This system will use logical rules to calculate eligibility dates and award amounts, minimizing the need for human intervention in processing most education claims. VBA does not have an integrated project management plan for the TEES initiative. According to agency officials, the plan had not been updated since 2004 when the program goals were modified. As a result, VBA lacks: critical elements needed to effectively guide TEES to completion— such as a full description of the scope of the initiative, identified stakeholders’ responsibilities, documented risks, and performance metrics; and: an overall approach for integrating its various system development and enhancement initiatives within the department to improve education claims processing. VBA has developed the functionality for FAS to pay new compensation claims, and all functionality for compensation and pension payments is scheduled to be completed in August 2008. While the current strategy calls for using FAS as the payment system for TEES, the rules-based processing system has not yet been developed and requirements for interfacing FAS with the system have not been assessed and updated. As a result, it is unclear when VBA will be positioned to revise the functional requirements for FAS to support TEES. To ensure the successful implementation of a system that provides consistent and effective delivery of education benefits payments, we are recommending that the Secretary of Veterans Affairs take several actions, including directing VA’s Chief Information Officer to develop a comprehensive, integrated project management plan to guide the development and implementation of TEES. Background: VBA Education Service: VBA’s Education Service administers the programs that provide educational resources to veterans, service members, reservists, and eligible family members of veterans. Four regional processing offices are responsible for processing education benefits claims: Muskogee, Oklahoma: Buffalo, New York: Atlanta, Georgia: St. Louis, Missouri: VBA reported that in fiscal year 2006, these offices processed about 1.4 million claims, a 2 percent increase over fiscal year 2005. Background: Education Benefits Programs: Beneficiaries are enrolled in one of five education benefits programs: Chapter Number: Chapter 30: 38 USC §§ 3001,ff; Program Name: All-Volunteer Force Educational Assistance Program—provides education benefits to current and former active duty personnel; Percent of total enrollment [A]: 67; Chapter Number: Chapter 35: 38 USC §§ 3500,ff; Program Name: Survivors’ and Dependents’ Educational Assistance—provides education benefits to survivors and dependents for military personnel killed or totally and permanently disabled; Percent of total enrollment [A]: 15; Chapter Number: Chapter 1606: 10 USC §§ 16131-16137; Program Name: Educational Assistance for Members of the Selected Reserve—provides education benefits for members of selected reserve; Percent of total enrollment [A]: 13; Chapter Number: Chapter 1607: 10 USC §§ 16161-16166; Program Name: Reserve Educational Assistance Program—provides education benefits to National Guard and other reservists called to duty; Percent of total enrollment [A]: 5; Chapter Number: Chapter 32 38 USC §§ 3201,ff; Program Name: Post-Vietnam Era Veterans’ Educational Assistance Program—enacted to promote the all-volunteer military program; Percent of total enrollment [A]: <1; [A] The percentages of total program enrollment are for fiscal year 2006. They do not add up to 100 percent due to rounding. Background: Current Automated Processing: VBA currently relies on its aging Benefits Delivery Network to process and store education claims and calculate benefits. The Benefits Delivery Network comprises numerous separate and stove- piped systems that support education claims processing by, among other things, calculating benefits and payment authorizations for 38 U.S.C., chapters 30, 32, 35, and 10 U.S.C., chapters 1606 and 1607 beneficiaries. Background: Current Manual Processing Functions: The current claims process for all original and most supplemental education claims also requires human intervention to manually complete key processing functions. For an original claim, VBA claims examiners must: determine claimant eligibility, verify that claims payment is for an approved education program, verify that the claimant’s original signature has been received, verify that an enrollment certification has been received, and: approve education claims. For a supplemental claim, a VBA claims examiner must: process school enrollment certifications for all Chapter 35, 1606, 1607, and approximately 86 percent of Chapter 30 beneficiaries, and: manually add an amount to the rate calculated by the Benefits Delivery Network (for Chapter 30 beneficiaries who have the $600 supplement). 8: 8: Under Chapter 30, claimants can elect to contribute up to an additional $600 to their required contribution level to increase the amount of their benefit. Background: Planned TEES Development: VA defines TEES as its proposed approach to achieving the goal of processing to 90 percent of veteran education benefit applications, enrollment certifications, and notices of change in student status to final resolution without human intervention. 9: The approach would replace the existing education processing functionality of the Benefits Delivery Network with a new system that is expected to: introduce a rules-based processing capability that will automatically process a majority of all claims received electronically, process all benefit chapters and all training types, and: work within the VA/VBA corporate database and data warehouse. In addition, the approach would include development of auxiliary systems to facilitate the automated processing of education claims. VBA’s stated goal is to receive all claims information electronically. VBA began its first TEES effort in December 1999 with a scheduled completion date of September 2005. This effort was focused on developing a rules-based system. However, VA rebaselined the project in May 2004, shifting its focus from developing a rules-based system to migrating existing education systems off the Benefits Delivery Network as soon as possible. Under the rebaselined initiative, the Benefits Delivery Network migration had a scheduled completion date of December 2007 and completion of the rules-based system was moved from 2005 to 2010. Objective: Development Results To Date and Remaining Work: Since initiating TEES development in December 1999, according to VBA, $12.3 million has been allocated and used to undertake certain initiatives to eliminate reliance on the Benefits Delivery Network and enable the receipt of education claims information electronically. To reduce reliance on the Benefits Delivery Network, VBA has developed functional requirements for its rules-based processing system. According to VA officials, these requirements now need to be updated before proceeding with developing the rules-based system. For the electronic receipt and processing of claims information, VBA has: developed and maintained the Web Automated Verification of Enrollment system (which allows students to submit enrollment verification online) and the VA Online Certification of Enrollment system (which allows educational institutions to submit enrollment certification online); developed the Web Enabled Approval Management System (which merged two record-keeping systems for approved academic programs and for licensing and certification programs) and added functionality to automate a database of approved national examinations; and: implemented four education forms (application for VA education benefits, request for change of program or place of training, application for survivors and dependents’ education assistance and request for change of survivors and dependents) in the Veterans Online Application system to enable electronic submission of claims. In addition, TEES funding was used to support: in-house personnel costs for system development and maintenance (for example, the Electronic Certification Automated Processing and Chapter 1607 systems) and: enhancement and maintenance of existing systems supporting education claims processing. According to VBA, these initiatives have helped improve education claims processing by providing a capability to receive electronic claims data, thus reducing manual effort needed for verification and certification. However, they do not substantially contribute to meeting the TEES goal to move the functionality from the Benefits Delivery Network to VA’s new technology environment. VBA has not taken action on numerous other initiatives that it identified10 as necessary to eliminate its reliance on the Benefits Delivery Network and eliminate most human intervention. VBA officials stated that no funds had been allocated for TEES development from 2004 until 2007 because VA leadership had directed funding to other initiatives. In February 2007, VBA presented a revised approach for completing TEES. The initiatives identified as part of the revised approach include the following: 1.Enhancements/Replacements for Current Systems—According to VBA, enhancements and replacements need to be made to three existing systems before work begins on the rules-based application. The estimated start date for the enhancement/replacement activities is August 2007 with an implementation date of December 2008. Chapter 30 PC replacement—an initiative to replace a stand-alone workstation located at each regional processing office and used to process all claims that cannot be processed through the Benefits Delivery Network. VA officials stated that this system runs on outdated and unsupported hardware and software. Electronic Certification Automated Processing—a rules-based application that processes approximately 14 percent of Chapter 30 claims through to authorization with no human intervention. This effort is a redesign of the existing system to (1) centralize automated claims processing and (2) provide additional functionality. Work-study system—an automated application that is used to process work study claims. 2. Functional Requirements Definition—VBA plans to gather and define business requirements for building a new rules-based automated eligibility and award processing system. The estimated start date is July 2007, and completion is set for September 2008. 3. Rules-Based System—This element includes data conversion, design and build, test and certify, and implementation of the system for processing benefits under chapters 30, 35, 1606, and 1607. VA does not expect to begin development of this component until October 2008, with completion anticipated in September 2011. In the February 2007 revised approach, the Benefits Delivery Network migration initiative, which was the focus of the rebaselining in 2004, has been eliminated from the TEES initiative. Objective 2: Plans for Completing TEES: A project management plan is a necessary component for guiding the development of an effort of this size and importance. Information technology project management principles and industry best practices emphasize he importance of using a project management plan to define the technical and managerial processes needed to satisfy project requirements. The plan should, among other things, define the scope of the project; identify budget and resource requirements (i.e., personnel and facilities), identify schedules associated with key project tasks; document an assessment of risks; identify data management requirements; specify the authorities and responsibilities of all users and stakeholders; include a work breakdown structure for all tasks to be performed in developing, testing, and deploying the system; identify performance metrics; and: articulate a security policy. VA does not have a project management plan to coordinate and manage the TEES efforts. Although VBA had developed a project management plan for TEES in 2001, its officials stated that the plan was not updated after the 2004 effort to rebaseline TEES. According to agency officials, the most current detailed information on planned TEES-related efforts is the fiscal year 2008 TEES OMB Exhibit 300. However, the Exhibit 300 is not intended to be a project management plan. The Exhibit 300 is a reporting mechanism intended to enable an agency to demonstrate to its own management, as well as OMB, that it has employed the disciplines to develop a strong business case for the investment and meet other administrative priorities in defining cost, schedule,and performance goals proposed for the investment. 11: 11: We have reported elsewhere on the purpose and use of the Exhibit 300. See, GAO, Information Technology: Agencies Need to Improve the Accuracy and Reliability of Investment Information, GAO-06-250 (Washington, D.C.: Jan. 12, 2006). Lacking an integrated project plan, VA does not have a defined approach to accomplish the tasks that it has identified to modernize its education systems, and it risks duplication of initiatives. To illustrate this lack of integration, in December 2006, VBA officials stated that they were proceeding with a contract to develop functional requirements for the Benefits Delivery Network migration initiative in spring 2007. However, in September 2006, VA’s Office of Information and Technology had initiated a separate, department-level code conversion initiative to end VBA’s reliance on the Benefits Delivery Network to process education claims. The code conversion effort would duplicate most of the Benefits Delivery Network migration effort to end reliance on the existing system that had previously been identified as the focus of the rebaselined effort. In addition, the need for most elements of the code conversion effort could be eliminated by the development of the TEES rules-based initiative scheduled for completion at exactly the same time— September 2011. In response to our inquiries regarding this separate initiative, VA officials stated that they would not include the Benefits Delivery Network migration effort in the 2009 business case for the TEES initiative. VA officials also stated that they had not maintained current plans for TEES initiatives because there had been no funds available to continue development. At the conclusion of our review, VA officials acknowledged the importance of having an integrated plan to guide the development and completion of TEES. Objective 3: Feasibility of Deploying TEES before FAS: Moving to a rules-based system to increase automated processing would require an automated payment system to replace the Benefits Delivery Network payment function or an interface with the existing Benefits Delivery Network. The current strategy calls for using the Finance and Accounting System to replace the payment function in the Benefits Delivery Network and serve as the payment system for TEES. VBA’s Compensation and Pension business line is currently developing FAS, and substantial parts of the system have been completed (FAS is currently scheduled to replace the Benefits Delivery Network payment function for new compensation claims in August 2007 and all functionality for compensation and pension payments is scheduled to be completed in August 2008). In order for the education program to utilize FAS, an interface must be developed. However, according to VA officials, existing functional requirements for a TEES interface with FAS, which were developed in 2003, have not yet been updated. Subject matter experts needed to define business requirements for interfacing FAS with TEES have not been assigned by a key stakeholder. Thus, it is uncertain when VBA will validate and update functional requirements for the FAS interface with TEES. 31 Conclusions: Since TEES began in 1999, VBA has completed several new initiatives to provide more automated assistance for processing education claims. However, VBA has not taken action on other initiatives, and the TEES functionality that would move its education claims processing to new technology and a new architecture and eliminate most human intervention, is not scheduled to begin development until December 2008. TEES lacks an integrated project plan to guide its efforts and ensure program completion in an efficient manner. In the absence of such a plan, the department-level Benefits Delivery Network code conversion effort has not been integrated with TEES. Until VA has an integrated project plan for TEES that is approved by stakeholders, the Department faces the risk of wasting millions of dollars on education claims processing initiatives that overlap or, in some cases, could be duplicative. Since much of the functionality of FAS has been developed, VBA could implement a rules-based system that integrates with FAS. However, functional requirements for an interface between FAS and the future VBA education systems have not been assessed and as necessary revised, and there are no committed resources from stakeholders identified to support the revision of this interface. Recommendations: To ensure the successful implementation of a system that provides consistent and effective delivery of education benefits payments, we are recommending that the Secretary of Veterans Affairs, before approving any new funding for TEES and related systems, direct VA’s Chief Information Officer to take the following three actions: (1) Develop a comprehensive, integrated TEES project plan that: defines critical elements, including but not limited to project scope, budget, schedule, risk, planning for data management, needed knowledge and skills, and stakeholder involvement; and: addresses redundancy between the requests for developing a TEES rules- based system, the department-level code conversion effort, and integration of the TEES development with FAS. (2) Once an integrated plan among related TEES initiatives has been completed and documented, revise the OMB exhibit 300s for TEES and related efforts to ensure that duplication in efforts and budgets is eliminated and that funding is consistent with timelines. (3) Obtain stakeholders’ commitment of resources to ensure that functional requirements are revalidated and updated as necessary for an interface between FAS and the future VBA education systems. Agency Comments and Our Evaluation: VA provided oral comments on a draft of this briefing. In the comments, the Associate Deputy Under Secretary for Policy and Program Management expressed concern that we had not adequately reflected the department's accomplishments for the TEES initiative, such as development of the prototype for electronic certification of Chapter 30 supplemental claims. We have now included additional information in the briefing on this prototype. In addition, the department provided technical comments, which we have incorporated as appropriate. [End of section] Appendix II: Comments from the Department of Veterans Affairs: The Deputy Secretary Of Veterans Affairs: Washington: July 18, 2007: Ms. Valerie C. Melvin: Director: Human Capital and Management Information: and System Issues: U. S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Ms. Melvin: The Department of Veterans Affairs (VA) has reviewed the Government Accountability Office's (GAO) draft report, VETERANS AFFAIRS: Improved Planning Needed to Guide Development and Implementation of Education Benefits System (GAO-07-1045) and generally agrees with your findings and concurs with your recommendations. VA acknowledges there is still much to be done to ensure The Education Expert System (TEES) is properly planned and developed. VA now has a more robust enterprise architecture and a new consolidated information technology organization that is turning its attention to TEES. The enclosure specifically addresses your recommendations and describes VA's plans for successfully implementing TEES. VA appreciates the opportunity to comment on your draft report. Sincerely yours, Signed by: Gordon H. Mansfield: Enclosure: Department of Veterans Affairs (VA) comments to Government Accountability Office (GAO) draft report Veterans Affairs: Improved Planning Needed to Guide Development and Implementation of Education Benefits System (GAO-07-1045) To ensure the successful implementation of a system that provides consistent and effective delivery of education benefits payments, GAO recommends that the Secretary of Veterans Affairs, before approving any new funding for TEES and related systems, direct VA's Chief Information Officer to take the following three actions: 1. Develop a comprehensive, integrated TEES project plan that: * Defines critical elements, including but not limited to project scope, budget, schedule, risk, planning for data management, needed knowledge and skills, and stakeholder involvement; and: * Addresses redundancy between the requests for developing a TEES rules- based system, the department-level code conversion effort, and integration of the TEES development with FAS. Concur – The VA Information Technology (IT) realignment has resulted in the development of an IT Governance Plan that will address how the Department will manage IT resources under a centralized organization and ensure the successful implementation of IT systems, including The Education Expert System (TEES). The VA Chief Information Officer (CIO) has taken steps to establish the necessary governance boards approved in the IT Governance Plan. The governance boards include the Business Needs and Investment (BNI) Board; Planning, Architecture, Technology, and Services (PATS) Board; and IT Leadership Board (ITLB). These boards, in conjunction with the Strategic Management Council (SMC), constitute the decision making structure needed to centrally manage IT resources. For TEES, it should be noted that a signed charter, project management plan, and requirements management plan exist. The 2001 charter contains signatures from Education Service, Veterans Benefits Administration (VBA) Under Secretary's Office, VBA Office of Resource Management (ORM), and VBA's Office of Information Management. The Veterans Benefits Information Technology Development Program Executive Office (VBIT) agrees that this project management documentation should be updated to present the "going forward" plan and resubmitted for key stakeholder concurrence. Department of Veterans Affairs (VA) comments to Government Accountability Office (GAO) draft report VETERANS AFFAIRS: Improved Planning Needed to Guide Development and Implementation of Education Benefits System (GAO-07-1045): (Continued): Action Plan: The Education Project Management Service (EPMS) has a pending acquisition package to obtain independent contractor support to conduct a TEES Program Assessment and Validate the Consolidated TEES Functional Requirements Document (August 2003). The program assessment will result in a written report that documents the outcomes of the assessment, provides recommendations on how the program should proceed, and provides an integrated schedule that maps out how the program objectives can be accomplished. Concurrently with the assessment, the consolidated functional requirements will be validated and the document updated to reflect any change in the requirements/business rules. The validated functional requirements document will also incorporate requirements and business rules for new benefits (such as the Chapter 1607) and new or changed business processes. The anticipated award date for this contract is September 15, 2007, and the period of performance for these activities is 8 months. In conjunction with this effort, the EPMS is defining and implementing a program-level governance organization, gathering current project documentation so that it can be updated and routed for signature by key stakeholders, and developing an integrated TEES project schedule. 2. Once an integrated plan among related TEES initiatives has been completed and documented, revise the OMB exhibit 300s for TEES and related efforts to ensure that duplication in efforts and budgets is eliminated and funding is consistent with timelines. Concur – Again, the VA IT realignment has resulted in a governance process that will influence TEES as well as other Exhibit 300 initiatives in ensuring duplication of efforts and budgets are eliminated. It should be noted that VBIT is restructuring its OMB Exhibit 300s for FY09 to reflect delivery of functional capabilities across the enterprise vs. business specific programs/projects. Requirements for TEES will be included in the new Automated Claims Processing Assistance Solution OMB Exhibit 300. The Education Project Management Service (EPMS) will work with the responsible project manager for the Automated Claims Processing Assistance Solution OMB Exhibit 300 to ensure TEES timelines, requirements, risks and budget requirements are accurately reflected in the VA's FY09 submission. In addition, based on timeline constraints, the EPMS will work with the appropriate Office of Information and 2 Department of Veterans Affairs (VA) comments to Government Accountability Office (GAO) draft report VETERANS AFFAIRS: Improved Planning Needed to Guide Development and Implementation of Education Benefits System (GAO-07-1045): (Continued): Technology (01&T) staff to re-baseline the FY08 TEES OMB Exhibit 300 to ensure that timelines and funding are consistent with stated TEES objectives. Action Plan: The EPMS will use the results of the TEES Program Assessment and integrated scheduling efforts to revise and update the OMB Exhibit 300 submissions. The EPMS will coordinate with related program offices, such as Benefits Delivery Network (BDN) Rehosting and the Finance and Accounting System (FAS), to ensure that intersections and dependencies have been identified and accounted for in all schedules, which will result in an elimination of the possibility of redundant efforts. 3. Obtain stakeholders' commitment of resources as necessary to ensure that functional requirements are revalidated and updated as necessary for an interface between FAS and the future VBA education systems. Concur - The EPMS currently has an acquisition underway to obtain support to conduct a validation of the existing Consolidated TEES Functional Requirements document. Through the revised TEES Charter and Program Management Plan, the EPMS will obtain stakeholder commitment of resources to ensure that the consolidated functional requirements (including FAS interface requirements) are revalidated and properly documented. Action Plan: The EPMS will revise current project documentation such as the charter, project management plan, requirements management plan, risk management plan, and integrated schedule to reflect TEES program direction and objectives. The EPMS will obtain concurrence from key stakeholders on the revised documents to ensure buy-in and support. The requirements validation effort being undertaken by the EPMS will elicit any requirements associated with new education benefits programs, such as chapter 1607, and new education business processes. The current document requirements contain the functional requirements associated with sending education transactions to FAS for accounting and payment. These requirements were originally developed, documented and concurred upon by the ORM. Stakeholder commitment to these activities will be obtained through project documentation such as the charter and program management plan. [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: Valerie C. Melvin (202) 512-6304 or melvinv @gao.gov: Staff Acknowledgments: In addition to the contact named above, key contributions to this report were made by Barbara Oliver, Assistant Director; Charles Youman; B. Scott Pettis; Matthew R. Grote; and Amos A. Tevelow. Footnotes: [1] Education benefit claims are processed at four VA regional offices. GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office: 441 G Street NW, Room LM: Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400: U.S. Government Accountability Office, 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Susan Becker, Acting Managing Director, AndersonP1@gao.gov (202) 512- 4800: U.S. Government Accountability Office, 441 G Street NW, Room 7149: Washington, D.C. 20548: