This is the accessible text file for GAO report number GAO-14-144 entitled 'Transitioning Veterans: Improved Oversight Needed to Enhance Implementation of Transition Assistance Program' which was released on March 5, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Committees: March 2014: Transitioning Veterans: Improved Oversight Needed to Enhance Implementation of Transition Assistance Program: GAO-14-144: GAO Highlights: Highlights of GAO-14-144, a report to congressional committees. Why GAO Did This Study: Over the next few years, over a million military servicemembers are expected to transition to civilian life and some may face challenges such as finding employment. To help them, TAP provides departing servicemembers employment assistance and information on VA benefits, among other things. Begun in 2011, efforts to revamp TAP are underway based on the VOW to Hire Heroes Act of 2011 and the administration's recommendations. The act also mandated GAO to review TAP. This report addresses: 1) the status of TAP implementation; 2) the extent to which elements of effective implementation and evaluation of TAP have been addressed; and 3) any challenges that may remain. To do this GAO identified five elements of effective implementation and evaluation based on relevant federal laws and previously established GAO criteria for training programs; reviewed related GAO work; assessed reports, plans, and policies provided by agencies that administer TAP; interviewed officials from entities that support servicemembers and veterans; and conducted four nongeneralizable discussion groups with servicemembers who had taken TAP at three military installations. What GAO Found: The Departments of Defense (DOD), Labor (DOL), and Veterans Affairs (VA) have implemented most of the key components of the Transition Assistance Program (TAP), a gateway to information and services available to servicemembers transitioning to civilian life. However, the agencies are still in the process of implementing other key components of TAP. While originally planned for October 2013, agencies now plan to implement virtually all components by the end of March 2014, with full implementation expected by June 2014. Agencies' efforts are underway to adequately address three of five elements that GAO identified as important for effective implementation and evaluation of TAP: 1--Track attendance: DOD has systems to collect and report on attendance, which help measure the extent to which TAP achieves its attendance goals. 2--Ensure training quality: The agencies collect and plan to use participant feedback on instruction, content, and facilities to improve training. Each agency also plans to monitor its respective TAP components through site visits. 3--Assess career readiness: The agencies developed standards to assess servicemembers' career readiness. During a capstone assessment, commanders are expected to verify and document whether standards were met. Agencies' efforts to address the remaining two elements are mixed: 4--Ensure participation and completion: DOD has assigned commanders the responsibility for overseeing participation and has required the services to schedule training and communicate its importance to servicemembers. While the Army and Air Force gauge participation at the command level, the Navy and Marines lack a similar oversight mechanism. 5--Measure performance and evaluate results: The agencies have established certain measures to assess program performance, but their TAP evaluation approach is incomplete. For example, the agencies have established measures to track program outputs, such as the percentage of servicemembers who have participated in TAP. However, the agencies' efforts to evaluate TAP results have focused on basic end-of-course evaluations and gauging servicemembers' readiness prior to separation instead of higher impact program evaluations, such as assessing the effectiveness of TAP on servicemembers 6 months after they have separated from the military. According to agency officials, such evaluations are being considered for certain components of TAP, but they could not provide GAO with a justification for including or excluding specific components of TAP in their evaluation planning efforts. Based on GAO's prior work and according to officials from the agencies and organizations GAO spoke with, a key remaining challenge for TAP may be the unfavorable timing and location of program delivery for National Guard and Reserve members. Unlike active duty servicemembers, National Guard and Reserve members receive TAP services closer to their transition and in locations that are generally neither where they work nor live. As a result, they may be distracted and have less time to benefit from TAP services. DOD is not well positioned to verify these concerns because it is not collecting data about these members' experiences with the timing and location of TAP service delivery. What GAO Recommends: GAO recommends that DOD improve oversight and implementation of TAP, including actions to gauge participation for all of the services and collect data about National Guard and Reserve members' experiences. DOD disagreed with two of GAO's three recommendations. GAO continues to believe that the recommendations are needed as discussed in the report. View [hyperlink, http://www.gao.gov/products/GAO-14-144]. For more information, contact Andrew Sherrill at (202) 512-7215 or sherrilla@gao.gov. [End of section] Contents: Letter: Background: Agencies Report That Most Locations Have Implemented Key TAP Components, but Some Locations Are Experiencing Delays: Agencies' Efforts for TAP Adequately Address Three of Five Key Elements Associated with Effective Program Implementation and Evaluation: DOD Lacks a Process to Assess the TAP Experience of National Guard and Reserve Members: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Technical Appendix: Appendix II: DOD Form 2958, Individual Transition Plan (ITP) Checklist: Appendix III: Comments from the Department of Defense: Appendix IV: Comments from the Department of Veterans Affairs: Appendix V: GAO Contact and Staff Acknowledgments: Tables: Table 1: Selected Statutory and Task Force Requirements for Transition Assistance Program (TAP): Table 2: Elements and Relevant Attributes and Extent They Have Been Addressed: Figures: Figure 1: Revamped Transition Assistance Program (TAP) Components, Activities, Providers, and Timeframes: Figure 2: Status of Implementing Key Components of Revamped Transition Assistance Program (TAP) as of December 2, 2013: Abbreviations: DOD: Department of Defense: DODI: DOD Instruction: DOL: Department of Labor: DTAP: Disabled Transition Assistance Program: ITP: Individual Transition Plan: MOU: Memorandum of Understanding: Post 9/11 GI Bill: Post 9/11 Veterans Educational Assistance Act of 2008: SBA: Small Business Administration: TAP: Transition Assistance Program: Transition GPS: Transition Goals, Plans, Success: VA: Department of Veterans Affairs: VEI: Task Force DOD-VA Veterans Employment Initiative Task Force: VOW Act: VOW to Hire Heroes Act of 2011: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: March 5, 2014: Congressional Committees: Over the next 4 to 5 years, more than a million servicemembers are expected to leave the military and transition into civilian life.[Footnote 1] Some of these new veterans will face significant challenges, such as finding and maintaining employment. To help them, the Transition Assistance Program (TAP) has provided departing servicemembers counseling and offered employment assistance and information on federal veteran benefits, among other things. Amid concerns about the effectiveness of TAP and indicators that some post-9/11 veterans were having difficulty transitioning to civilian employment, Congress passed the VOW to Hire Heroes Act of 2011 (VOW Act).[Footnote 2] Among other changes to TAP, the VOW Act mandates that the Department of Defense (DOD) require departing servicemembers to participate in a workshop on finding employment as part of TAP. Concurrently, the administration initiated a redesign of TAP. As part of the redesign, all departing servicemembers are required to complete a set of tasks that demonstrate their readiness for a civilian career. According to the administration, these efforts are the first major redesign of TAP since the program's inception over 20 years ago and represent a transformed approach. To oversee the design and development of the revamped TAP and related efforts, the administration called on DOD and the Department of Veterans Affairs (VA) to lead the DOD-VA Veterans Employment Initiative Task Force (VEI Task Force) with the help of other federal partners.[Footnote 3] The VOW Act also mandated that GAO review TAP.[Footnote 4] Specifically, this report addresses: 1. the status of efforts to implement changes to key components of TAP, 2. the extent to which the agencies administering TAP have addressed elements important for the effective implementation and evaluation of TAP, and: 3. remaining challenges, if any, that the agencies administering TAP may need to resolve. To determine the status of efforts to implement changes to key components of TAP, we reviewed (1) VEI Task Force's and related reports; (2) relevant federal laws and regulations; (3) program documentation, including policies, procedures, and supporting implementation plans; and (4) testimonies from the federal agencies responsible for administering TAP and several veterans' service organizations. We also interviewed officials from these agencies and organizations. In addition, we obtained information about the extent to which the components of TAP were implemented at installations providing TAP. This information was provided by DOD and the services and included the status of implementation by service and installation. To assess the reliability of these data, we interviewed officials and obtained information from each service on their processes for collecting and reviewing these data, which included the revised VA benefits I and II briefings, three career tracks, and a capstone event. The remaining TAP components (such as pre-separation counseling and the Department of Labor (DOL) employment workshop) were essentially implemented prior to our review, so we did not obtain implementation dates by TAP location for these TAP components. Instead we relied on DOD reports and interviews with officials from the agencies and services to determine if these components were implemented at all TAP locations. We found the data to be sufficiently reliable for the purposes of reporting on the status of implementation. To assess efforts to implement and evaluate the revamped TAP, we identified five elements important for the effective implementation and evaluation of TAP. The elements are: (1) systems to track attendance; (2) processes for ensuring quality of instruction, content, and facilities; (3) mechanisms to assess career readiness; (4) mechanisms to ensure participation and completion; and (5) measures of performance and processes to systematically evaluate the effectiveness or benefits of TAP. We identified the elements based on a review of relevant federal laws, including the VOW Act, and VEI Task Force recommendations for changing TAP, as well as a GAO assessment guide that identifies elements of effective training and development programs.[Footnote 5] We also provided the elements to DOD and VA officials--as co-leaders of the VEI Task Force--for review and comment, and they were generally in agreement. Additional information about these elements and relevant attributes is provided in appendix I. To determine the extent to which agencies have addressed the five elements, we reviewed program documentation from the agencies and discussed the policies and mechanisms with knowledgeable officials. We compared the elements with this documentation and results of our discussions with officials at these agencies. To identify any remaining challenges, we reviewed prior GAO and other reports related to TAP and veteran employment programs;[Footnote 6] relevant federal laws and regulations; and testimonies from the federal agencies responsible for administering TAP and several veterans' service organizations. We also interviewed officials from federal agencies responsible for administering TAP about any challenges they have faced in implementing the revamped program. In addition, we interviewed officials from organizations that support transitioning servicemembers and veterans to obtain their perspectives on any challenges the agencies face and what additional actions may be needed. To increase our understanding and obtain servicemembers' views of TAP, we conducted three site visits during which we observed portions of the revised TAP training sessions available at the time of our site visits. We also conducted 4 discussion groups with 28 servicemembers from all four services who had taken the training. We visited Joint Base Lewis-McChord, Washington (US Army/Air Force), Marine Corps Base Quantico, and Naval Station Norfolk. We selected locations that had implemented changes to TAP, and that provided for geographic diversity and representation of all four services, but the information we collected from these sites is not generalizable.[Footnote 7] Because interagency partners were in the process of implementing changes to TAP and the revamped program was not fully operational at the time of our review, we were not able to determine whether the policies and procedures were in place at all sites or if they were working as intended.[Footnote 8] Similarly, we were not able to comment on whether the changes to TAP are yielding desired benefits or improvements in outcomes. We conducted this performance audit from January 2013 to March 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: Statutory and VEI Task Force Changes to TAP: Prior to the redesign initiated in 2011, TAP consisted of four core components: (1) pre-separation counseling, (2) an employment workshop, (3) an optional briefing on federal veteran benefits, and (4) the Disabled Transition Assistance Program. Pre-separation counseling, which includes VA benefits information, was required by law to be provided prior to the VOW Act.[Footnote 9] A number of revisions, new requirements, and components were added to TAP by the VOW Act and the VEI Task Force. For example, the VOW Act mandates that DOD require that transitioning servicemembers participate in an employment workshop, with some exceptions. Included among the VEI Task Force's changes to the program are: * an extended curriculum with segments on translating military skills to civilian job requirements, financial planning, and individual counseling and assessment with the goal of each servicemember developing an Individual Transition Plan; * an updated employment workshop and briefings on federal veteran benefits divided into two sessions.[Footnote 10] The VA benefits I and II briefings embed information relevant for those who have or think they have a service-connected disability rather than as part of a separate Disabled Transition Assistance Program component; * a series of 2-day, career-specific tracks that focus on (1) pursuing college education, (2) entering a technical skills training program, or (3) starting a business. The track that a servicemember chooses depends on his or her post-separation goals; * a capstone event during which servicemembers are to demonstrate--and military service commanders verify--that they have met required career- readiness standards. The standards pertain to employment, education, and technical training, depending on the servicemember's post- separation goals; and: * a referral process--called a "warm handover"--to connect servicemembers who do not meet the career readiness standards with the appropriate partner agency (VA or DOL) to provide continued support and services as veterans. Additional details about the previous TAP compared to the revamped program are shown in table 1. Table 1: Selected Statutory and Task Force Requirements for Transition Assistance Program (TAP): Activity: Pre-separation counseling; TAP requirements before the VOW Act and VEI Task Force[A]: Participation in pre-separation counseling is generally mandated by law, and is to include information on VA benefits, among other information; VOW Act requirements: The VOW Act made the use of the services under 10 U.S.C. §1144, including assistance in identifying employment and training opportunities, and help in obtaining such employment and training, mandatory; VEI Task Force requirements: Pre-separation counseling now to include a needs assessment as well as introduce the requirement to develop an Individual Transition Plan (ITP). Activity: Individual Transition Plan (ITP); TAP requirements before the VOW Act and VEI Task Force[A]: As part of pre-separation counseling, creation of an ITP is required; VOW Act requirements: No change; VEI Task Force requirements: All servicemembers are required to consider their personal circumstances and transition plans in their ITP and any indicators of risk are determined by what is included in the ITP. Activity: Financial planning workshop; TAP requirements before the VOW Act and VEI Task Force[A]: As part of pre-separation counseling, the financial planning workshop is required; VOW Act requirements: No change; VEI Task Force requirements: Servicemembers are required to prepare a 12-month, post-separation budget that reflects their career or training goals. Activity: Translating military skills to civilian job requirements; TAP requirements before the VOW Act and VEI Task Force[A]: DOD was required to provide service members a certification/verification of the job skills/experience they acquired that may apply to the civilian sector; VOW Act requirements: DOL is required to conduct a study on equivalency between military and civilian jobs; DOD is to use the results to conduct individualized military-to-civilian skills review; VEI Task Force requirements: Servicemembers are to receive help documenting their military experiences and identifying gaps between this experience and that needed for civilian jobs. Activity: DOL employment workshop; TAP requirements before the VOW Act and VEI Task Force[A]: DOD was required to "encourage and otherwise promote maximum participation" in the program; VOW Act requirements: Effective Nov. 21, 2012, DOD is generally required to mandate participation in the program, with exceptions[B]; VEI Task Force requirements: The DOL employment workshop was revised. The revised workshop's three-day format is tailored to the career readiness standards developed by the Task Force. Activity: VA benefits briefing; TAP requirements before the VOW Act and VEI Task Force[A]: The VA benefits briefing consisted of a lengthy slide deck presentation and lecture; VOW Act requirements: No change; VEI Task Force requirements: Revised to consist of a 4 hour interactive overview of VA benefits and 2 hours of interactive instruction on healthcare, eBenefits, disability compensation, and how to apply for benefits.[C] The briefings are made mandatory. Activity: Career-specific tracks; TAP requirements before the VOW Act and VEI Task Force[A]: None; VOW Act requirements: None; VEI Task Force requirements: Servicemembers may select one or more two- day career tracks: (1) higher education, (2) career technical training, and (3) entrepreneurship, depending on their post-separation goals. Activity: Capstone & career readiness standards; TAP requirements before the VOW Act and VEI Task Force[A]: None; VOW Act requirements: None; VEI Task Force requirements: The capstone serves to verify that servicemembers have meet career readiness standards. Activity: Military Life Cycle Transition Model; TAP requirements before the VOW Act and VEI Task Force[A]: None; VOW Act requirements: None; VEI Task Force requirements: By Oct. 2014, the services are required to embed preparation to meet career readiness standards throughout the span of a servicemember's military career. Sources: GAO analysis of Veterans' Employment Initiative Task Force information and relevant federal laws. Notes: The legal requirements pertaining to TAP cited in the table are not exhaustive. We highlight certain key legal features. [A] This column contains legal requirements and a non-legal requirement. [B] The VOW Act allows DOD to waive the participation requirement for those for whom participation would not be of assistance because they are unlikely to face major readjustment, health care, employment, or other challenges associated with transition to civilian life; and servicemembers possessing specialized skills who, due to unavoidable circumstances, are needed to support a unit's imminent deployment. The VOW Act requires that the DOL employment workshop use contracted instructors, effective November 21, 2013. [C] The VA benefits briefing embeds information for those who have or think they have a service-connected disability. Under the previous VA benefits briefing, information provided about service-connected disabilities was called the Disabled Transition Assistance Program (DTAP). [End of table] Figure 1 shows each of the components in the order a servicemember would currently participate in the redesigned program. The services generally implement these core components at the installation level and may include other components in addition to TAP. Figure 1: Revamped Transition Assistance Program (TAP) Components, Activities, Providers, and Timeframes: [Refer to PDF for image: illustration] Preseparation counseling: Activity: Servicemember consulted and briefed on upcoming training; * Career interest assessment; * Develop Individual Transition Plan; * eBenefits registration; Provider: DOD services; Duration: Varies. Core curriculum: DOD core curriculum: * Crosswalk of military and civilian occupations; * Financial planning; * Resiliency. Provider: DOD services; Duration: 8-10 hours. Employment workshop: * Customized employment plan; Provider: DOD contractors; Duration: 3 days. Veterans' benefits: * Briefings I and II[A]; Provider: VA; Duration: 6 hours. Tailored tracks: Career-specific/tailored tracks: * Education; * Technical and skills training; * Entrepreneurship; Provider: DOD, VA, SBA; Duration: 2 days. Capstone event: Verify transitioning servicemembers: * Have achieved career readiness standards; * If not, they receive a referral “warm handover” to agencies serving veterans. Source: GAO analysis of implementing agencies' documentation. Note: For clarity purposes, under the heading "DOD core curriculum" we group a number of modules. These DOD-led modules include (1) a TAP overview and orientation session; (2) a Military Occupation Codes Crosswalk module to begin the process of identifying skills, experience, credentials, and education that have been obtained while in the military and crosswalking them to civilian opportunities; (3) a Personal Financial Planning for Transition module to develop the financial aspects of servicemembers' transition goals and a written plan to achieve them; and (4) Resilient Transitions module to introduce servicemembers to topics such as transition stress, family considerations, and the value of a mentor. [A] VA benefits I and II briefings are two sessions that provide an interactive overview of VA benefits and instruction on healthcare, eBenefits, disability compensation, and how to apply for benefits. [End of figure] Rather than TAP continuing to be an end-of-career event, DOD plans to shift to a Military Life Cycle Transition Model after October 2014. This model is intended to integrate transition preparation-- counseling, assessments, and access to resources to build skills or credentials--throughout the course of a servicemember's military career. Administration of the Revamped TAP: The VEI Task Force oversaw the design and development of the revised TAP and was led by DOD and VA. Other agencies participating on the VEI Task Force include DOL, the Department of Education, the Office of Management and Budget, the Office of Personnel Management, and the Small Business Administration (SBA). Members of the White House staff and senior representatives from each service also participated. Each agency is responsible for different activities. For example, DOD provides guidance and monitors compliance with TAP provisions, and DOL facilitates the employment workshop. Also, each service coordinates with agencies on scheduling TAP workshops and briefings. The respective roles and responsibilities are spelled out in a memorandum of understanding (MOU), which the agencies signed in January 2014. [Footnote 11] A new TAP governance structure, established in October 2013, steers implementation of TAP and will modify the program, as needed, through 2016. The new governance structure is co-led by DOD in 2014 and co-chaired by VA and DOL. Other Federal Employment Programs That Serve Transitioning Servicemembers and Veterans: With the draw down from the wars in Iraq and Afghanistan and as the military makes ongoing and planned force structure reductions, many servicemembers are projected to depart the military through 2017. TAP is one of a number of federal programs to assist transitioning servicemembers and veterans in developing job skills and securing civilian employment. TAP serves as a gateway to additional information and services that are available, either while servicemembers are on active duty or after they have separated from the military. For example the DOL employment workshop highlights many of the skills and techniques helpful in obtaining employment. After completing the workshop, servicemembers can benefit further by returning to the TAP offices on installations, using services at local VA and DOL offices, or using websites introduced to participants during TAP training. Once servicemembers separate from the military, a number of other federal programs offer assistance. These programs include five employment and training programs overseen by DOL and VA that target veterans.[Footnote 12] DOL administers four of these employment programs targeted to veterans. For example, DOL provides grants to states to support state workforce agency staff positions, Disabled Veterans' Outreach Program Specialists and Local Veterans Employment Representatives, who serve veterans through the Jobs for Veterans State Grant Program. In addition, VA provides employment services to certain veterans with disabilities through the Vocational Rehabilitation and Employment Program. As of 2012, the program was offered in 56 regional offices and 169 satellite offices. In addition, DOD helps National Guard and Reserve members obtain civilian employment though its operation of other programs, including the Yellow Ribbon Reintegration Program and Employer Support of the Guard and Reserve. For example, the Yellow Ribbon Reintegration Program serves National Guard and Reserve members and their families by hosting events that provide information on employment opportunities, health care, education/training opportunities, finances, and legal benefits. Agencies Report That Most Locations Have Implemented Key TAP Components, but Some Locations Are Experiencing Delays: As of December 2, 2013, DOD, DOL, and VA have implemented changes to the program's key components at most of the 206 military installations that provide TAP. However, a few program components have not yet been fully implemented by the agencies. For example, the agencies are still using the previous version of the VA benefits briefing at a number of locations and are offering the career technical training track at fewer than half of the TAP locations. Although some agencies had planned to fully implement the revamped TAP at all locations by October 1, 2013, they missed their targeted time frame. According to the revised plan, agencies now expect to implement virtually all components by the end of March 2014, with full implementation planned by June 2014. The planned start dates and the status of agencies' efforts to implement the key components at the 206 TAP locations are summarized in figure 2. Figure 2: Status of Implementing Key Components of Revamped Transition Assistance Program (TAP) as of December 2, 2013: [Refer to PDF for image: illustration] Preseparation counseling: Activity: Servicemember consulted and briefed on upcoming training; Provider: DOD services; Planned start: Nov. 2012; Percentage of locations implemented (as of Dec. 2013): 100%. Core curriculum: Activity: DOD core curriculum; Provider: DOD services; Planned start: Nov. 2012; Percentage of locations implemented (as of Dec. 2013): 100%. Activity: Employment workshop; Provider: DOD contractors; Planned start: Nov. 2012; Percentage of locations implemented (as of Dec. 2013): 100%. Activity: Veterans' benefits; Provider: VA; Planned start: Oct. 2013; Percentage of locations implemented (as of Dec. 2013): 92%. Tailored tracks: Activity: Career-specific/tailored tracks; Provider: DOD, VA, SBA; Planned start: Oct. 2013; Percentage of locations implemented (as of Dec. 2013): * Education: 72%; * Technical training: 43%; * Entrepreneurshipa: 72%. Capstone event: Activity: Verify career readiness standards are met[B]; Provider: DOD services; Planned start: Oct. 2013; Percentage of locations implemented (as of Dec. 2013): 92%. Source: GAO analysis of implementing agencies' documentation. Notes: This figure is intended to present the key components of TAP. For clarity, we omitted the Military Life Cycle Transition Model, which will embed transition preparation over the course of a servicemember's military career, and we group a number of training modules for which DOD is responsible under DOD core curriculum. Percentages are rounded. [A] While the entrepreneurial track is offered at about 72 percent of TAP locations, the track is only offered quarterly, and SBA officials estimate it is meeting about 12 percent of the total demand. [B] If servicemembers do not meet the career readiness standards, they are to be referred to the appropriate partner agency, a process called the "warm handover". While the capstone event is nearly implemented, the agencies are finalizing the details for how the warm handover will work. [End of figure] Pre-separation counseling and DOD Core Curriculum: According to DOD officials, eligible servicemembers were participating in pre- separation counseling and the modules in the new DOD core curriculum by November 21, 2012 at all TAP locations. As noted previously, part of the DOD core curriculum includes a module on translating military skills to civilian job requirements. Current military crosswalks map the majority of military occupations to a single civilian occupation. Based on those crosswalks and supplemented by analyses from Army and Navy credentialing websites, tools like DOL's My Next Move and My Next Move for Veterans can suggest multiple occupations for career exploration. To enhance the existing electronic tools used for the crosswalk, DOL contracted with an organization to identify equivalencies between military and civilian jobs, as required by the VOW Act. According to DOL officials, the results of the military equivalencies study will enhance the military-civilian crosswalk by enabling a mapping of a single military occupation into multiple civilian occupations based on an analysis of embedded skill sets in addition to the similarity of tasks performed. This will be done for a selected set of military occupations that represent 59 percent of current active duty servicemembers. DOL officials said they plan to complete the update of the electronic tools beginning in 2014. DOL Employment Workshop: Effective November 21, 2012, DOD was generally required to mandate participation in the program with some exceptions. According to DOD and DOL officials, the agencies met this participation requirement in the VOW Act by offering eligible servicemembers, with some exceptions,[Footnote 13] the previous or revised version of the DOL employment workshops at domestic and overseas locations by November 21, 2012. Moreover, as required by the act, the DOL employment workshops are being conducted by contractors at all TAP locations.[Footnote 14] As of the spring of 2013, the revised workshop was being offered at all TAP locations, according to DOD and DOL officials. VA Benefits Briefing: All departing servicemembers are generally required to be provided the VA benefits briefing. Officials from DOD and VA stated that this participation requirement was met by offering servicemembers the previous version of the VA benefits briefing while implementing a phased rollout of the revised VA benefits I and II briefings. As of December 2, 2013, the revised VA benefits I and II briefings were unavailable at about 8 percent of TAP locations. VA officials said that the revised VA benefits briefings are offered at all domestic locations. However, the revised briefings are not currently offered in all overseas locations, such as Air Force locations in Germany, South Korea, and Japan. Although VA planned to implement the revised benefits briefings by October 1, 2013, full implementation is now expected no later than March 31, 2014, according to VA officials. DOD and VA officials said that VA faced challenges, such as training enough personnel to facilitate the revised briefings at these overseas locations during the extended furlough and delays with negotiating agreements with foreign nations hosting U.S. military forces. Career-specific Tracks: Since late 2012 the agencies had been planning to fully implement these tracks by October 2013. However, as of December 2, 2013, the tracks were not fully implemented, particularly at overseas locations: * Entrepreneurial Track: SBA offers the entrepreneurial track at least quarterly at all domestic locations, but this only meets 12 percent of the estimated total demand for the track, according to SBA officials. Moreover, the track is not offered at the majority of overseas locations. After reviewing a draft of this report, SBA officials stated that the track can be extended to meet domestic and overseas demand given additional funding recently provided in the fiscal year 2014 budget. Full implementation at all TAP locations is expected by the end of fiscal year 2014, according to SBA officials. * Career Technical Training Track: VA is offering the career technical training track at about 43 percent of the TAP locations. In particular, many overseas locations lack this track. According to VA officials, the delays were mainly due to efforts to incorporate substantial feedback received from participants during the pilot phase. VA completely revised the curriculum and piloted it in the summer of 2013. As a result, VA delayed training the facilitators until the curriculum was finalized and began its roll out in September 2013. According to the services' implementation plans, full implementation of this track is expected by April 30, 2014. * Higher Education Track: DOD is offering the higher education track at about 72 percent of TAP locations. According to DOD officials, they plan to offer the track at all locations, but sequestration and other resource constraints as well as delays in hiring and training facilitators slowed the roll out of this track. Full implementation is expected by April 9, 2014, according to the services' implementation plans. For servicemembers lacking access to the entrepreneurial track, content from the track is available online. The online version, however, is a poor substitute for the classroom-based track, according to SBA officials. Unlike the technical training and higher education tracks, the entrepreneurial track does not have associated readiness standards. According to SBA officials, the readiness standards for the other tracks were developed prior to the inclusion of this track in TAP. While they considered creating associated standards, SBA officials decided that the track's main purpose is to help participating servicemembers determine whether or not starting a business in general or their specific ideas for a business is right for them, so standards were not needed. If the higher education or career technical training tracks are not available for servicemembers who wish to attend an institution of higher education or seek technical training, other options are available to meet the readiness standards associated with the tracks. For servicemembers in remote locations or who are rapidly separating from the military, one option is to access a virtual TAP curriculum, according to a draft DOD policy under consideration.[Footnote 15] However, classroom instruction is the preferred method. Another option for servicemembers located overseas is to take the track at a domestic location when they return to the United States, according to DOD officials. Moreover, according to DOD policy, a servicemember could meet the standards outside of the track by completing the required documents or activities associated with those standards, such as by completing a comparison of academic institution choices and a college or university application. While servicemembers may meet the readiness standards without taking the tracks, they would miss instruction, for example, on resources to cope with challenges transitioning into college as nontraditional students (those who are older or with family obligations). Capstone Event: The services, working with DOD, DOL, and VA, are finalizing implementation of a "capstone event"--a final check to verify that servicemembers have met TAP requirements--and a referral process, known as a "warm handover," for those servicemembers who do not meet the requirements. Although as of December 2013 the services were holding capstone events at most locations, according to the services' implementation plans, the details for how the capstone event and warm handover will work are still being finalized by DOD, DOL, VA, and the services. Specifically, the agencies and services must clarify the roles and responsibilities of servicemembers, TAP staff, commanders, and the partner agencies as well as develop policies and guidance that underpin this effort, according to a December 2013 DOD report about implementation of the capstone event at TAP locations. In addition to implementing the key components of TAP at military installations, the services continue to update physical infrastructure at a number of locations to provide an optimal experience for servicemembers participating in TAP components. General standards for infrastructure are set forth in DOD policy. Approximately 6 percent of locations lack computer availability and according to officials from DOD, the Navy and Marine Corps, other infrastructure is still being put in place at a number of their domestic and overseas locations. According to officials from DOD, they expect all of the TAP locations to meet infrastructure standards by March 31, 2014. Military Life Cycle Model: According to the agencies' implementation plan, the envisioned end state for the redesigned TAP involves integrating transition preparation throughout the course of a servicemember's career. The agencies refer to this end state as the military life cycle transition model. The services plan to fully implement the military life cycle by October 2014. Under this new transition model, for example, the Army intends for all new servicemembers to receive counseling and initiate an individual development plan regarding their military career goals within 30 days of reporting to their first permanent duty station. Overall, DOD intends such counseling and planning to continue throughout a servicemember's military career at various "touchpoints", such as when they are promoted.[Footnote 16] For example, servicemembers will be expected to create plans for achieving their military and post- military goals for employment, education, or starting their own business. Further, servicemembers are to be made aware of the career readiness standards they must meet long before they separate. Agencies' Efforts for TAP Adequately Address Three of Five Key Elements Associated with Effective Program Implementation and Evaluation: Agencies have efforts underway to address three of the five key elements associated with effective program implementation and evaluation for TAP. However, agencies' efforts to address the remaining two elements are mixed. Agencies Are Adequately Addressing Three Elements: Element 1: Tracking Attendance: Effective training programs have systems to track data,[Footnote 17] and we found that DOD and the services have systems to collect and report on attendance rates and are taking steps to improve the reliability of these data. DOD developed a DOD-wide system to track attendance for all TAP components, which has been in use since October 2012.[Footnote 18] This system is populated with attendance data from the services. The Army and Air Force each input TAP attendance data into their own systems, which they then transmit to the DOD system, while the Navy and the Marines input TAP attendance data directly into the DOD system because they do not have service-level systems that allow for tracking individual TAP attendance. DOD and the services are taking steps to improve the reliability of these data. According to DOD officials, accuracy will improve now that the capstone event is in place because servicemember attendance at each of the three required TAP training components will be verified at this event. According to DOD officials, the services are taking other steps to improve the accuracy of TAP attendance data, as well. For example, the Navy plans to replace the paper data collection systems that exist at some installations with electronic attendance tracking, which may be less prone to data-entry error. DOD tracks attendance to measure progress toward its performance goal of eligible servicemembers attending the mandatory components each year, beginning in fiscal year 2013.[Footnote 19] In February 2014, DOD reduced the fiscal year 2014 performance goal from 90 percent to 85 percent. According to DOD officials, the expected attendance rate is less than 100 percent because some servicemembers do not complete all required TAP training for various reasons.[Footnote 20] For example, some may separate quickly, such as for medical or disciplinary reasons, and not have an opportunity to fully take TAP. For fiscal year 2013, DOD expects an attendance rate of about 75 percent, primarily because servicemembers that transitioned in that year may have taken several TAP components in fiscal year 2012, prior to when DOD and the services began tracking these rates. DOD anticipates meeting their goal in fiscal year 2014. Element 2: Ensuring Training Quality: Effective training programs incorporate feedback into training efforts,[Footnote 21] and each agency involved in TAP plans to use this type of information, as well as monitor their respective components, to improve TAP. To measure participants' reactions to each TAP component they attend, including the career-specific tracks, DOD launched a participant assessment in April 2013. The assessment seeks feedback on the quality of the instruction, content, and facilities. [Footnote 22] It also measures participants' knowledge of the information presented in each component of the training. According to agency officials, the agencies plan to use the results from the assessment to monitor the performance and outcomes for the redesigned TAP, assess trends, determine areas of improvement, and modify TAP components as appropriate. DOD is leading this effort. To help facilitate this effort, DOD officials said that they plan to analyze feedback at the DOD, service, and installation levels and share this information with partner agencies and the four services in a quarterly report. For example, the participant assessment collects demographic information that will allow for a comparison of the responses of servicemembers in different military branches, locations, and groups, such as enlisted personnel and officers. DOD performed a similar analysis in mid-2013 for responses to two questions in the participant assessment. This analysis provided DOD with information that could be used to understand how the program's usefulness is perceived by different populations. Moreover, according to DOD's TAP evaluation plan, DOD plans to convene teams with the partner agencies on an as-needed basis to make recommendations to address challenges, concerns, and areas for improvement. According to VA and DOL officials, they also plan to analyze the feedback to improve their respective components. The agencies also monitor their respective TAP components through site visits or plan to do so. According to DOD's monitoring plan, DOD visits installations to monitor the TAP components it is responsible for as well as overall TAP implementation. According to DOL officials, their staff intends to perform annual monitoring visits to each domestic installation where the employment workshop is provided, but not at overseas installations. They said that at overseas locations they plan to rely on feedback from participant assessments, the company hired to facilitate the workshops, and the military staff at those DOD sites who will monitor the facilitators' instruction and whether they are engaging and knowledgeable. VA officials said that they have not completed their monitoring plan for the VA benefits I and II briefings and its career technical training track. However, they expect to have a quality assurance plan completed by March 2014. Also, in comments provided after reviewing a draft of this report, VA officials stated that VA is participating in the DOD site visits to installations to monitor the TAP components. These are joint, agency- administered, on-site staff assistance visits led by DOD. Also, VA currently has contract staff at each military installation to monitor implementation of the VA benefits I and II briefings and the career technical training track. Side bar: Steps to Career-Readiness: TAP career-readiness standards require: 1. Completing a 12-month post-separation budget; 2. Documenting requirements and eligibility for certification, licensure, and apprenticeship; 3. Evaluating transferability of military skills to civilian workforce and gap analysis; 4. Registration on eBenefits; 5. Completing an Individual Transition Plan; 6. Completing a Continuum of Military Service Opportunity counseling; 7. Completing a job application package or presenting a job offer letter; 8. Completing the assessment tool to identify personal interests and leanings regarding career selection; 9. Receiving a DOL Gold Card and understanding that post 9/11 veterans have priority for 6 months at DOL American Job Centers. Career-readiness standards also exist for the associated higher education and career technical training tracks. Source: DOD. [End of side bar] Element 3: Assessing Career Readiness: Effective training programs measure and document learning and make managers part of the process.[Footnote 23] In assessing transitioning servicemembers' career readiness, we found that agencies have taken the following steps: * Created the career readiness standards commanders use to ensure that TAP participants have completed products, such as resumes and job applications, which demonstrate that they are career ready. * Created a capstone event to verify that standards have been met; the capstone event can be completed one-on-one, in large groups, or in small groups. For example, the Marines model is one-on-one. * Assigned the task of verifying career readiness to commanders and identified steps to ensure that commanders or their designees are properly trained to assess an individual servicemember's career readiness.[Footnote 24] To document whether or not a servicemember is career ready, DOD developed a new form called the Individual Transition Plan (ITP) Checklist in which each one of the career readiness standards is listed. We have reprinted this form in appendix II. * Developed procedures for providing a warm handover, or referral, for servicemembers not meeting the career readiness standards. Generally, the warm handover is to be a confirmed person-to-person contact. According to the December 2013 DOD capstone report, during the recent pilot of the capstone event, several implementation challenges emerged. For example, confusion exists among servicemembers, TAP staff, commanders, and the agencies as to their roles and responsibilities in the capstone event. According to the DOD capstone report, to address this challenge, the agencies are taking actions, such as the services implementing a training program for their TAP staff and planning efforts to educate commanders and servicemembers on the requirements, purpose, and importance of the capstone event. As noted previously, according to DOD guidance, if a servicemember has not met the career readiness standards at the time of the capstone event, they are to be referred to the appropriate curriculum or services before they transition. If they do not meet the standards before separating from the service, the services plan to provide them with a "warm handover". The delivery of the warm handover may vary at domestic and overseas locations because DOL and VA have limited capacity at overseas locations.[Footnote 25] According to the DOD capstone report, DOL staff will likely not be present at capstone events overseas and will conduct the warm handover in other ways, and VA staff will be located only at larger overseas installations. For example, servicemembers overseas will be able to contact DOL's call center within established hours, which, due to time differences, may limit opportunities to make a person-to-person contact. Agencies' Efforts to Address the Remaining Two Elements are Mixed: Element 4: Ensuring Participation and Completion: Effective training programs encourage participation and hold both individuals and their leaders responsible. While DOD has taken steps that are consistent with most attributes of effective training programs, two services currently lack an oversight mechanism at the commander level to help ensure participation. DOD's efforts underway include: (1) prioritizing training for servicemembers based on agreed- upon goals and priorities, (2) encouraging servicemembers to buy in to training goals, and (3) communicating the importance of training. [Footnote 26] For example, DOD and the services communicate the importance of TAP training by providing information on their websites, including how TAP aids in a successful transition, and through other communications, such as brochures with similar information. In addition, DOD has assigned unit commanders the responsibility of ensuring that eligible servicemembers have full access to and successfully complete required TAP components.[Footnote 27] However, only the Army and the Air Force possess the capability to gauge the rate at which servicemembers under an individual unit commander participate in TAP. Specifically, the Army and Air Force provide commanders and their leaders information on their unit's participation levels.[Footnote 28] In contrast, the Navy and the Marines do not have such systems. According to Navy officials, they obtained funding and plan to develop such a system by late June 2014 and plan to start using the data for oversight no later than fiscal year 2015. Marine officials said that because they have a long- standing culture of requiring servicemembers to attend TAP training, such an oversight mechanism was not necessary.[Footnote 29] In our 2002 and 2005 reviews of TAP, we found that servicemembers sometimes faced difficulties being released from military duties to attend TAP because of the priority accorded their military mission or the lack of supervisory support for TAP.[Footnote 30] More recently, based on the pilot of the capstone event, DOD reported that ensuring servicemember participation in capstone events was a challenge for most of the services. According to DOD, lack of servicemembers' awareness of this requirement and lack of commanders' support may have hampered participation in capstone events. Without routine information on servicemembers' participation by commander, it may be difficult to hold accountable those directly responsible for ensuring participation. Element 5: Measuring Performance and Evaluating Results: Agencies should have outcome-oriented performance measures for training programs,[Footnote 31] and the agencies implementing TAP have established such measures to assess the program's performance. For example, one set of measures tracks program outputs, such as the percentage of servicemembers who participated in required TAP components, the percentage enrolled in eBenefits,[Footnote 32] and the percentage who met career readiness standards before separating. [Footnote 33] Another set measures outcomes, such as the usage rate of the Post 9/11 Veterans Educational Assistance Act of 2008 (Post 9/11 GI Bill) among recent eligible veterans, and another measures the percentage of unemployed post-9/11 veterans, 18-24 years of age, compared to their non-veteran peers. Officials from the agencies acknowledged that these outcome measures may be affected by other variables, such as the economy, making it difficult to attribute results to the program. While the agencies have established performance measures, training programs can be assessed at differing levels, ranging from lower level basic end-of-course evaluations to higher level impact evaluations. [Footnote 34] The agencies plan to evaluate TAP at lower levels. Their evaluation plan includes (1) gauging participant reaction to all TAP components through end-of-course evaluations and (2) determining whether servicemembers met the career readiness standards prior to separation. Higher level evaluations are also important to help gauge the effectiveness of TAP, and the agencies have not demonstrated a strategic approach to planning such evaluations. Since the program is administered by multiple agencies, planning an evaluation is more challenging than if it were administered by a single agency. In December 2011, the agencies stated, in an internal report, that they would develop a methodology to assess the effectiveness of TAP by engaging new veterans approximately six months after they have separated from the military. Despite the modular nature of TAP, this report did not specify whether this type of higher level evaluation would assess all of TAP or certain components. DOL and VA are considering higher level evaluations for the employment workshop and VA benefits briefings. For example, DOL is considering different methodologies it could use to conduct an impact evaluation of the employment workshop. However, the agencies could not provide a rationale for not using higher-level evaluations to assess either TAP overall or some of the other TAP components and career-specific tracks. Best practices call for the development of a written evaluation strategy, which the agencies prepared for lower evaluation levels, but not for higher levels. Without a written evaluation strategy that identifies the TAP components to be evaluated, and includes the appropriate level and timing of the evaluation and methods, the agencies may miss important opportunities to obtain information needed to make fully informed decisions on whether and how to modify TAP and may either over invest or under invest in evaluations. DOD Lacks a Process to Assess the TAP Experience of National Guard and Reserve Members: Based on GAO's prior work and interviews with stakeholders, we identified a key remaining challenge; namely, that DOD lacks a process to assess the TAP experience of National Guard and Reserve members. DOD's policy generally requires all eligible servicemembers, including members of the National Guard and Reserve, to be exposed to the entire TAP experience even if their circumstances differ. Unlike active duty servicemembers, many members of the National Guard and Reserve hold civilian occupations,[Footnote 35] and federal law protects their employment rights when they return from active duty.[Footnote 36] If they can document their civilian employment or acceptance to an institution of higher education or accredited technical training, members of the National Guard and Reserve and other eligible servicemembers can be exempted from attending the DOL employment workshop. Nevertheless, they are generally required to complete pre- separation counseling, the VA benefits I and II briefings and, under DOD policy, participate in the capstone event. Further, the policy states that they must attend the DOD core curriculum and applicable career specific tracks if they cannot meet the career readiness standards associated with these two components. However, according to many stakeholders that we talked with, including officials from the services and organizations that support members of the National Guard and Reserve and other servicemembers, DOD has not resolved long-standing concerns that eligible members of the National Guard and Reserve attend TAP offerings in locations and on a timetable that diminish their experience.[Footnote 37] Specifically, given that eligible National Guard and Reserve members generally demobilize at locations where they neither work nor live, they may be distracted by their desire to return home, which can affect how much the training benefits them. Separating servicemembers need to be aware of the benefits and services and know how to access them to take advantage of them, such as the Post-9/11 GI Bill. In a January 2013 report, the Defense Business Board Task Group recommended that eligible National Guard and Reserve members be afforded the opportunity to demobilize and transition in their home unit's geographical area.[Footnote 38] According to stakeholders, eligible National Guard and Reserve members also typically have less time to complete the program because they often demobilize more quickly than active duty servicemembers separate.[Footnote 39] For example, according to DOD's 2011 Handbook, the Reserve demobilization timeline makes scheduling a pre-separation counseling appointment not later than 90 days prior to leaving active duty impractical for Reserve members.[Footnote 40] According to officials from DOD and the services, they took steps to help meet the needs of National Guard and Reserve members eligible for TAP. For example, the agencies tailored the content of TAP components to better suit the needs of National Guard and Reserve members after analyzing the results of feedback from pilots of TAP.[Footnote 41] However, only one of the steps taken directly addresses the concerns of stakeholders related to the location and timing of TAP. Specifically, eligible members of the Army National Guard are allowed to participate in the DOL employment workshop and the capstone event in their home unit's geographical area, according to DOD and Army officials. These eligible Army National Guard members will remain on active duty while participating in these two components. Nonetheless, DOD may not be well positioned to determine whether its actions successfully address the long-standing challenges in designing transition services for the National Guard and Reserve. This is because while DOD collects participant feedback through the online assessment, this tool does not ask whether the timing and location met the needs of servicemembers, including the National Guard and Reserve. [Footnote 42] Moreover, DOD's planned survey to active duty servicemembers contains questions about the timing of TAP, but DOD has not drafted similar questions for its survey to the National Guard and Reserve.[Footnote 43] According to OMB, being able to track and measure specific program data can help agencies diagnose problems, identify drivers of future performance, evaluate risk, support collaboration, and inform follow-up actions.[Footnote 44] Without a systematic process targeted to identify any remaining long-standing concerns, DOD will not be able to make fully informed decisions about the extent to which eligible members of the National Guard and Reserve reap the full benefits of TAP. Conclusions: Helping servicemembers successfully transition to civilian life after their service ends is an important government goal. The agencies undertook an ambitious effort to redesign the 1990s-era program to provide servicemembers with training and skills to successfully transition to civilian life. The increase in the number of servicemembers attending the revised TAP components, along with the significant coordination effort among multiple agencies and military services, pose significant implementation challenges. Efforts to implement the redesigned program continue to evolve. The importance of serving the ongoing and projected wave of servicemembers departing the military increases the urgency to fully implement TAP components as well as to finalize related policies and procedures. To the extent that the program remains behind schedule in implementing TAP, some transitioning servicemembers may be denied the full benefit of the revamped program. Because TAP is not fully implemented, the full impact of the revamped TAP, particularly the warm handover--a key effort to serve those deemed at greater risk of not being career ready--remains to be seen. The revamped TAP exhibits elements important for the effective implementation and evaluation of the program. Yet, we found room for improvement in several key areas. In particular, while federal law generally requires DOD to mandate participation in the employment workshop and DOD policy requires that all eligible transitioning servicemembers participate in TAP overall, the Navy and the Marines lack the ability to provide unit commanders and their leaders with information on participation levels of servicemembers under their command. Without the capability to gauge and report participation in TAP components by unit commander--those directly responsible for ensuring servicemember participation--leaders may find that holding them accountable is difficult. While the administration cites the revamped TAP as a key strategy of its crosscutting goal to improve the career readiness of veterans, it may not be positioned to determine the extent to which the program prepared veterans to pursue their post-separation goals. Without a written evaluation strategy that identifies the TAP components to be evaluated and includes the appropriate level and timing of the evaluation methods, the agencies may miss important opportunities to obtain information needed to make fully informed decisions on whether and how to modify TAP and may either over invest or under invest in evaluations. The circumstances of National Guard and Reserve members differ from those of other active duty servicemembers. Many stakeholders have raised concerns that these circumstances diminish eligible National Guard and Reserve members' TAP experience. Absent efforts to systematically collect information about eligible National Guard and Reserve members' experiences under the revamped program, DOD may not be well positioned to determine whether there are problems with how TAP is provided to these groups. Recommendations for Executive Action: Based on our review, we recommend that the Secretary of Defense take the following actions: 1. To better ensure servicemember participation in and completion of TAP, direct the Under Secretary for Personnel and Readiness to require that all services provide unit commanders and their leaders information on TAP participation levels of servicemembers under their command, similar to that provided by the Army and Air Force. Such information could be used to help hold leaders accountable for ensuring TAP participation and completion. 2. To provide information on the extent to which the revamped TAP is effective, direct the Under Secretary for Personnel and Readiness to work with the partner agencies to develop a written strategy for determining which components and tracks to evaluate and the most appropriate evaluation methods. This strategy should include a plan to use the results of evaluations to modify or redesign the program, as appropriate. 3. To ensure that decisions about the participation of eligible members of the National Guard and Reserves in TAP are fully informed, direct the Under Secretary for Personnel and Readiness to systematically collect information on any challenges facing demobilizing members of the National Guard and Reserves regarding the logistics of the timing and location to attend TAP. For example, agencies might add questions to their online assessment tool specific to eligible members of the National Guard and Reserves who participate in TAP. Agency Comments and Our Evaluation: DOD also stated that the Navy is funding IT system upgrades to provide the ability to analyze and report program compliance at the command level. DOD added that the Marine Corps has mandated participation since the program's inception, and Marine Corps commanders leverage the capabilities of the personnel system to identify eligible Marines and schedule their TAP attendance. DOD stated that despite the limitations of a data tracking system that underreports compliance figures, the Marine Corps had the highest compliance rate of all services, as of December 2013. DOD said it recommends that department- wide compliance data be allowed to normalize to show true compliance percentages before the services are required to fund and implement expensive systems for the sole purpose of providing an additional mechanism for commander accountability. However, we continue to believe that our recommendation is needed. Three of the services either have or are working to develop the kind of accountability mechanism that we are recommending, but the Marine Corps does not plan to develop such a system. While DOD reports that the Marine Corps has the highest TAP compliance rate of all services, DOD did not provide any data on these rates. Finally, in response to DOD's recommendation to wait until compliance data normalize, we believe that this would not be appropriate. We do not see any advantage to delaying the implementation of appropriate accountability mechanisms to provide assurance that the ongoing and projected wave of servicemembers departing the military receive the expected level of services from TAP. DOD disagreed with our recommendation to systematically collect information on challenges facing demobilizing members of the National Guard and Reserve regarding the timing and location to attend TAP. Specifically, DOD stated that it has long understood that the National Guard and Reserves operate under schedules and logistical constraints that differ from those of active duty servicemembers. DOD stated that several processes (which we note in our report) are already in place to identity and rectify any misalignments between TAP services and the needs of eligible National Guard and Reserve members, including regular coordinating councils that include representation from the National Guard and Reserve and TAP managers, as well as a participant assessment that provides ample opportunity for eligible National Guard and Reserve members to voice concerns. In addition, DOD highlighted the implementation of the military life cycle transition model in which DOD plans to provide the services, including to the National Guard and Reserve, with the latitude to optimize placement of certain elements of TAP throughout a servicemember's military service. According to DOD, the military life cycle transition model may help address some of the challenges related to the timing and location of program delivery. As we note in the report, full implementation of the military life cycle transition model is planned by October 2014. Nonetheless, at this time DOD is not fully positioned to know whether or not the revamped program addresses the long-standing challenges facing eligible members of the National Guard and Reserve in taking TAP. As we describe in the report, all eligible servicemembers, including the National Guard and Reserve members, have an opportunity to provide feedback about the instruction, content, and facilities for TAP. However, the participant assessment does not ask questions specific to the National Guard and Reserve experience even though they face different circumstances than active duty servicemembers. Given these differences, we continue to believe that unless DOD systematically collects information on any challenges facing eligible members of the National Guard and Reserve, DOD is at risk of not fully knowing whether the revamped TAP addresses long-standing challenges. In addition, the move to a military life cycle transition model could enable active duty servicemembers and members of the National Guard and Reserve to benefit from transition-related assistance throughout their military service. Recognizing that the military life cycle transition model is not implemented and specific policies and plans are not completed, many unknowns remain about how it will work in general and how National Guard and Reserve members will fare specifically. During our review, we asked DOD how this program would work in practice, including for National Guard and Reserve members, but DOD did not provide us details. We are sending copies of this report to appropriate congressional committees; the Secretary of Defense; the Secretary of Veterans Affairs, the Secretary of Labor; the Acting Administrator of the Small Business Administration; and other interested parties. In addition, this report will be available at no charge on GAO's website at [hyperlink, http://www.gao.gov]. If you or your staff members have any questions regarding this report, please contact me at (202) 512-7215 or sherrilla@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V. Signed by: Andrew Sherrill: Director, Education, Workforce and Income Security Issues: List of Committees: The Honorable Carl Levin: Chairman: The Honorable James M. Inhofe: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Bernie Sanders: Chairman: The Honorable Richard Burr: Ranking Member: Committee on Veterans' Affairs: United States Senate: The Honorable Howard P. "Buck" McKeon: Chairman: The Honorable Adam Smith: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable Jeff Miller: Chairman: The Honorable Michael H. Michaud: Ranking Member: Committee on Veterans' Affairs: House of Representatives: [End of section] Appendix I: Technical Appendix: This appendix summarizes our work to assess efforts to implement and evaluate the revamped TAP. Elements and Relevant Attributes and the Extent to Which They Have Been Addressed: For each of the five elements that we identified as important for the effective implementation and evaluation of TAP, we identified relevant attributes that we used to determine the extent to which the elements were addressed in the revamped TAP (see table 2).[Footnote 45] Table 2: Elements and Relevant Attributes and Extent They Have Been Addressed: Adequately Addressed[A]: Element: 1. Track Attendance; Underway; Relevant attributes: The agency uses systems to manage and track whether eligible servicemembers receive training. Element: 2. Processes for Ensuring Quality of Instruction, Content, and Facilities; Underway; Relevant attributes: 2a. The agency takes actions to foster an environment conducive to effective training (approaches may include using facilities and equipment that enable effective implementation of training and ensuring that facilitators are engaging, knowledgeable, and experienced). Underway; Relevant attributes: 2b. The agency measures participants' reactions to and satisfaction with the training they receive. Underway; Relevant attributes: 2c. The agency has a clear methodology, timetable, and delegation of responsibility for the collection of data that will be used to assess training and development programs. Underway; Relevant attributes: 2d. The agency incorporates monitoring and evaluation feedback into its training efforts. Element: 3. Mechanisms to Assess Career Readiness; Underway; Relevant attributes: 3a. The agency measures the extent to which learning has occurred (approaches may include having an overall approach for measuring; taking steps to analyze such data; issuing reports detailing results of analysis; and issuing guidance on how evaluation results will be used to inform, modify, and improve planning). Underway; Relevant attributes: 3b. The agency ensures that data are collected systematically and in a timely manner. Underway; Relevant attributes: 3c. The agency provides evidence that it requires commanders or their designees to be properly trained to coach, evaluate, and conduct discussions with employees regarding their career readiness. Element: Mixed[B]: Element: 4. Mechanisms to Ensure Participation and Completion; Underway; Relevant attributes: 4a. Agency leaders communicate the importance of training (approaches may include issuing memoranda or announcements and mentioning the training on web sites). Underway; Relevant attributes: 4b. An organization or entity is held accountable, along with the line executives, for ensuring that participants successfully complete training (approaches include reinforcing separation of work time from training time). Not completed; Relevant attributes: 4c.The agency adjusts servicemembers' work schedules so that they can participate in training (approaches include agencies tracking managerial support for training through participation rates--which assumes enrollment--in their units). Underway; Relevant attributes: 4e. The agency prioritizes training for servicemembers based on agreed-upon goals and priorities. Underway; Relevant attributes: 4f. The agency encourages servicemembers to buy in to the goals of training so that they participate fully and apply new knowledge and skills (approaches may include using individual development plans, seeking employees' ideas to improve training, or issuing guidance on managing their career paths). Element: 5. Measures of Performance and Processes to Systematically Evaluate the Effectiveness or Benefits; Underway; Relevant attributes: 5a. The agency has performance measures to assess the results achieved through TAP. Not completed; Relevant attributes: 5b. The agency has a clear methodology, timetable, and delegation of responsibility for evaluating the benefits or impact of TAP. Underway; Relevant attributes: 5c. The agency analyzes and considers different stakeholder perspectives (including those of line managers and staff, customers, and experts in areas such as business and human capital management) in assessing the benefits or impact of training. Underway; Relevant attributes: 5d. The agency plans to or incorporates results of evaluations into its training. Source: GAO analysis of data provided by DOD, the four services, DOL, SBA and VA. [A] We considered the agency to be "adequately addressing" the element if the overall status of the efforts to address the element was determined to be completed or underway. [B] We considered the overall efforts to address the element to be "mixed" if some attributes related to an element had efforts completed or underway to address them, but one or more others did not. [End of table] To determine the extent to which agencies have addressed each element and attribute, we made our assessments in two steps. First, an analyst reviewed all of the evidence. Based on that review, the analyst documented evidence that conformed to the elements and attributes and made and documented a judgment about the status of these efforts. Then a second analyst separately reviewed the assessment and documented their agreement or disagreement. The two analysts discussed any differences in their assessments and made changes based on their verbal resolution of those differences. We assessed the status of the agencies' efforts to address each relevant attribute as "completed", "underway", or "not completed". Based on this analysis, we then determined the status of efforts to address each respective attribute overall. For example, for the status of efforts to address an attribute overall to be considered "completed", efforts to address each relevant attribute had to be completed. For the status of efforts to be considered "underway", it had to be documented that efforts to address each relevant attribute were progressing toward being finalized, or were a mix of underway efforts and finalized efforts. For the status of efforts to be considered "not completed", efforts to address at least one relevant attribute were not determined to be finalized or underway. We considered the agency to be "adequately addressing" the element if the overall status of the efforts to address the relevant attributes was determined to be completed or underway. However, if some attributes related to an element had efforts completed or underway to address them, but one or more other attributes did not have efforts completed or underway to address it (them), then we consider the overall efforts to address the element to be "mixed". We used the following general decision rules for characterizing the status of efforts to address each attribute as "completed", "underway", and "not completed": * "Completed" means interagency partners (or relevant agency) have a system, policy, or procedure in place to address a relevant attribute. * "Underway" means interagency partners (or relevant agency) have an authoritative internal or public document that demonstrate progress toward implementing actions to address the attribute (e.g., agencies have draft policies, operating procedures, or guidance, or they have interim systems in place and/or are developing systems). * "Not completed" means that interagency partners (or relevant agency) have shown little to no action toward implementing actions to address the attribute, meaning they lack an authoritative internal or public document (e.g., no draft policies, operating procedures, or guidance or interim systems). As we conducted this analysis, we kept in mind how far along agencies (mainly DOD and the military services) are in implementing each component of TAP (informed by objective 1). Because agencies were in the process of implementing the revamped TAP and the program was not fully operational at the time of our review, we were not able to determine whether the policies and procedures are in place at all sites or if they are working as intended. Similarly, we were not able to comment on whether the changes to TAP are yielding desired benefits or improvements in outcomes. [End of section] Appendix II: DOD Form 2958, Individual Transition Plan (ITP) Checklist: [Refer to PDF for image: 2 page checklist] [End of section] Appendix III: Comments from the Department of Defense: Office Of The Assistant Secretary Of Defense: Readiness And Force Management 4000 Defense Pentagon: Washington, D.C. 20301-4000: Mr. Andrew Sherill: Director, Education, Workforce, and Income Security Issues: U.S. Government Accountability Office: 441 G Street, NW: Washington DC 20548: Dear Mr. Sherill: This is the Department of Defense (DoD) response to the GAO Draft Report GAO-14-144, "Transitioning Veterans: Improved Oversight Needed to Enhance Implementation of Transition Assistance Program," dated January 15,2014 (GAO Code 131231). The Department is providing official written comments for inclusion in the report. Sincerely, Signed by: Dr. Susan Kelly: Director, Transition to Veterans Program Office: Attachment: Gao Draft Report Dated January 15, 2014: GAO-14-144 (GAO Code 131231): “Transitioning Veterans: Improved Oversight Needed to Enhance Implementation of Transition Assistance Program” Department Of Defense Comments To The GAO Recommendation: Recommendation 1: The GAO recommends that the Secretary of Defense direct the Under Secretary for Personnel and Readiness to require that all services provide unit commanders and their leaders information on TAP participation levels of service members under their command, similar to that provided by the Army and Air Force. Such information could be used to help hold leaders accountable for ensuring TAP participation and completion. DoD Response: Nonconcur. GAO justifies the need for a system providing Commander accountability based on concerns that without one, Commanders will not support the release of service members to attend TAP. These concerns are based on observations that preceded full implementation of Capstone, which launched in October 2013 as a mandatory process by which Commanders verify TAP participation and positively affirm that the Service members under their commands have or have not met Career Readiness Standards. Capstone implementation was accompanied by a robust communications campaign to inform both Commanders and Service TAP providers of this key requirement. Capstone completion is monitored Department-wide to ensure compliance. Furthermore, the Navy is already funding IT system upgrades that will provide the ability to analyze and report program compliance down to the Unit Identification Code level. The Marine Corps has mandated participation since the program's inception, and Marine Corps Commanders leverage the capabilities of the personnel system to identify eligible Marines and schedule their TAP attendance. It should also be noted that as of December 2013, despite the limitations of a data tracking system that under-reports compliance figures, the Marine Corps has the highest compliance rate of all services. The Department recommends that Department-wide compliance data be allowed to normalize to show true compliance percentages before imposing further requirements to fund and implement expensive systems for the sole purpose of providing an additional mechanism for Commander accountability. Recommendation 2: The GAO recommends that the Secretary of Defense direct the Under Secretary for Personnel and Readiness to work with the partner agencies to develop a written strategy for determining which components and tracks to evaluate the most appropriate evaluation methods. This strategy should include a plan to use the results of evaluations to modify or redesign the program, as appropriate. DoD Response: Concur. The Department supports, and will continue to support interagency collaborative efforts to systematically evaluate and improve TAP and the Transition GPS curriculum. This collaboration is ongoing and has been formalized as of a January 31, 2014 Memorandum of Understanding between the Department of Defense, the Department of Veterans Affairs, the Department of Labor, the Department of Education, the Department of Homeland Security, the Small Business Administration, and the Office of Personnel Management, which includes a joint responsibility of the signed parties to “develop, assess, evaluate, and improve the Transition GPS.” The Department concurs with the recommendation to work with partner agencies toward the development of a written strategy to evaluate the effectiveness of Transition GPS components and tracks. Recommendation 3: The GAO recommends that the Secretary of Defense direct the Under Secretary for Personnel and Readiness to systematically collect information on any challenges facing demobilization members of the National Guard and Reserves regarding the logistics of the timing and location to attend TAP. For example agencies might add questions to their online assessment tool specific to eligible members of the National Guard and Reserves who participate in TAP. DoD Response: Nonconcur. The Department has long understood that the National Guard and Reserves operate under schedules and logistical constraints that differ from those of the Active components. However, law and policy mandate that all eligible Service members, absent an exception, including those affiliated with the National Guard and Reserves, receive TAP services prior to separation from active duty. Furthermore, several processes are already in place to identify and rectify misalignments between TAP services and Reserve component needs, including regular coordinating councils which include representation from the Reserve components and TAP managers, a dedicated “Reserve Component Tiger Team” focusing on TAP participation among National Guard and Reserve Service members, and a Transition GPS participant assessment which already provides ample opportunity for Reserve component Service members to voice concerns regarding their ability to participate in transition assistance. Most importantly, with the implementation of the Military Life Cycle model of TAP in fiscal year 2014, Reserve components have been provided with the latitude to optimize placement of certain elements of TAP throughout a Service member's military service, which will enable them to tailor transition assistance to their Service member schedules and needs. [End of section] Appendix IV: Comments from the Department of Veterans Affairs: Department Of Veterans Affairs: Washington DC 20420: February 12, 2014: Mr. Andrew Sherrill: Director: Education, Workforce, and Income Security Issues: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Sherrill: The Department of Veterans Affairs (VA) has reviewed the Government Accountability Office's (GAO) draft report, "Transitioning Veterans: Improved Oversight Needed to Enhance Implementation of Transition Assistance Program" (GAO-14-144). VA generally agrees with GAO's findings. The 'enclosure contains technical comments related to the draft report. VA appreciates the opportunity to comment on your draft report. Sincerely, Signed by: Jose D. Riojas: Chief of Staff: [End of section] Appendix V: GAO Contact and Staff Acknowledgments: GAO Contact: Andrew Sherrill, (202) 512-7215 or sherrilla@gao.gov. Staff Acknowledgments: In addition to the contact named above, individuals making key contributions to this report were Patrick Dibattista, Assistant Director; Julianne Hartman Cutts; and James Whitcomb. In addition, key support was provided by James Bennett, Rachael Chamberlin, David Chrisinger, Brett Fallavollita, Alex Galuten, Kathy Leslie, David Moser, and Michael Silver. Training programs can be evaluated by measurements at varying levels, from lower level evaluations which include (1) participant reaction and (2) changes in their skills and knowledge; to higher level ones which include (3) changes in their behavior or performance and (4) the impact of the training. See GAO-04-546G. [End of section] Footnotes: [1] Over each of the next 4 years, the Department of Defense estimates that approximately 170,000 to 185,000 active duty servicemembers will separate from the military and about 60,000 National Guard and Reserve members will be demobilized and deactivated from active duty. [2] Pub. L. No. 112-56, Title II, 125 Stat. 711, 712. After the VOW Act became law, the unemployment rate for post-9/11 veterans improved-- decreasing from 13.1 percent in December 2011 to 7.3 percent in December 2013--but remained above the overall unemployment rate for non-veterans (6.4 percent), according to the Bureau of Labor Statistics. According to the Bureau of Labor Statistics, veterans between the ages of 18 to 24 maintain the highest rate of unemployment. In 2012, the unemployment rate for male post-9/11 veterans age 18 to 24 was 20.0 percent, higher than the rate for nonveterans of the same age group (16.4 percent). [3] The VEI Task Force was created in August 2011 to build on the existing TAP program and also incorporated the requirements of the VOW Act. The VEI Task Force renamed the revamped curriculum of TAP, Transition Goals, Plans, Success (Transition GPS). [4] Pub. L. No. 112-56, § 226, 125 Stat. 711, 719. On September 20 and 24 and November 12 and 20, 2013, we briefed the House and Senate Committees on Armed Services and the House and Senate Committees on Veterans' Affairs on our preliminary observations. This report formally conveys the information provided during those briefings. [5] See GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO-04-546G] (Washington, D.C.: March 2004). The guide summarizes elements of effective training programs and presents related questions on the components of the training and development process in four broad, interrelated components: (1) planning/front-end analysis, (2) design/ development, (3) implementation, and (4) evaluation. We mainly used the guide's implementation and evaluation components to identify the detailed strategic training and development criteria applicable to TAP. These criteria remain relevant today because they are the most recent relevant guidance available to assess how agencies plan, design, implement, and evaluate effective federal training and development programs. [6] For example, we reviewed a January 2013 Defense Business Board report that identified some of the remaining challenges associated with implementing and evaluating TAP. [7] To select the sites, we worked with DOD officials to identify installations that had implemented selected revamped TAP training sessions, including the DOL employment workshop, the VA benefits training sessions, and the DOD core curriculum. We provided DOD and officials at each installation visited with criteria that the installation officials used to select servicemembers for our discussion groups. [8] For example, because DOD was in the process of, or planning to, monitor the program content and delivery and collect data on attendance and outcomes, we did not test the application of DOD's policies and procedures related to these areas. In addition, we were not able to review DOD's plans to implement its military life cycle transition model because it is in the early planning stage and is not due to be operational until October 2014. [9] TAP was established through the National Defense Authorization Act for Fiscal Year 1991, Pub. L. No. 101-510, § 502(a)(1), 104 Stat. 1485, 1551 (1990). Federal law spells out many topics that must be covered in TAP, including employment and relocation assistance, education opportunities, health and life insurance, and financial planning. Moreover, eligibility and the time frames for provision of TAP services are spelled out in law: all service members who have been on active duty for at least 180 days are eligible for TAP services, but those separating because of a disability are eligible regardless of the length of their active duty service. Eligible service members must be provided TAP while they are on active duty, either as soon as possible within the 2 years prior to their anticipated retirement date or in the 1 year prior to their anticipated separation date. In either case, TAP services must generally commence no later than 90 days prior to their discharge or release. The exceptions to this rule occur when retirements or separations are not anticipated until 90 or fewer days of active duty remain, or a member of the reserve is being demobilized under circumstances in which the 90 day requirement is unfeasible. In such cases, TAP services must be provided as soon as possible within the remaining period of service. [10] At the completion of the employment workshop, servicemembers are given a "Gold Card" that entitles them to enhanced intensive services for six months by an employment specialist through DOL's American Job Centers. [11] In addition to carrying out these agreements at the national level, based on the January 31, 2014 MOU, the federal agencies encourage similar agreements among state and regional offices and the installations located in their areas. [12] For more information see GAO, Veterans' Employment and Training: Better Targeting, Coordinating, and Reporting Needed to Enhance Program Effectiveness, [hyperlink, http://www.gao.gov/products/GAO-13-29] (Washington, D.C.: December 13, 2012). [13] While some servicemembers may be exempted from attending the DOL employment workshop--for example, DOD policy exempts servicemembers retiring after 20 or more years of service in the military--DOD is generally required to provide pre-separation counseling to servicemembers. Further, DOD requires servicemembers to attend the other components of TAP. However, some servicemembers will not be able to complete all the required TAP training for various reasons, according to DOD officials. This issue is discussed later in the report. [14] The VOW Act requires DOL to utilize one or more contractors to facilitate the employment workshop by November 21, 2013. [15] DOD is updating a DOD Instruction (DODI) for TAP to reflect changes to the program. The DODI is in draft form and subject to revision. The draft DODI will be finalized after going through the federal rulemaking process, which generally requires agencies to publish a notice of proposed rulemaking in the Federal Register. [16] According to DOD's draft TAP Department of Defense Instruction (DODI), DOD's intent is for the services to identify key touch points of the military life cycle, such as the following: (1) first permanent duty station/home station; (2) reenlistment; (3) promotion; (4) deployment or redeployment; (5) mobilization/activation, demobilization/deactivation; (6) major life events (e.g., change in family status, etc.); and (7) retirement/separation. [17] [hyperlink, http://www.gao.gov/products/GAO-04-546G]. [18] DOD and the services began this tracking at the beginning of fiscal year 2013 in anticipation of meeting the November 2012 VOW Act requirement. Additionally, all other modules of TAP can be captured through this system, including the career-specific tracks and the capstone event. [19] DOD measures progress toward this goal by comparing data on attendance at three required components--pre-separation counseling, the DOL employment workshop, and the VA benefits I and II briefings-- to information on servicemembers discharged over a given time period as well as other data DOD collects about individual servicemembers. [20] The revamped TAP is a key strategy of the administration's Cross- Agency Priority Goal to improve the career readiness of veterans. The administration's goal is to increase the rate of eligible servicemembers served by "career readiness and preparedness programs" to 90 percent by September 30, 2013. The GPRA Modernization Act of 2010 requires that OMB develop federal government performance goals (crosscutting goals) and a federal government performance plan, which is to be updated annually and released concurrently with the President's budget. [21] [hyperlink, http://www.gao.gov/products/GAO-04-546G]. [22] Servicemembers are provided with the link to a web-based participant assessment and encouraged to complete it at the end of each TAP component they attend. [23] [hyperlink, http://www.gao.gov/products/GAO-04-546G]. [24] DOD is also taking steps to ensure that commanders or their designees are properly trained to coach, evaluate, and conduct employee career discussions. For example, DOD developed an individual transition plan checklist for commanders to use. Also, the Army includes TAP in its leadership training at every installation and is developing a commander and career counselor guide that outlines the responsibilities, measures of success, and tools for implementing TAP at the unit level. Furthermore, DOD plans to establish quality standards for products required by the career readiness standards, such as job applications, to be used by commanders to objectively validate and evaluate the extent to which servicemembers met these standards. Similarly, according to the December 2012 DOD-VA VEI Task Force Implementation Plan, the services are developing scripts to structure sessions that the commanders or their designees hold with eligible servicemembers to discuss their transition plans. [25] SBA staff are not required to participate in the capstone to support the warm handover for any of the services and, instead, plan to provide SBA contact and other information, such as website resources, during the entrepreneurial track. [26] DOD guidance requires the services to schedule servicemembers to participate in TAP components at appropriate points during their careers. According to officials from the four services, it can be challenging to ensure attendance for servicemembers separating under short notice, such as those separating for disciplinary reasons. In such cases, officials said that the services have offered alternatives to the regular TAP training if time or circumstances did not permit a servicemember to take a TAP class. [27] Commanders are assigned responsibility through a DOD Directive- type Memorandum, which is an interim policy that was valid though May 2013, but, according to DOD officials, has been extended through May 2014. The interim policy will be incorporated into a TAP DOD Instruction (DODI) that DOD is drafting. DOD officials stated that the draft DODI will be finalized after going through the federal rulemaking process, which generally requires agencies to publish a notice of proposed rulemaking in the Federal Register. [28] This includes, for example, the number and percentage of eligible soldiers who completed the DOL employment workshop not later than 9 months prior to separation. [29] It is DOD's intent that the IG of each service monitors TAP implementation and compliance, which may include monitoring this issue at the commander level. However, such monitoring is not designed to provide regular and timely information that could be used to manage the rates at which TAP participants successfully complete training. [30] GAO, Military and Veterans' Benefits: Enhanced Services Could Improve Transition Assistance for Reserves and National Guard, [hyperlink, http://www.gao.gov/products/GAO-05-544] (Washington, D.C.: May 20, 2005); Military and Veterans' Benefits: Observations on the Transition Assistance Program, [hyperlink, http://www.gao.gov/products/GAO-02-914T] (Washington, D.C.: July 18, 2002). [31] Outputs are the direct products and services delivered by a program, and outcomes are the results of those products and services. See GAO, Performance Measurement and Evaluation: Definitions and Relationships, [hyperlink, http://www.gao.gov/products/GAO-11-646SP] (Washington, D.C.: May 2011). [32] eBenefits is a central portal or location for servicemembers, veterans, and their families to research, find, and access their benefits and personal information. [33] As of February 2014 the performance goal was reduced from at least 85 percent of servicemembers meeting career readiness standards prior to their separation to 65 percent. [34] Training programs can be evaluated by measurements at varying levels, from lower level evaluations which include (1) participant reaction and (2) changes in their skills and knowledge; to higher level ones which include (3) changes in their behavior or performance and (4) the impact of the training. See [hyperlink, http://www.gao.gov/products/GAO-04-546G]. [35] According to the Bureau of Labor Statistics, post 9/11 veterans who were current or past members of the Reserve or National Guard had an unemployment rate of 7.2 percent in August 2012, lower than the rate for those veterans who had not been members of the Reserve or National Guard (13.7 percent). Members of the National Guard and Reserve train for their military mission during certain weeks and weekends each year. [36] Among other rights, servicemembers who meet the statutory requirements are generally entitled to reinstatement to the positions they would have held if they had never left their employment or to positions of similar seniority, status, and pay. 38 U.S.C. § 4311 et seq. [37] In 2005, we reported concerns about TAP timing and location for this population. At that time most members of the National Guard and Reserves did not have the opportunity to attend an employment workshop during demobilization. See [hyperlink, http://www.gao.gov/products/GAO-05-544]. [38] Defense Business Board, Employing Our Veterans Part II: Service Member Transition, a report prepared at the request of the Department of Defense, Jan. 24, 2013. [39] Members of the National Guard and Reserve being released from active duty have a short time period when they come back from deployment, while active duty servicemembers generally have at least 90 days--and up to 24 months--to participate in TAP before they separate. Typically, the demobilization process is rapid, taking a few weeks once members of the National Guard and Reserve arrive in the United States from overseas. During demobilization, they are expected to participate in numerous briefings on various topics, such as legal and medical issues, and scheduled activities, such as physical examinations. [40] DOD, Compensation and Benefits Handbook for Wounded, Ill and Injured Members of the Armed Forces, 00054-624 (Oct. 2011). As noted previously, eligible servicemembers must generally be provided TAP services as soon as possible within the 2 years prior to their anticipated retirement or in the 1 year prior to their anticipated separation date. In either case, services must be provided no later than 90 days prior to their discharge or release. The exceptions to this rule occur when retirements or separations are not anticipated until 90 or fewer days of active duty remain or a member of the reserve is being demobilized under circumstances in which the 90 day requirement is unfeasible. In such cases, TAP services must be provided as soon as possible within the remaining period of service. [41] In addition, DOD and the services convened a task team and as a result, decided to include representatives from the National Guard and Reserve as nonvoting members of the internal DOD TAP Coordinating Council. [42] In addition, although the participant assessment collects immediate participant reaction to the training, it may not capture concerns that National Guard and Reservists may face after leaving TAP. For example, officials from two veteran service organizations expressed concern that TAP may not focus enough on how National Guard and Reserve members should address potential employer discrimination. [43] The Defense Manpower Data Center conducts a series of Web-based surveys called Status of Forces surveys, which help decision makers within DOD to (1) evaluate existing programs and policies, (2) establish baselines before implementing new programs and policies, and (3) monitor the progress of programs and policies and their effects on the total force. The Status of Forces surveys include a survey of active duty military personnel, called the Status of Forces Active; a survey of national guard and reserve military personnel, called the Status of Forces Reserve Survey; and a survey of civilian employees, called the Status of Forces Survey of Civilian Employees. [44] Furthermore, according to government standards for internal control, information should be available to help inform management in making decisions and identifying risks and problem areas. In addition, promising practices of program administration that we have identified include a strong capacity to collect and analyze accurate, useful, and timely data. (Harold I. Steinberg, Using Performance Information to Drive Performance Improvement, Association of Government Accountants CPAG Research Series: Report No. 29 (Alexandria, VA: Dec. 2011)). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]