Improved Oversight Needed to Enhance Implementation of Transition Assistance Program
GAO-14-144: Published: Mar 5, 2014. Publicly Released: Mar 5, 2014.
What GAO Found
The Departments of Defense (DOD), Labor (DOL), and Veterans Affairs (VA) have implemented most of the key components of the Transition Assistance Program (TAP), a gateway to information and services available to servicemembers transitioning to civilian life. However, the agencies are still in the process of implementing other key components of TAP. While originally planned for October 2013, agencies now plan to implement virtually all components by the end of March 2014, with full implementation expected by June 2014.
Agencies' efforts are underway to adequately address three of five elements that GAO identified as important for effective implementation and evaluation of TAP:
1-Track attendance : DOD has systems to collect and report on attendance, which help measure the extent to which TAP achieves its attendance goals.
2-Ensure training quality : The agencies collect and plan to use participant feedback on instruction, content, and facilities to improve training. Each agency also plans to monitor its respective TAP components through site visits.
3-Assess career readiness : The agencies developed standards to assess servicemembers' career readiness. During a capstone assessment, commanders are expected to verify and document whether standards were met.
Agencies' efforts to address the remaining two elements are mixed:
4-Ensure participation and completion : DOD has assigned commanders the responsibility for overseeing participation and has required the services to schedule training and communicate its importance to servicemembers. While the Army and Air Force gauge participation at the command level, the Navy and Marines lack a similar oversight mechanism.
5-Measure performance and evaluate results : The agencies have established certain measures to assess program performance, but their TAP evaluation approach is incomplete. For example, the agencies have established measures to track program outputs, such as the percentage of servicemembers who have participated in TAP. However, the agencies' efforts to evaluate TAP results have focused on basic end-of-course evaluations and gauging servicemembers' readiness prior to separation instead of higher impact program evaluations, such as assessing the effectiveness of TAP on servicemembers 6 months after they have separated from the military. According to agency officials, such evaluations are being considered for certain components of TAP, but they could not provide GAO with a justification for including or excluding specific components of TAP in their evaluation planning efforts.
Based on GAO's prior work and according to officials from the agencies and organizations GAO spoke with, a key remaining challenge for TAP may be the unfavorable timing and location of program delivery for National Guard and Reserve members. Unlike active duty servicemembers, National Guard and Reserve members receive TAP services closer to their transition and in locations that are generally neither where they work nor live. As a result, they may be distracted and have less time to benefit from TAP services. DOD is not well positioned to verify these concerns because it is not collecting data about these members' experiences with the timing and location of TAP service delivery.
Why GAO Did This Study
Over the next few years, over a million military servicemembers are expected to transition to civilian life and some may face challenges such as finding employment. To help them, TAP provides departing servicemembers employment assistance and information on VA benefits, among other things. Begun in 2011, efforts to revamp TAP are underway based on the VOW to Hire Heroes Act of 2011 and the administration's recommendations. The act also mandated GAO to review TAP. This report addresses: 1) the status of TAP implementation; 2) the extent to which elements of effective implementation and evaluation of TAP have been addressed; and 3) any challenges that may remain. To do this GAO identified five elements of effective implementation and evaluation based on relevant federal laws and previously established GAO criteria for training programs; reviewed related GAO work; assessed reports, plans, and policies provided by agencies that administer TAP; interviewed officials from entities that support servicemembers and veterans; and conducted four nongeneralizable discussion groups with servicemembers who had taken TAP at three military installations.
What GAO Recommends
GAO recommends that DOD improve oversight and implementation of TAP, including actions to gauge participation for all of the services and collect data about National Guard and Reserve members' experiences. DOD disagreed with two of GAO's three recommendations. GAO continues to believe that the recommendations are needed as discussed in the report.
For more information, contact Andrew Sherrill at (202) 512-7215 or firstname.lastname@example.org.
Recommendations for Executive Action
Comments: Although DOD disagreed with this recommendation, it has taken steps to make TAP participation information available at the installation and unit-command level. Specifically, in November 2016, DOD launched a new system to electronically collect TAP-related data: the TAP-IT Enterprise System. DOD officials said this system will allow the services to generate installation and unit-level data reports once fully operational. This system is currently being used at all installations to collect data on VOW compliance and Career Readiness Standards attainment. However, installation and unit-command specific reports describing TAP performance levels may not be available until October 2018, according to a senior DOD official. We continue to believe that providing unit-level data to commanders is needed because these commanders are responsible for ensuring that eligible servicemembers have full access to and successfully complete required TAP components. Consequently, providing commanders and their leaders information on TAP participation levels of servicemembers under their command could promote accountability and oversight. Servicemember participation in TAP is generally required by law and DOD policy, and also relates to a Cross-Agency Priority Goal, reinforcing the need for such a mechanism. For GAO to close this recommendation, DOD should require all services to provide unit commanders and their leaders information on TAP participation levels of servicemembers under their command.
Recommendation: To better ensure servicemember participation in and completion of TAP, the Secretary of Defense should direct the Under Secretary for Personnel and Readiness to require that all services provide unit commanders and their leaders information on TAP participation levels of servicemembers under their command, similar to that provided by the Army and Air Force. Such information could be used to help hold leaders accountable for ensuring TAP participation and completion.
Agency Affected: Department of Defense
Comments: DOD has partially addressed GAO's recommendation on evaluating the effectiveness of the revamped TAP, information that is currently limited. DOD's 2016 presentation to OMB on the status of TAP implementation and evaluation efforts lists specific TAP components and tracks in which VA, DOL, and SBA planned to assess TAP effectiveness or post-transition outcomes. In June 2016, the Small Business Administration released results of an evaluation on the Boots to Business program (White Paper: Operation Boots to Business Veteran Entrepreneurship Assessment). According to DOL officials, the agency has an ongoing quasi-experimental TAP evaluation of the employment workshop. Further, VA is in the process of contracting for a survey of veterans that will include questions regarding the transition services received under TAP, according to agency officials. However, DOD and the agencies have not fully demonstrated a strategic approach to planning higher level evaluations. GAO will consider the recommendation met once the TAP evaluation plan articulates a strategy to rigorously evaluate TAP's impact on post-program outcomes and specifies how the interagency partners plan to use the results of any evaluations.
Recommendation: To provide information on the extent to which the revamped TAP is effective, the Secretary of Defense should direct the Under Secretary for Personnel and Readiness to work with the partner agencies to develop a written strategy for determining which components and tracks to evaluate and the most appropriate evaluation methods. This strategy should include a plan to use the results of evaluations to modify or redesign the program, as appropriate.
Agency Affected: Department of Defense
Status: Closed - Implemented
Comments: DOD has taken three steps to systematically collect information to assess the program delivery experience of demobilizing Guard and Reserve members, according to DOD officials. One step includes adding questions to the "Fiscal Year 2016 Status of Forces Survey - Reserves" regarding any challenges with the timing and location for attending TAP. Two others steps include analyzing information from the TAP online assessment tool to identify trends in Guard and Reserve members' responses and surveying representatives from the Guard and Reserve on a TAP task team.
Recommendation: To ensure that decisions about the participation of eligible members of the National Guard and Reserves in TAP are fully informed, the Secretary of Defense should direct the Under Secretary for Personnel and Readiness to systematically collect information on any challenges facing demobilizing members of the National Guard and Reserves regarding the logistics of the timing and location to attend TAP. For example, agencies might add questions to their online assessment tool specific to eligible members of the National Guard and Reserves who participate in TAP.
Agency Affected: Department of Defense