Skip to main content

Veterans Employment: Promising VA Technology Education Pilot Would Benefit from Better Outcome Measures and Plans for Improvement

GAO-23-105343 Published: Oct 27, 2022. Publicly Released: Oct 27, 2022.
Jump To:

Fast Facts

More than 6,700 veterans enrolled in a Veterans Affairs' technology education program from May 2019-May 2022. About 66% of the enrollees completed training.

But VA doesn't have a full picture of how well this pilot program is working or ways to improve it. For example, VA doesn't know why some veterans (13%) dropped out of the program. And VA can't assess how effective the program has been with helping veterans get jobs because it doesn't collect the relevant data.

VA has found some ways to improve the program, but doesn't have a way to ensure that changes are made before the pilot ends in 2 years. Our recommendations address these issues.

A person in military fatigues typing on a laptop surrounded by IT equipment.

Skip to Highlights

Highlights

What GAO Found

Over 6,700 veterans enrolled in the Department of Veterans Affairs' (VA) Veterans Employment Through Technology Education Courses (VET TEC) pilot program from May 2019 through May 2, 2022. They were generally more racially and ethnically diverse and more likely to have a service-connected condition, compared to working-age veterans in the U.S. population. Most veterans (66 percent) who enrolled in the program completed their training (see figure).

Fig High v01_105343

VA calculates an employment measure for certain VET TEC participants for whom VA has made a final milestone payment decision. However, VA does not calculate an employment rate for all VET TEC participants who completed the program, in accordance with other government and industry approaches. As a result, VA lacks sufficient information to compare VET TEC to other programs or to assess the effectiveness of the program at getting veterans into jobs. In addition, VA does not use all available data or collect additional data on employment outcomes. For example, VA does not use information it collects on the type of employment program participants obtain (e.g., full-time, part-time, self-employed, etc.). In addition, VA does not systematically collect data on whether veterans retain employment. Without fully using available data or collecting additional relevant data, VA does not have a full picture of VET TEC employment outcomes.

Training providers told GAO that key benefits of VET TEC include an accelerated path to employment and virtual training. Three of five training providers said VET TEC allows veterans to enter the workforce quickly. Veterans completed training in 3 months, on average, according to GAO analysis of VA data. Training providers also cited challenges with some aspects of VET TEC. For example, three training providers said they faced delays in getting their applications approved, and four training providers said some application instructions were unclear. VA has taken some steps to address these challenges, such as revising certain aspects of its training provider approval processes.

VA has identified several actions for enhancing VET TEC. Specifically, VA has identified specific areas of improvement for VET TEC, with over 80 related tasks. For example, VA is planning to develop a scorecard to assess training provider quality and update its employment certification form. However, as of May 2022, VA had not developed consistent, clear, and measurable program objectives—a leading practice of effective pilot program design. Over the course of GAO's review, VA officials provided various versions of program objectives for VET TEC that were not consistent. These objectives were also not measurable because they did not include an indicator of how VA will measure progress. As a result, assessing and evaluating VET TEC by the end of the pilot will likely be difficult for VA.

Why GAO Did This Study

The federal government helps veterans pursue skills needed in the civilian workforce. Specifically, the Harry W. Colmery Veterans Educational Assistance Act of 2017 instructed VA to develop a 5-year pilot program to help veterans obtain high-technology jobs. VA created VET TEC to support veterans who enroll in high-technology education programs through VA-approved training providers.

The act includes a provision for GAO to assess VET TEC. This report examines the (1) demographic characteristics of participants, (2) employment outcomes of participants, (3) benefits and implementation challenges reported by training providers, and (4) steps VA has taken to improve the VET TEC program.

GAO analyzed data on VET TEC participants and reviewed relevant VA documents, federal laws, and regulations. GAO interviewed VA officials and a nongeneralizable sample of approved training providers selected for variation in the number of veterans served and locations. GAO also compared VA's efforts to leading practices for effective pilot design.

Recommendations

GAO is making six recommendations, including that VA develop an employment rate calculation consistent with standard approaches; determine data needed to fully inform employment outcomes; and develop clear, measurable objectives for VET TEC. VA neither agreed nor disagreed with the recommendation to develop a standardized employment rate calculation and generally agreed with the other recommendations.

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Veterans Affairs The Secretary of VA should require training providers to inform VA of the reasons why veterans drop out of the VET TEC program. (Recommendation 1)
Open
As of September 2023, VA does not plan to address this recommendation unless the VET TEC pilot program becomes permanent. Should Congress pass legislation to make the program permanent, VA will develop a timeline to begin regularly collecting data on reasons veterans drop out of the program. GAO will close this recommendation if the program does not become permanent or once VA begins collecting this information.
Department of Veterans Affairs The Secretary of VA should adjust the terminology it uses to describe its current employment rate calculation to clarify that the calculation is not measuring the employment rate but is instead measuring employment at a specific payment milestone. (Recommendation 2)
Open – Partially Addressed
VA issued a procedural advisory in October 2023 that revised its terminology to describe its employment calculation as a 180-day employment rate. However, the procedural advisory does not fully describe the populations VA included and excluded in its calculation. For example, the advisory does not specify that the calculation excludes those VET TEC participants who are within 180 days of program completion but have not found employment. GAO will close this recommendation once VA provides complete information on how it calculates the 180-day employment rate.
Department of Veterans Affairs The Secretary of VA should develop an employment rate calculation consistent with standard approaches used by government or industry entities. (Recommendation 3)
Open
As of September 2023, VA does not plan to calculate an employment rate that is consistent with approaches used by government and industry entities. VA does not plan to do so because such approaches do not produce the employment outcomes that VA is trying to convey. However, we continue to believe that developing an employment rate calculation that is consistent with other standard measures would allow VA to share information about the program's success in a manner that is commonly understood and may also allow VA to more easily compare VET TEC outcomes to other programs and measures. VA agreed that it would look to restructure its data collection if the VET TEC pilot program is made permanent. GAO will close this recommendation if the program does not become permanent or once VA begins providing the overall employment rate for the program.
Department of Veterans Affairs The Secretary of VA should use available data, determine what additional data are needed to fully inform VET TEC employment outcomes, and develop concrete time frames for collecting and reporting these data. (Recommendation 4)
Open
As of September 2023, VA plans to collect the data provided on VA Form 22-10201, inclusive of information regarding employer, job title, and salary information, and does not plan to collect any additional data. GAO continues to believe that VA should determine what additional data, if any, are needed to inform employment outcomes for VET TEC. GAO will close this recommendation once VA begins to systematically collect the data in Form 22-10201 and demonstrates that it has considered what other employment outcomes to assess, if any.
Department of Veterans Affairs The Secretary of VA should establish time frames for addressing actions it has identified for improving VET TEC. (Recommendation 5)
Closed – Implemented
In March 2022, VA identified 84 recommended actions or tasks for improving VET TEC. As of June 2023, VA had implemented 20 of the tasks, re-prioritized 37 tasks for completion by October 1, 2023, and determined it would not take further action on 27 actions. VA provided an implementation schedule showing due dates for actions it plans to implement.
Department of Veterans Affairs The Secretary of VA should identify and document a single set of clear, measurable objectives for the VET TEC program. (Recommendation 6)
Open – Partially Addressed
VA established goals and objectives for the VET TEC pilot, and in June 2023, VA had formalized a document stating the program objectives. However, it is unclear how VA plans to measure progress toward meeting the goals and objectives. As of September 2023, VA plans to complete a program evaluation to report on the stated goals and outcomes of the VET TEC pilot program. VA plans to publish the program evaluation report by the end of the first quarter of fiscal year 2025. GAO will close this recommendation once VA provides information on how it plans to assess progress toward the objectives it identified for VET TEC or once VA shares its assessment of the objectives through its program evaluation.

Full Report

GAO Contacts

Office of Public Affairs

Topics

Agency evaluationsBest practicesEducation or training costsEducational assistanceEducational standardsEmployment outcomesEmployment rateTraining programsVeteransVeterans affairsVeterans employment