Youthbuild Program:

Analysis of Outcome Data Needed to Determine Long-Term Benefits

GAO-07-82: Published: Feb 28, 2007. Publicly Released: Feb 28, 2007.

Additional Materials:

Contact:

William B. Shear
(202) 512-4325
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Since 1993, the Department of Housing and Urban Development (HUD) has provided funding for Youthbuild, a competitive grant program that trains and educates disadvantaged youth and helps build low-income housing. In 2006, Youthbuild was transferred to the Department of Labor (Labor) to better align the Youthbuild program with existing youth workforce and training programs. In response to concerns about the overall quality of Youthbuild, a Senate report directed GAO to assess the program. GAO's objectives included (1) evaluating how HUD assessed and oversaw the program, (2) determining what results the program achieved, and (3) assessing how successful grantees were in obtaining outside funding. GAO analyzed Youthbuild performance data, visited Youthbuild sites, and interviewed agency officials.

While HUD requires grantees to report basic performance data, such as the number of program participants and graduates and job placements, HUD has not aggregated or analyzed the data and conducted limited oversight of grantees. According to HUD officials, they did not have staff available to analyze the closeout reports that grantees must submit, and a lack of resources also limited oversight of grantees. The monitoring HUD did primarily focused on compliance with program requirements such as documentation rather than on performance. As a result, HUD largely was unable to tell how the individual Youthbuild grantee programs performed. Limited outcome data preclude any overall assessment of the performance of the Youthbuild program; further, the few other analyses available such as the one GAO did in this study to augment limited existing data cannot be generalized programwide. GAO analyzed 245 closeout reports, representing 46 percent of the grantees or 12,863 participants. While GAO could determine percentages of participants who received high school diplomas or were placed in jobs, GAO could not determine outcomes over time, partly because the reports lacked baseline information and grantees were not required to and generally did not follow participants after graduation. Further, while closeout reports include information about impediments to program success and "best practices," HUD did not systematically review this information or share it with its primary technical assistance contractor. Consequently, the lack of programwide evaluations, follow-up data, and dissemination of best practices make it very difficult to assess the performance of Youthbuild over time and determine which programs and strategies have worked best. Reporting on post-program performance outcomes, such as the number of participants placed and retained in construction-related employment, could increase the value of the closeout reports and better measure program results. Labor officials indicated that they would consider including such measures for program reporting. Grantees had varying success in obtaining funds from outside sources, but YouthBuild USA data suggest that continued (multiyear) HUD funding was critical to sustaining grantee operations and attracting leveraged funds. Grantees' success in obtaining additional funds varied widely, from 21 grants reporting no additional funding sources to 40 reporting more than $1 million. While most grantees have generated outside funding, YouthBuild USA reported that most grantees have had difficulty continuing operations without continued HUD funding. Their data show that 90 percent of grantees ceased operations if not funded for 3 consecutive years by HUD. Further, YouthBuild USA also noted that grantees with follow-on HUD funding achieved better performance outcomes, such as higher rates of job placements, than grantees that did not receive subsequent HUD funding.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: We recommended that the Secretary of HUD analyze closeout reports by grant and share information on identified problems and best practices with its technical services contractor and Labor. HUD contracted with a Youthbuild technical assistance provider to develop a methodology to analyze the closeout performance reports for the past several years. In August 2008, Partners for Community Solutions supplied HUD with a draft report. The background section cites the GAO recommendations and states that the report was being done in response to those recommendations. The report reviewed 205 final reports from 143 Youthbuild organizations between 2000 and 2003. It concluded that several best practices and common impediments were found which should help existing Youthbuild organizations manage their programs more effectively and help new grantees to design programs based on years of refinement from other grantees.

    Recommendation: To improve the reporting and assessment of performance for Youthbuild grantee programs and develop the bases for an overall assessment of the program, the Secretary of HUD should analyze closeout reports by grant and share information on identified problems and "best practices" with its technical services contractor and Labor.

    Agency Affected: Department of Housing and Urban Development

  2. Status: Closed - Implemented

    Comments: In a letter dated 4/24/2007, DOL wrote that it agreed with the recommendation and was building a web-based MIS that will collect individual data for all Youthbuild participants enrolled in the program. The letter also said that Youthbuild grantees will be required to collect and report on individual demographics, assessment information such as educational achievement, services, and outcomes including placement in employment/education, attainment of degree/certificate, retention rates, recidivism rates, wage rates, etc.; that DOL will collect the data quarterly; and that DOL will aggregate the data at the grantee level as well as share the data with grantees. In the 2007 Youthbuild Program Notice of Funds Availability, DOL confirmed that grantees will be required to submit quarterly progress reports, including in-depth information on accomplishments and progress towards performance outcomes, including updates on products, curricula, and training development.

    Recommendation: To improve the reporting and assessment of performance for Youthbuild grantee programs and develop the bases for an overall assessment of the program, the Secretary of Labor should develop and monitor post-program performance outcome measures for the YouthBuild program, such as the types of employment graduates attained and retained, wage rates, and degrees or certifications received, and share the data with the grantees.

    Agency Affected: Department of Labor

  3. Status: Closed - Implemented

    Comments: We reported that data showed that 90 percent of Youthbuild programs not funded for 3 consecutive years ceased operations. In commenting on the draft report, the Department of Labor's (Labor)Assistant Secretary for Employment and Training in a February 2007 letter wrote that Labor agreed that there could be value on multiyear funding and that the Department was considering such an approach. In a letter dated April 24, 2007 the Assistant Secretary wrote that she agree that there is value in developing a multi-year grant cycle and that it was weighing options for providing multiple years of funding in the grant announcement that would be published in the spring of 2007. The grant announcement, dated April 26, 2007 stated that applicants can apply for three-year grants.

    Recommendation: To improve the reporting and assessment of performance for Youthbuild grantee programs and develop the bases for an overall assessment of the program, the Secretary of Labor should consider whether multiyear funding could be useful in helping YouthBuild grantees attract additional outside funding.

    Agency Affected: Department of Labor

 

Explore the full database of GAO's Open Recommendations »

Sep 4, 2018

Jun 29, 2018

Apr 27, 2017

Apr 13, 2017

Sep 7, 2016

Sep 6, 2016

Aug 24, 2016

May 26, 2016

Apr 18, 2016

Mar 24, 2016

Looking for more? Browse all our products here