Employment and Training Administration:

Increased Authority and Accountability Could Improve Research Program

GAO-10-243: Published: Jan 29, 2010. Publicly Released: Jan 29, 2010.

Additional Materials:

Contact:

George A. Scott
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

With current rising unemployment rates and the need for a more skilled workforce, it is important for the Department of Labor's (Labor) Employment and Training Administration (ETA) to invest in sound research that identifies the most effective and efficient ways to train and employ workers for 21st century jobs. While ETA traditionally has played an important role in providing job training, employment assistance, and labor market information for the nation's workers, the current unemployment crisis has made this role more critical. The Bureau of Labor Statistics estimates that jobless rates have increased over the past year in all 50 states and the District of Columbia. In particular, several states have reported unemployment rates well over 10 percent. As ETA's new leadership works to help the nation meet these economic challenges, it must have solid information that is supported by sound research to guide decision-making. Since 2002, GAO and others have criticized ETA for not focusing sufficient attention on its research program, particularly with regard to complying with congressional mandates, conducting policy-relevant research, and disseminating key research findings in a timely way. In this context, we have examined the structure and processes of ETA's research and evaluation center in terms of the elements that leading national research organizations cite as essential to a sound program; that is, research independence, transparency and accountability, and policy relevance. Based on these elements, we addressed the following questions: (1) How does ETA's organizational structure provide for research independence? (2) What steps has ETA taken to promote transparency and accountability in its research program? (3) How does ETA ensure that its research is relevant to workforce development policy and practice?

In summary, we found that ETA's research center lacks independent authority for research, has limitations with regard to transparency and accountability processes, has not routinely involved stakeholders in developing its research agenda, and has been slow to evaluate the programs and activities carried out under the Workforce Investment Act of 1998 (WIA). (1) ETA is currently in the process of revising its organizational structure. Previously, its research and evaluation center, within the Office of Policy Development and Research (OPDR), was several levels removed from the Office of the Assistant Secretary and lacked independent authority for conducting research. Under ETA's revised structure, it is unclear whether OPDR will report directly to the Assistant Secretary or the Deputy Assistant Secretary. Furthermore, unlike the heads of some other research and evaluation centers, the head of ETA's research and evaluation center does not have the authority to set the research agenda, to approve requests for funded projects, or to disseminate research and evaluation reports. (2) While ETA has recently made improvements to its research program, some limitations remain with regard to its accountability processes and timely dissemination of research products. In 2007, ETA documented all stages of its research process from project selection to dissemination. ETA also began coordinating its research process with the Office of Management and Budget (OMB) and renewed its practice of sponsoring biennial research conferences to address a range of policy and program issues. Despite these recent changes, ETA's research processes lack specificity, including specific time frames for key milestones and an established criteria for peer review. In addition, ETA's research program lacks an information system to track its research from completion to dissemination and does not have a mechanism to ensure that research findings are disseminated in a timely manner. In 2008, ETA disseminated 34 research products to the public. However, 20 of these products, which had a combined cost of about $28 million, were delayed for between 2 and 5 years. (3) Labor, consistent with recent efforts of OMB, has taken steps to emphasize the value of research and evaluation, but ETA does not have sufficient mechanisms to ensure that its research can inform policy decisions. OMB currently has efforts underway to help agencies to enhance their research and evaluation programs. Labor also announced plans to create a position of chief evaluation officer and a new evaluation center within the Office of the Assistant Secretary for Policy to foster research relevant to policy. However, ETA has been slow to comply with a congressional mandate to evaluate WIA1 and it also lacks a standard process and advisory bodies to consistently involve stakeholders in the development of its research agenda. Similarly, ETA has no formal process in place to ensure that research findings are used to inform strategic planning and policy.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In 2010, the Department of Labor (DOL) established the Chief Evaluation Office (CEO) to oversee the department's research and evaluation efforts. CEO responsibilities include maintaining a comprehensive inventory of all past, ongoing, and planned evaluation activities within DOL; and ensuring evaluation program and findings are transparent, credible, and accessible to the public. Since that time, the Employment and Training Administration (ETA) has formally incorporated the CEO in its formal research process. While ETA has not changed its organizational structure, we believe the creation and involvement of this office helps to improve the transparency and accountability of ETA's research program.

    Recommendation: Concerning the structure and processes of ETA's research and evaluation center, the Secretary of Labor should take steps to ensure that ETA clarifies its organizational structure and OPDR reports directly to ETA's Assistant Secretary.

    Agency Affected: Department of Labor

  2. Status: Closed - Implemented

    Comments: ETA's Research and Evaluation Division reported in FY 2014 that it submits an annual operating plan to the Secretary of Labor, which identifies milestones and outputs for the coming year, as well as performance levels. Some of their strategies include: (1) using an annual consultation process with all ETA program offices to develop and implement a 5-year research agenda; and (2) ensuring completion of internal research and analysis and procurement and management of grants and contractors for pilots, demonstrations, research and evaluations. With increased involvement of outside stakeholders, DOL's Chief Evaluation Officer (CEO) and others, ETA's research and evaluation center should have sufficient decision-making authority to ensure transparency and the dissemination of research.

    Recommendation: Concerning the structure and processes of ETA's research and evaluation center, the Secretary of Labor should provide sufficient decision-making authority to ETA's research and evaluation center regarding its research.

    Agency Affected: Department of Labor

  3. Status: Closed - Implemented

    Comments: The Department of Labor's Employment and Training Administration (ETA) has developed dissemination procedures for reviewing and finalizing reports. The agency also added specific timeframes for dissemination to the performance standards for federal project officers who oversee the research.

    Recommendation: Concerning the structure and processes of ETA's research and evaluation center, the Secretary of Labor should direct ETA's research and evaluation center to establish more specific processes, including time frames for dissemination of research.

    Agency Affected: Department of Labor

  4. Status: Closed - Implemented

    Comments: The Department of Labor's Employment and Training Administration (ETA), in collaboration with the DOL's Chief Evaluation office (CEO), has been working with a contractor to develop a more robust system with enhanced reporting functions that tracks planning, procurement, and project management throughout the project lifecycle. A demonstration and presentation of the software was made to ETA and CEO in April and May 2014. Final changes are being completed in September 2014. ETA anticipates staff training in October 2014.

    Recommendation: Concerning the structure and processes of ETA's research and evaluation center, the Secretary of Labor should create an information system to track research projects.

    Agency Affected: Department of Labor

  5. Status: Closed - Implemented

    Comments: In preparing its new research agenda, the Employment and Training Administration (ETA) hosted a series of consultations with external stakeholders and has informally engaged in dialogues with outside research and program experts. In FY12, ETA reported that it had taken steps to engage outside experts more fully in setting its research agenda (plan). Specifically, the agency: 1) included outside experts in the development of its 5-year research plan; 2) formed advisory and peer review groups for major evaluations; 3) developed, with the Coalition for Evidence-Based Policy, the capability to conduct peer reviews on an ad hoc basis; and 4) had DOL's CEO, who is independent of ETA, review reports. And ETA now requires the identification of experts as members of a technical working group to provide assistance at key points in the research process for large scale contracts. In addition, ETA is collecting and synthesizing feedback from a variety of sources for developing its 5-year research plan including input received from experts and workforce system practitioners at ETA's conferences, and feedback solicited from stakeholders in the workforce community. ETA also plans to analyze public comments that it will solicit when the draft plan is announced in the Federal Register and posted on the ETA web site. ETA also reported that it strengthened its collaboration with other federal agency experts at the Departments of Education and Health and Human Services in planning and carrying out research and evaluation studies. Also see related recommendations in GAO-11-285.

    Recommendation: Concerning the structure and processes of ETA's research and evaluation center, the Secretary of Labor should instruct ETA's research and evaluation center to develop processes to involve outside experts in setting its research agenda.

    Agency Affected: Department of Labor

 

Explore the full database of GAO's Open Recommendations »

Nov 13, 2014

Jul 23, 2014

May 15, 2014

Apr 9, 2014

Mar 5, 2014

Jan 23, 2014

Dec 20, 2013

Dec 6, 2013

Dec 2, 2013

Nov 4, 2013

Looking for more? Browse all our products here