Employment and Training Administration:

Increased Authority and Accountability Could Improve Research Program

GAO-10-243: Published: Jan 29, 2010. Publicly Released: Jan 29, 2010.

Additional Materials:

Contact:

George A. Scott
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

With current rising unemployment rates and the need for a more skilled workforce, it is important for the Department of Labor's (Labor) Employment and Training Administration (ETA) to invest in sound research that identifies the most effective and efficient ways to train and employ workers for 21st century jobs. While ETA traditionally has played an important role in providing job training, employment assistance, and labor market information for the nation's workers, the current unemployment crisis has made this role more critical. The Bureau of Labor Statistics estimates that jobless rates have increased over the past year in all 50 states and the District of Columbia. In particular, several states have reported unemployment rates well over 10 percent. As ETA's new leadership works to help the nation meet these economic challenges, it must have solid information that is supported by sound research to guide decision-making. Since 2002, GAO and others have criticized ETA for not focusing sufficient attention on its research program, particularly with regard to complying with congressional mandates, conducting policy-relevant research, and disseminating key research findings in a timely way. In this context, we have examined the structure and processes of ETA's research and evaluation center in terms of the elements that leading national research organizations cite as essential to a sound program; that is, research independence, transparency and accountability, and policy relevance. Based on these elements, we addressed the following questions: (1) How does ETA's organizational structure provide for research independence? (2) What steps has ETA taken to promote transparency and accountability in its research program? (3) How does ETA ensure that its research is relevant to workforce development policy and practice?

In summary, we found that ETA's research center lacks independent authority for research, has limitations with regard to transparency and accountability processes, has not routinely involved stakeholders in developing its research agenda, and has been slow to evaluate the programs and activities carried out under the Workforce Investment Act of 1998 (WIA). (1) ETA is currently in the process of revising its organizational structure. Previously, its research and evaluation center, within the Office of Policy Development and Research (OPDR), was several levels removed from the Office of the Assistant Secretary and lacked independent authority for conducting research. Under ETA's revised structure, it is unclear whether OPDR will report directly to the Assistant Secretary or the Deputy Assistant Secretary. Furthermore, unlike the heads of some other research and evaluation centers, the head of ETA's research and evaluation center does not have the authority to set the research agenda, to approve requests for funded projects, or to disseminate research and evaluation reports. (2) While ETA has recently made improvements to its research program, some limitations remain with regard to its accountability processes and timely dissemination of research products. In 2007, ETA documented all stages of its research process from project selection to dissemination. ETA also began coordinating its research process with the Office of Management and Budget (OMB) and renewed its practice of sponsoring biennial research conferences to address a range of policy and program issues. Despite these recent changes, ETA's research processes lack specificity, including specific time frames for key milestones and an established criteria for peer review. In addition, ETA's research program lacks an information system to track its research from completion to dissemination and does not have a mechanism to ensure that research findings are disseminated in a timely manner. In 2008, ETA disseminated 34 research products to the public. However, 20 of these products, which had a combined cost of about $28 million, were delayed for between 2 and 5 years. (3) Labor, consistent with recent efforts of OMB, has taken steps to emphasize the value of research and evaluation, but ETA does not have sufficient mechanisms to ensure that its research can inform policy decisions. OMB currently has efforts underway to help agencies to enhance their research and evaluation programs. Labor also announced plans to create a position of chief evaluation officer and a new evaluation center within the Office of the Assistant Secretary for Policy to foster research relevant to policy. However, ETA has been slow to comply with a congressional mandate to evaluate WIA1 and it also lacks a standard process and advisory bodies to consistently involve stakeholders in the development of its research agenda. Similarly, ETA has no formal process in place to ensure that research findings are used to inform strategic planning and policy.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: Concerning the structure and processes of ETA's research and evaluation center, the Secretary of Labor should create an information system to track research projects.

    Agency Affected: Department of Labor

    Status: Open

    Comments: Education and Training Administration (ETA) is planning to implement a centralized tracking system for its research projects. However, the time frames for implementing this system are unclear. In FY2012, ETA reported that it is still determining how to develop the centralized tracking system for its research and evaluation projects. Labor did not provide an update in FY2013. When this system is implemented, GAO will be consider this recommendation implemented.

    Recommendation: Concerning the structure and processes of ETA's research and evaluation center, the Secretary of Labor should direct ETA's research and evaluation center to establish more specific processes, including time frames for dissemination of research.

    Agency Affected: Department of Labor

    Status: Closed - Implemented

    Comments: The Education and Training Administration (ETA) has developed dissemination procedures for reviewing and finalizing reports. In addition, they have added specific timeframes for dissemination to the performance standards for Federal Project Officers who oversee the research. In FY12, ETA provided a copy of these performance standards.

    Recommendation: Concerning the structure and processes of ETA's research and evaluation center, the Secretary of Labor should provide sufficient decision-making authority to ETA's research and evaluation center regarding its research.

    Agency Affected: Department of Labor

    Status: Closed - Implemented

    Comments: In the spring of 2010, the Department of Labor (DOL) established a new Chief Evaluation Officer position within their Office of the Assistant Secretary for Policy. The Chief Evaluation Officer's job is to coordinate department-wide evaluation efforts, however, it is unclear to what extent the Chief Evaluation Officer's position within the department interacts with Education and Training Administration's (ETA) research and evaluation center and to what extent the Chief Evaluation Officer has responsibilities to ETA's research. In FY12, ETA reported that it formally incorporated the routine involvement of the CEO into its research process (this also addresses GAO's recommendation in GAO-11-285). Such routine involvement has included: reviewing select research, demonstration and evaluation statements of work; discussing select research and evaluation methodologies with ETA and OMB; discussing, on a monthly basis, on-going ETA research and evaluation projects; participating in meetings with grantees involved in major research demonstrations to help explain the importance of rigorous evaluations; and providing input on how to best incorporate research in ETA's Solicitations for Grant Applications (SGAs). In the coming year, ETA plans to reflect the existing collaboration in its formal research process documentation. For example, the plan will be updated to document the involvement of the CEO at critical research milestones including the agenda setting phase where research, demonstration and evaluation priorities are determined, the conceptualization and design phase, the procurement phase, the implementation phase, and the review and dissemination phases.

    Recommendation: Concerning the structure and processes of ETA's research and evaluation center, the Secretary of Labor should take steps to ensure that ETA clarifies its organizational structure and OPDR reports directly to ETA's Assistant Secretary.

    Agency Affected: Department of Labor

    Status: Open

    Comments: In an organizational chart from August of 2009, the Education and Training Administration's (ETA) Office of Policy Development and Research (OPDR) reported to a Deputy Assistant Secretary who reported to ETA's Assistant Secretary. This was changed in November 2009 to a structure where OPDR reported directly to the Assistant Secretary. However, as of April 2010, OPDR reported to a Deputy Assistant secretary who reports to ETA's Assistant Secretary. In FY12, ETA reported that in its organizational chart, dated May 2011, OPDR still reports to a Deputy Assistant Secretary who reports to ETA's Assistant Secretary (http://www.doleta.gov/pdf/OrgChart.pdf). This structure remains the same in ETA's July 31, 2013 organizational chart. GAO will close this recommendation when clarifies its structure in some way, such as updating the organizational chart. Labor did not provide an update on its actions for FY13.

    Recommendation: Concerning the structure and processes of ETA's research and evaluation center, the Secretary of Labor should instruct ETA's research and evaluation center to develop processes to involve outside experts in setting its research agenda.

    Agency Affected: Department of Labor

    Status: Closed - Implemented

    Comments: In preparing its new research agenda, the Education and Training Administration (ETA) hosted a series of consultations with external stakeholders and has informally engaged in dialogues with outside research and program experts. In FY12, ETA reported that it had taken steps to engage outside experts more fully in setting its research agenda. Specifically, the agency: 1) included outside experts in the development of its five-year research plan (research agenda); 2) formed advisory and peer review groups for major evaluations; 3) developed, with the Coalition for Evidence-Based Policy, the capability to conduct peer reviews on an ad hoc basis; and 4) had the CEO, who is in DOL but independent of ETA, also review reports. In addition, ETA is collecting and synthesizing feedback from a variety of sources for developing its five-year research plan (research agenda) including input received from experts and workforce system practitioners at ETA's conferences, and feedback solicited from stakeholders in the workforce community. ETA also plans to analyze public comments that it will solicit when the draft plan is announced in the Federal Register and posted on the ETA web site. ETA also reported that it strengthened its collaboration with other federal agency experts at the Departments of Education and Health and Human Services in planning and carrying out research and evaluation studies. ETA plans to continue engaging outside experts in developing research policies and processes. Within the coming year, ETA also plans to reflect such engagement in its formal research processes. ETA formally incorporated the routine involvement of outside experts in its research process, as indicated in its response to a related GAO report on its research efforts(GAO-11-285).

    Jul 23, 2014

    May 15, 2014

    Apr 9, 2014

    Mar 5, 2014

    Jan 23, 2014

    Dec 20, 2013

    Dec 6, 2013

    Dec 2, 2013

    Nov 4, 2013

    Jul 19, 2013

    Looking for more? Browse all our products here