Education Research:

Further Improvements Needed to Ensure Relevance and Assess Dissemination Efforts

GAO-14-8: Published: Dec 5, 2013. Publicly Released: Dec 5, 2013.

Additional Materials:

Contact:

George A. Scott
(202) 512-7215
scottg@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Department of Education's (Education) Institute of Education Sciences (IES) supports high-quality research, but lacks certain key procedures needed to fulfill other aspects of its mission. According to stakeholders, IES has substantially improved the quality of education research since its inception over a decade ago. However, GAO identified concerns with IES's ability to produce timely and relevant research. For example, IES's efforts are slow to respond to stakeholders' needs, in part, because the time IES's products have spent in peer review has substantially increased in recent years--from an average of 117 days in fiscal year 2011 to 150 days in fiscal year 2013--and IES does not monitor some aspects of these timeframes. In addition, IES does not have a structured process for incorporating stakeholder input into its research agenda, which previous GAO work has shown to be key to sound federal research programs. Lastly, IES's performance measures do not fully reflect its current programs, which is not consistent with leading practices GAO has identified for performance management. For example, IES does not publicly report on the overall performance of the Regional Educational Laboratories (REL) program, which constitutes one of the agency's largest investments. IES officials said that they have begun to develop new performance measures for all of their programs, but these will not be in place until fiscal year 2015.

Although Education's research and technical assistance groups have taken steps to disseminate relevant research to the education field, IES does not always assess these efforts. Some stakeholders raised concerns about the dissemination of relevant products from the RELs and Research and Development Centers (R & D Center). For example, they told GAO that these groups do not always adapt their products for use by both policymaker and practitioner audiences. Further, IES has not fully assessed REL and R & D Center dissemination efforts. As a result, IES does not know if these efforts are effective in meeting their mandated goal of providing usable research and information to stakeholders. GAO's prior work on information dissemination suggests that further assessment could help inform IES's oversight of the RELs and R & D Centers to improve these groups' dissemination to key audiences.

IES coordinates with other relevant federal research agencies to increase the use of research evidence in federal decision-making. For example, IES and the National Science Foundation recently released guidelines to help improve the quality of evidence resulting from federally-funded education research, which stakeholders said will benefit the education field. Within the department, IES plans evaluations of Education programs through coordination with various other offices. However, Education lacks statutory authority to combine evaluation funds for Elementary and Secondary Education Act (ESEA) programs, which officials said limits the department's ability to conduct high-quality evaluations of programs it considers most important. GAO's prior work has shown that many Education programs, especially smaller programs, have not been evaluated, limiting the ability of Congress to make informed policy decisions. For example, in 2009, GAO reported that 11 of Education's teacher quality programs had not been evaluated in more than 7 years. Officials said that the ability to combine evaluation funds would allow the department to conduct needed evaluations of ESEA programs.

Why GAO Did This Study

The federal government has a longstanding role in conducting education research and collecting education data, and IES has a broad mission to provide this information to a wide variety of stakeholders. With a fiscal year 2013 budget of just under $600 million, IES is Education's primary research and evaluation arm. GAO was asked to review IES's performance. This report examines: (1) the extent to which IES has demonstrated its ability to support high-quality research and fulfill its mission, (2) the extent to which selected Education research and technical assistance groups disseminate relevant products to the education field, and (3) how IES coordinates its activities with other relevant federal research agencies and within Education. GAO reviewed relevant federal laws and regulations, agency documents and data, interviewed agency officials and stakeholders, and analyzed information from selected research and technical assistance groups. GAO also compared IES's practices to federal internal control standards and leading practices for performance management and collaboration.

What GAO Recommends

GAO recommends that Congress consider granting Education authority to combine funds authorized for evaluation of ESEA programs, and IES use available data to manage its peer review process, develop a structured process to gather stakeholder input, develop performance measures that reflect all key agency activities, and assess REL and R & D Center dissemination strategies. Education agreed with our recommendations.

For more information, contact George Scott at (202) 512-7215 or scottg@gao.gov.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Matter for Congressional Consideration

    Matter: To ensure that Education can conduct critical program evaluation work, Congress should consider granting Education authority to combine funds authorized for evaluations of ESEA programs and target them to high-priority evaluations, with appropriate measures to ensure transparency and accountability for how the funds will be used.

    Status: Open

    Comments: When we confirm what action has been taken, we will update.

    Recommendations for Executive Action

    Recommendation: To improve the management and accountability of IES's research and evaluation efforts, IES should use available data to routinely monitor all stages of its peer review process and identify opportunities to improve the timeliness of its reviews.

    Agency Affected: Department of Education: Institute of Education Sciences

    Status: Open

    Comments: In order to ensure that all the steps in the peer review process are implemented in a timely manner, IES intends to develop a comprehensive system for monitoring the time required for all steps in the report production process. GAO will monitor the development of this comprehensive monitoring system for IES's peer review process.

    Recommendation: To improve the management and accountability of IES's research and evaluation efforts, IES should develop a structured process to systematically gather input from policymakers and practitioners and use this input when developing its research agenda.

    Agency Affected: Department of Education: Institute of Education Sciences

    Status: Open

    Comments: IES currently obtains input from practitioners and policymakers in a variety of ways, including through the National Board for Education Sciences and by convening technical working groups for input on its research programs. In order to further expand opportunities for practitioner and policymaker input, IES will annually solicit practitioner and policy input on its research agenda through the IES website and with an IES Newsflash e-mail alert. We believe this is an important step towards increasing input from these key IES stakeholder groups and will close this recommendation once these solicitations have been issued.

    Recommendation: To improve the management and accountability of IES's research and evaluation efforts, IES should develop performance measures, including targets and goals that clearly demonstrate results, and that reflect its current programs and all key agency activities, such as the performance of the RELs and its new grant programs supporting researcher-practitioner partnerships.

    Agency Affected: Department of Education: Institute of Education Sciences

    Status: Open

    Comments: As part of the Department of Education's strategic planning process, IES has reviewed its existing performance measures to align them with its current research investments, and is developing new performance measures for its knowledge utilization activities, such as the Regional Educational Laboratories. The Department plans to include these new and revised measures in a future Congressional budget justification, at which time GAO will consider closing this recommendation.

    Recommendation: To improve the management and accountability of IES's research and evaluation efforts, in order to identify leading practices and target areas for improvement, IES should assess the effectiveness of REL and R & D Center dissemination strategies by, for example, collecting consistent data and lessons learned from these groups to inform future dissemination efforts.

    Agency Affected: Department of Education: Institute of Education Sciences

    Status: Open

    Comments: IES is evaluating how to collect better data on its dissemination activities in order to assess their effectiveness, including activities conducted by the Regional Educational Laboratories (REL) and Research and Development (R & D) Centers. Once IES has identified ways to assess the effectiveness of REL and R & D Center dissemination strategies, we will consider closing this recommendation.

    Aug 22, 2014

    Jul 16, 2014

    Jun 16, 2014

    Jun 4, 2014

    Apr 30, 2014

    Mar 12, 2014

    Feb 27, 2014

    Feb 18, 2014

    Feb 7, 2014

    Looking for more? Browse all our products here