Science, Technology, Engineering, and Mathematics Education:

Strategic Planning Needed to Better Manage Overlapping Programs across Multiple Agencies

GAO-12-108: Published: Jan 20, 2012. Publicly Released: Jan 24, 2012.

Additional Materials:

Contact:

George A. Scott
(202) 512-3000
scottg@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

In fiscal year 2010, 13 federal agencies invested over $3 billion in 209 programs designed to increase knowledge of STEM fields and attainment of STEM degrees. The number of programs within agencies ranged from 3 to 46, with the Departments of Health and Human Services and Energy and the National Science Foundation administering more than half of these programs. Almost a third of the programs had obligations of $1 million or less, while some had obligations of over $100 million. Beyond programs specifically focused on STEM education, agencies funded other broad efforts that contributed to enhancing STEM education.

Eighty-three percent of the programs GAO identified overlapped to some degree with at least 1 other program in that they offered similar services to similar target groups in similar STEM fields to achieve similar objectives. Many programs have a broad scope—serving multiple target groups with multiple services. However, even when programs overlap, the services they provide and the populations they serve may differ in meaningful ways and would therefore not necessarily be duplicative. Nonetheless, the programs are similar enough that they need to be well coordinated and guided by a robust strategic plan. Currently, though, less than half of the programs GAO surveyed indicated that they coordinated with other agencies that administer similar STEM education programs. Current efforts to inventory federal STEM education activities and develop a 5-year strategic plan present an opportunity to enhance coordination, align governmentwide efforts, and improve efficiency of limited resources by identifying opportunities for program consolidation and reducing administrative costs.

Agencies’ limited use of performance measures and evaluations may hamper their ability to assess the effectiveness of their individual programs as well as the overall STEM education effort. Specifically, program officials varied in their ability to provide reliable output measures—for example, the number of students, teachers, or institutions directly served by their program. Further, most agencies did not use outcomes measures in a way that is clearly reflected in their performance planning documents. This may hinder decision makers’ ability to assess how agencies' STEM education efforts contribute to agencywide performance goals and the overall federal STEM effort. In addition, a majority of programs did not conduct comprehensive evaluations since 2005 to assess effectiveness, and the evaluations GAO reviewed did not always align with program objectives. Finally, GAO found that completed STEM education evaluation results had not always been disseminated in a fashion that facilitated knowledge sharing between both practitioners and researchers.

Why GAO Did This Study

Science, technology, engineering, and mathematics (STEM) education programs help to enhance the nation’s global competitiveness. Many federal agencies have been involved in administering these programs. Concerns have been raised about the overall effectiveness and efficiency of STEM education programs.

GAO examined (1) the number of federal agencies and programs that provided funding for STEM education programs in fiscal year 2010; (2) the extent to which STEM education programs have similar objectives, serve similar target groups, and provide similar types of services, and, if necessary, what opportunities exist to increase coordination; and (3) the extent to which STEM education programs measured effectiveness. To answer these questions, GAO reviewed relevant federal laws, regulations, and plans; surveyed federal STEM education programs; analyzed programs’ STEM evaluations; and interviewed relevant federal officials. An electronic supplement—GAO-12-110SP—provides survey results.

What GAO Recommends

GAO recommends that as OSTP leads the governmentwide STEM education strategic planning effort, it should work with agencies to better align their activities with a governmentwide strategy, develop a plan for sustained coordination, identify programs for potential consolidation or elimination, and assist agencies in determining how to better evaluate their programs. OSTP provided technical comments that we incorporated as appropriate. OMB had no concerns with the report.

For more information, contact George A. Scott at (202) 512-7215 or scottg@gao.gov.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: The Director of Office of Science and Technology Policy (OSTP) should direct National Science and Technology Council (NSTC) to develop guidance for how agencies can better incorporate each agency's STEM education efforts and the goals from NSTC's 5-year STEM education strategic plan into each agency's own performance plans and reports.

    Agency Affected: Executive Office of the President: Office of Science and Technology Policy

    Status: Open

    Comments: The administration named STEM education as an interim cross agency priority goal in the President's 2013 budget submission and in May 2013 NSTC released its STEM Education 5-Year Strategic Plan. However, the Plan did not include guidance for agencies on how to incorporate STEM efforts into agency performance plans and reports, as GAO recommended. As a cross-agency priority goal, the administration must report quarterly on the progress agencies have made towards the goal of increasing the number of graduates with a STEM degree by 1 million over the next 10 years. While the administration released its FY13 second quarter progress update, there was no mention of agency performance plans and reports. As of August 2013, some agencies have already begun to align performance plans with the draft goals and priorities of NSTC's Strategic Plan and their STEM education efforts. For instance, according to OSTP, the National Science Foundation has an agency priority goal that is focused on the use of evidence-based practices in undergraduate STEM education. Although some progress has been made, issuing guidance to all agencies identified as having STEM education programs would better position agencies to align their efforts to the broader government-wide goals and concentrate federal resources on advancing those goals.

    Recommendation: The Director of OSTP should direct NSTC to develop a framework for how agencies will be monitored to ensure that they are collecting and reporting on NSTC strategic plan goals. This framework should include alternatives for a sustained focus on monitoring coordination of STEM programs if the NSTC Committee on STEM terminates in 2015 as called for in its charter.

    Agency Affected: Executive Office of the President: Office of Science and Technology Policy

    Status: Open

    Comments: In May 2013, NSTC released its STEM Education 5-Year Strategic Plan, which includes roadmaps and implementation plans for STEM education investment priority areas and interagency coordination approaches; however, the framework for monitoring and reporting on these areas still needs to be developed. The strategic plan states that CoSTEM will create implementation subcommittees for the STEM education investment priority areas and interagency coordination approaches who will be responsible for 1) reviewing roadmaps and implementation plans, 2) tracking implementation of priority areas, 3) developing and monitoring metrics for progress, and 4) developing a framework and process for more coordinated Federal STEM education budget planning. According to OSTP officials, as of August 2013, the implementation subcommittees have meet twice but no formal timeline has been set for its activities. Although progress has been made, GAO would need to see the monitoring and reporting frameworks developed by the implementation subcommittees to close the recommendation. A framework for monitoring agency progress towards NSTC's Strategic Plan is necessary to improve transparency and strengthen accountability of NSTC's strategic planning and coordination efforts.

    Recommendation: The Director of OSTP should direct NSTC to work with agencies, through its strategic planning process, to identify programs that might be candidates for consolidation or elimination. Specifically, this could be achieved through an analysis that includes information on program overlap, similar to the analysis conducted by GAO in this report, and information on program effectiveness. As part of this effort, OSTP should work with agency officials to identify and report any changes in statutory authority necessary to execute each specific program consolidation identified by NSTC's strategic plan.

    Agency Affected: Executive Office of the President: Office of Science and Technology Policy

    Status: Closed - Implemented

    Comments: OSTP directed NSTC through its strategic planning process to identify programs that might be candidates for consolidation or elimination. Specifically, in the Coordinating Federal Science, Technology, Engineering, and Mathematics (STEM) Education Investments: Progress Report, published in February 2012, NSTC identified a number of programs that could be eliminated in FY13. In addition, the President's FY14 budget calls for major restructuring of federal STEM education programs through the consolidation of 90 programs and the realignment of STEM education activities.

    Recommendation: The Director of OSTP should direct NSTC to develop guidance to help agencies determine the types of evaluations that may be feasible and appropriate for different types of STEM education programs and develop a mechanism for sharing this information across agencies. This could include guidance and sharing of information that outlines practices for evaluating similar types of programs.

    Agency Affected: Executive Office of the President: Office of Science and Technology Policy

    Status: Closed - Implemented

    Comments: In May 2013, NSTC released its STEM Education 5-Year Strategic Plan, which includes guidance to agencies in developing evaluations for STEM education programs. Specifically, NSTC's Strategic Plan includes a coordination objective focused on improving and aligning evaluation and research strategies across federal agencies and a set of design principles that provides guidance for how federal STEM programs should be structured and evaluated, based on current evidence and best practices. In addition to NSTC's Strategic Plan, agencies have already begun to develop shared strategies for generating evidence. For instance, the National Science Foundation and the Department of Education developed a common evidence framework and guidelines for various education research study types.

    Jul 16, 2014

    Jun 16, 2014

    Jun 4, 2014

    Apr 30, 2014

    Mar 12, 2014

    Feb 27, 2014

    Feb 18, 2014

    Feb 7, 2014

    Feb 5, 2014

    Looking for more? Browse all our products here