K-12 Education:

Education Needs to Improve Oversight of Its 21st Century Program

GAO-17-400: Published: Apr 26, 2017. Publicly Released: Apr 26, 2017.

Additional Materials:

Contact:

Jacqueline M. Nowicki
(617) 788-0580
nowickij@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Department of Education (Education) awards 21st Century Community Learning Centers (21st Century) grants to states, who in turn, competitively award funds to local organizations, which use them to offer academic enrichment and other activities to improve students' academic and behavioral outcomes. In their most recent grant competitions, states awarded 21st Century funds to nearly 2,400 organizations—including school districts and community-based organizations—based on a variety of criteria, such as the quality of their proposed program designs. Relevant research we reviewed that compared program participants to those of non-participants suggests that the 21st Century program is effective in improving students' behavioral outcomes, such as school-day attendance and reduced disciplinary incidents, more often than their academic outcomes. However, because Education's current 21st Century performance measures primarily focus on students' reading and math scores on state tests, Education lacks useful data about whether the program is achieving its objectives to improve students' behavioral outcomes such as attendance and discipline—the areas where the program most frequently has a positive effect. Education officials have not substantially revised the program's performance measures since 1998, in part because its authorization lapsed from fiscal years 2008 through 2016. Leading practices in performance measurement call for federal agencies to align performance measures with program objectives.

21st Century Community Learning Centers' Objectives and Performance Measures for Student Outcomes

21st Century Community Learning Centers' Objectives and Performance Measures for Student Outcomes

Education's technical assistance to states does not adequately address challenges states face in evaluating their 21st Century programs and sustaining them when program funding ends. About a third of states reported in GAO's 50-state survey that they face challenges in evaluating program performance, such as difficulty designing evaluations that shed light on program effects. Further, the 21st Century program was reauthorized twice, which resulted in significant changes to state requirements for evaluating programs. However, Education has not provided states written guidance on developing and conducting high-quality evaluations since 1999. Federal standards for internal control state that when significant changes occur agencies should periodically review policies for continued relevance and effectiveness in achieving their objectives. Absent written guidance to states on conducting high-quality evaluations, Education may miss opportunities to help states improve their capacity to conduct such evaluations to assure the program is meeting its goals.

Why GAO Did This Study

Education's 21st Century program—funded about $1 billion annually since 2002—supports a broad array of activities outside the school day to improve student outcomes in high-poverty or low-performing K-12 schools. A statement accompanying the Consolidated and Further Continuing Appropriations Act of 2015 included a provision for GAO to review Education programs outside the regular school day.

GAO examined (1) how 21st Century funds are awarded and used, (2) what is known about the effectiveness of these programs, (3) how Education manages and uses program data to inform decision making, and (4) Education's technical assistance for evaluating and sustaining programs. GAO conducted a 50-state survey of program officials, obtaining a 100 percent response rate. GAO also reviewed selected state program evaluations and academic studies on student outcomes, and observed program activities and interviewed officials in four states representing a range of grant size and location.

What GAO Recommends

GAO is making four recommendations, including that Education expand its performance measures for behavioral outcomes and provide written guidance to states on conducting high-quality program evaluations. Education neither agreed nor disagreed with the recommendations, and outlined steps it is taking to address them.

For more information, contact Jacqueline M. Nowicki at (617) 788-0580 or nowickij@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: Education neither agreed nor disagreed with our recommendations; rather, it generally noted that it will keep our recommendations in mind as it continues to implement changes in the program as a result of ESSA. Specifically, the department stated that it is in the process of re-examining whether additional or revised measures should be developed to align more significantly with the program's statutory objectives under ESSA. Education also expressed concern about collecting data on student attendance and disciplinary measures, noting that it will require effective collaboration between states, districts, and other eligible entities. However, we note that about half of states already collect data on at least one of these two measures; and that research has shown that 21st Century programs more often have positive effects on student attendance and reducing disciplinary incidents than on improving students' academic outcomes. Given these effects, we continue to believe that it is critical for Education to measure student attendance and disciplinary incidents to obtain more complete, accurate information on this program's effect on student outcomes.

    Recommendation: The Secretary of Education should direct the Office of Academic Improvement to expand its performance measures for the 21st Century program to address all program objectives. Specifically, Education should establish performance measures related to key behavioral, including student attendance and disciplinary incidents, and socio-emotional outcomes.

    Agency Affected: Department of Education

  2. Status: Open

    Comments: Education neither agreed nor disagreed with this recommendation. Education commented that it plans to build in additional data checks into the data system beyond its current checks on the data's completeness. Specifically, Education anticipates that new technology enhancements in the data system will be designed to flag for inconsistencies in data reporting. For example, the system may send a "flag" that participation data is significantly lower or higher than previously reported participation data. Further, Education indicated that it will consider whether auditors performing audits under the Single Audit Act can be asked and guided to do more checks on the accuracy and reliability of 21st Century program data.

    Recommendation: The Secretary of Education should direct the Office of Academic Improvement to conduct federal-level data checks on the accuracy of 21st Century program data submitted by states. Such checks could test for logical relationships between fields. Education should also publicly disclose and address any data limitations it identifies, as appropriate.

    Agency Affected: Department of Education

  3. Status: Open

    Comments: Education neither agreed nor disagreed with this recommendation. Education outlined several steps it has taken to assist states in the past. For example, Education said that it provided six states with individualized technical assistance on strategies related to developing statewide evaluations and measures. Education also noted that, to date, it has conducted two webinars on state evaluations and is in the process of including presentations from those webinars on its online learning portal so that states will have easy access to the information. In addition, Education stated that it included presentations on evaluation strategies in the past during its Summer Institute. Education also said it would consider whether additional guidance for all states was needed. While these are important steps, we do not believe they are sufficient. We continue to believe that Education should prepare written guidance to assist all states in developing and conducting high-quality program evaluations.

    Recommendation: The Secretary of Education should direct the Office of Academic Improvement to provide written, non-regulatory guidance to states on developing and conducting high-quality 21st Century state evaluations to help address the difficulties states face in measuring program performance and effectiveness.

    Agency Affected: Department of Education

  4. Status: Open

    Comments: Education neither agreed nor disagreed with this recommendation. Education noted that it hosts meetings twice a year for 21st Century state coordinators where strategies related to program sustainability are shared with states. These meetings covered topics such as reducing the amounts of 21st Century grant awards by a percentage each year. However, these meetings have not focused on topics on program sustainability for several years. We continue to believe that Education should take the lead in sharing information with states to help them address their sustainability challenges by sharing information on state policies and practices that have shown some success.

    Recommendation: The Secretary of Education should direct the Office of Academic Improvement to use the information it collects from its monitoring visits and ongoing interactions with states to share effective practices across states for sustaining their 21st Century programs once program funding ends. This information could be shared using existing mechanisms such as Education's meetings with 21st Century state coordinators.

    Agency Affected: Department of Education

 

Explore the full database of GAO's Open Recommendations »

Jul 18, 2017

Jul 12, 2017

May 24, 2017

May 18, 2017

May 4, 2017

Apr 26, 2017

Feb 9, 2017

Nov 30, 2016

Looking for more? Browse all our products here