Promise Neighborhoods Promotes Collaboration but Needs National Evaluation Plan
GAO-14-432: Published: May 5, 2014. Publicly Released: Jun 4, 2014.
What GAO Found
The Department of Education (Education) used a two-phase strategy for awarding Promise Neighborhoods (Promise) grants, and aligned grant activities with program goals. Education awarded 1-year planning grants to organizations with the potential to effectively align services for students in their respective neighborhoods. Planning grants were generally intended to enhance the grantees' capacity to plan a continuum of services. Through a separate competition, Education awarded 5-year implementation grants to organizations that demonstrated they were most ready to implement their plans. However, Education did not communicate clearly to grantees about its expectations for the planning grants and the likelihood of receiving implementation grants. As a result, some grantees experienced challenges sustaining momentum in the absence or delay of implementation grant funding.
The Promise program coordinates with related federal efforts primarily through a White House initiative that brings together neighborhood grant programs at five federal agencies. The Promise program's efforts are focused on ensuring that grants are mutually reinforcing by aligning goals, developing common performance measures, and sharing technical assistance resources. While Promise grantees incorporate a wide range of federal programs in their local strategies, Education coordinates with a more limited number of federal programs. Officials told us that they do this to avoid spreading program resources too thin. Further, Education did not develop an inventory of the federal programs that share Promise goals, a practice that could assist grantees; help officials make decisions about interagency coordination; and identify potential fragmentation, overlap, and duplication.
Education requires Promise grantees to develop information systems and collect extensive data, but it has not developed plans to evaluate the program. Specifically, implementation grantees must collect data on individuals they serve, services they provide, and related outcomes and report annually on multiple indicators. However, Education stated it must conduct a systematic examination of the reliability and validity of the data to determine whether it will be able to use the data for an evaluation. Absent an evaluation, Education cannot determine the viability and effectiveness of the Promise program's approach.
The Promise grant enabled grantees and their partners to collaborate in ways that align with leading practices GAO previously identified for enhancing collaboration among interagency groups including establishing common outcomes, leveraging resources, and tracking performance. For example, Education required grantees to work with partners to develop common goals and a plan to use existing and new resources to meet identified needs in target areas. Grantees were also required to leverage resources by committing funding from multiple sources. Implementation grantees were required to collect and use data to track performance. Some planning grantees used a leading collaborative strategy not required by Education that produced early benefits. For example, several grantees and partners told us they completed easily achievable projects during the planning year to help build momentum and trust. Grantees told us that collaboration yielded benefits, including deeper relationships with partners, such as schools, as well as the ability to attract additional funding. However, grantees also said they faced some challenges collaborating with partners, particularly in overcoming privacy concerns related to data collection.
Why GAO Did This Study
Education's Promise Neighborhoods program is a competitive grant program with goals to improve educational and developmental outcomes for children in distressed neighborhoods. The grants fund community-based organizations' efforts to work with local partners to develop and evaluate a cradle-to-career continuum of services in a designated geographic footprint. As it is one of several federal programs using this model GAO was asked to review the program.
This report examines: (1) the extent to which Education's strategy for awarding grants aligns with program goals; (2) how Education aligns Promise Neighborhoods efforts with other related programs; (3) how Education evaluates grantees' efforts; and (4) the extent to which grants have enabled collaboration at the local level, and the results of such collaboration.
GAO reviewed Federal Register notices, applications, and guidance; surveyed all 48 grantees on the application process, coordination of resources, collaboration, and early results; visited 11 grantees selected based on geography and grant type; and interviewed Education officials and technical assistance providers.
What GAO Recommends
GAO recommends that Education communicate grant expectations more clearly, identify federal resources that can contribute to the program's goals, and develop a strategy for evaluation. In commenting on a draft of this report, Education outlined the steps it will take to respond to recommendations.
For more information, contact Jacqueline M. Nowicki at (617) 788-0580 or firstname.lastname@example.org.
Recommendations for Executive Action
Comments: Education stated that it would communicate to planning grant applicants that implementation funding is contingent on the availability of funds and that it would clarify to grantees that planning grant funds could be used to achieve early, tangible benefits. However, Education has not awarded any new planning grants since 2012. In FY17, ED reported that if new funding becomes available for the Promise Neighborhoods' planning and implementation awards, the Department will emphasize to all interested applicants that grant awards are contingent on the availability of funds and the results of the competitive award process. Education also stated that it would provide more targeted technical assistance to planning grant recipients regarding strategies for continuing grantees' efforts absent implementation funding. In 2015, its technical assistance provider published information on planning for growth and sustainability of Promise Neighborhoods.
Recommendation: In order to improve grantees' planning and implementation efforts, increase the effectiveness of grantee efforts to integrate and manage resources, and learn more about the program's impact, the Secretary of Education should clarify program guidance about planning and implementation grants to provide reasonable assurance that planning grantees are better prepared to continue their efforts in the absence of implementation funding. Additional guidance could include encouraging grantees to set aside a small amount of the grant to identify and deliver early, tangible benefits to their neighborhoods.
Agency Affected: Department of Education
Comments: Education stated that it would work with its technical assistance providers to create a mechanism to distribute a comprehensive list of external funding opportunities, programs and resources on a regular basis to better support the grantees' implementation efforts. In FY15, ED reported that the program office held a grantee meeting in July 2015 featuring at least three workshops on sustainability and leveraging additional funding sources. The program office also had a website (promiseneighborhoods.ed.gov) with a number of resources under the "toolbox" tab that can assist interested programs in financing their ongoing needs. While the workshops and web resources were good first steps that can assist grantees, GAO maintains that Education, rather than individual grantees, is best positioned to develop and share such an inventory of federal programs that relate to the goals of the Promise Neighborhoods program. Without such an inventory, Education may be missing opportunities to better support grantees; find other federal program for future coordination efforts; and identify potential fragmentation, overlap and duplication at the federal level. In FY17, Education did not provide any updates on this recommendation, nor has it provided such an inventory.
Recommendation: In order to improve grantees' planning and implementation efforts, increase the effectiveness of grantee efforts to integrate and manage resources, and learn more about the program's impact, the Secretary of Education should develop and disseminate to grantees on an ongoing basis an inventory of federal programs and resources that can contribute to the Promise Neighborhoods program's goal to better support coordination across agency lines.
Agency Affected: Department of Education
Comments: Education stated that it would consider options for how and whether it can use the data collected from grantees to conduct a national evaluation. As a first step, Education said it would conduct a systematic evaluation of the reliability and validity of the data. In its 2016 Notice of Funding Availability for Implementation Grants, Education acknowledged that grantees have struggled to collect the full range of data necessary to conduct meaningful evaluation activities and emphasized the importance of helping grantees develop robust data systems. In addition, in its agency comments, Education had stated that it had not received sufficient funding to support a national evaluation. In FY2017, ED provided documentation of its request for funding for conducting an evaluation and the response to the request. However, the entity within ED that is responsible for impact evaluations maintains that it has no plans to conduct an impact evaluation, given that grantees were not randomly selected. GAO agrees that the program was not designed for impact evaluation, however, as we reported, there are other options for evaluating such programs that can provide meaningful information about how well grantees are addressing the problem of poor student outcomes in impoverished neighborhoods. Not evaluating the program limits Education and other agencies from learning about the extent to which model is effective and should be replicated. Developing an evaluation plan would provide critical information about the resources required to conduct an evaluation, and could better inform future funding requests for such an evaluation.
Recommendation: In order to improve grantees' planning and implementation efforts, increase the effectiveness of grantee efforts to integrate and manage resources, and learn more about the program's impact, the Secretary of Education should develop a plan to use the data collected from grantees to conduct a national evaluation of the program.
Agency Affected: Department of Education