Promise Neighborhoods Promotes Collaboration but Needs National Evaluation Plan
GAO-14-432: Published: May 5, 2014. Publicly Released: Jun 4, 2014.
What GAO Found
The Department of Education (Education) used a two-phase strategy for awarding Promise Neighborhoods (Promise) grants, and aligned grant activities with program goals. Education awarded 1-year planning grants to organizations with the potential to effectively align services for students in their respective neighborhoods. Planning grants were generally intended to enhance the grantees' capacity to plan a continuum of services. Through a separate competition, Education awarded 5-year implementation grants to organizations that demonstrated they were most ready to implement their plans. However, Education did not communicate clearly to grantees about its expectations for the planning grants and the likelihood of receiving implementation grants. As a result, some grantees experienced challenges sustaining momentum in the absence or delay of implementation grant funding.
The Promise program coordinates with related federal efforts primarily through a White House initiative that brings together neighborhood grant programs at five federal agencies. The Promise program's efforts are focused on ensuring that grants are mutually reinforcing by aligning goals, developing common performance measures, and sharing technical assistance resources. While Promise grantees incorporate a wide range of federal programs in their local strategies, Education coordinates with a more limited number of federal programs. Officials told us that they do this to avoid spreading program resources too thin. Further, Education did not develop an inventory of the federal programs that share Promise goals, a practice that could assist grantees; help officials make decisions about interagency coordination; and identify potential fragmentation, overlap, and duplication.
Education requires Promise grantees to develop information systems and collect extensive data, but it has not developed plans to evaluate the program. Specifically, implementation grantees must collect data on individuals they serve, services they provide, and related outcomes and report annually on multiple indicators. However, Education stated it must conduct a systematic examination of the reliability and validity of the data to determine whether it will be able to use the data for an evaluation. Absent an evaluation, Education cannot determine the viability and effectiveness of the Promise program's approach.
The Promise grant enabled grantees and their partners to collaborate in ways that align with leading practices GAO previously identified for enhancing collaboration among interagency groups including establishing common outcomes, leveraging resources, and tracking performance. For example, Education required grantees to work with partners to develop common goals and a plan to use existing and new resources to meet identified needs in target areas. Grantees were also required to leverage resources by committing funding from multiple sources. Implementation grantees were required to collect and use data to track performance. Some planning grantees used a leading collaborative strategy not required by Education that produced early benefits. For example, several grantees and partners told us they completed easily achievable projects during the planning year to help build momentum and trust. Grantees told us that collaboration yielded benefits, including deeper relationships with partners, such as schools, as well as the ability to attract additional funding. However, grantees also said they faced some challenges collaborating with partners, particularly in overcoming privacy concerns related to data collection.
Why GAO Did This Study
Education's Promise Neighborhoods program is a competitive grant program with goals to improve educational and developmental outcomes for children in distressed neighborhoods. The grants fund community-based organizations' efforts to work with local partners to develop and evaluate a cradle-to-career continuum of services in a designated geographic footprint. As it is one of several federal programs using this model GAO was asked to review the program.
This report examines: (1) the extent to which Education's strategy for awarding grants aligns with program goals; (2) how Education aligns Promise Neighborhoods efforts with other related programs; (3) how Education evaluates grantees' efforts; and (4) the extent to which grants have enabled collaboration at the local level, and the results of such collaboration.
GAO reviewed Federal Register notices, applications, and guidance; surveyed all 48 grantees on the application process, coordination of resources, collaboration, and early results; visited 11 grantees selected based on geography and grant type; and interviewed Education officials and technical assistance providers.
What GAO Recommends
GAO recommends that Education communicate grant expectations more clearly, identify federal resources that can contribute to the program's goals, and develop a strategy for evaluation. In commenting on a draft of this report, Education outlined the steps it will take to respond to recommendations.
For more information, contact Jacqueline M. Nowicki at (617) 788-0580 or firstname.lastname@example.org.
Recommendations for Executive Action
Comments: Education stated that it would continue to communicate to planning grant applicants that implementation funding is contingent on the availability of funds, and that it would provide more targeted technical assistance to planning grant recipients regarding strategies for continuing grantees' efforts absent implementation funding. Education also stated that it would clarify to grantees that planning grant funds could be used to achieve early, tangible benefits. In FY15, ED reported that it is waiting on the final appropriation to determine if the program will have available funds. In the event there are funds available, PN will rulemake in FY16 to improve the program. In addition, the program will issue guidance to potential applicants. Currently, the old guidance is posted on their website. GAO notes that additional funds were recently appropriated. We will await Education's action with respect to guidance.
Recommendation: In order to improve grantees' planning and implementation efforts, increase the effectiveness of grantee efforts to integrate and manage resources, and learn more about the program's impact, the Secretary of Education should clarify program guidance about planning and implementation grants to provide reasonable assurance that planning grantees are better prepared to continue their efforts in the absence of implementation funding. Additional guidance could include encouraging grantees to set aside a small amount of the grant to identify and deliver early, tangible benefits to their neighborhoods.
Agency Affected: Department of Education
Comments: Education stated that it would work with its technical assistance providers to create a mechanism to distribute a comprehensive list of external funding opportunities, programs and resources on a regular basis to better support the grantees' implementation efforts. In FY15, ED reported that the program office held a grantee meeting in July 2015 featuring at least three workshops on sustainability and leveraging additional funding sources. The program office also has a website (promiseneighborhoods.ed.gov) with a number of resources under the "toolbox" tab that can assist interested programs in financing their ongoing needs. While the workshops and web resources are good first steps that can assist grantees, GAO maintains that Education, rather than individual grantees, is best positioned to develop and share such an inventory of federal programs that relate to the goals of the Promise Neighborhoods program. Without such an inventory, Education may be missing opportunities to better support grantees; find other federal program for future coordination efforts; and identify potential fragmentation, overlap and duplication at the federal level.
Recommendation: In order to improve grantees' planning and implementation efforts, increase the effectiveness of grantee efforts to integrate and manage resources, and learn more about the program's impact, the Secretary of Education should develop and disseminate to grantees on an ongoing basis an inventory of federal programs and resources that can contribute to the Promise Neighborhoods program's goal to better support coordination across agency lines.
Agency Affected: Department of Education
Comments: Education stated that it would consider options for how and whether it can use the data collected from grantees to conduct a national evaluation. As a first step, Education will conduct a systematic evaluation of the reliability and validity of the data. In addition, Education stated that to date, it has not received sufficient funding to support a national evaluation. GAO agrees that conducting evaluations can be costly, however, absent an evaluation, it will be difficult to determine whether the agency is successfully addressing the problem of poor student outcomes in impoverished neighborhoods. Developing an evaluation plan would also provide critical information about the resources required to conduct an evaluation, and could better inform future funding requests for such an evaluation. ED reported that in September 2015, the program office collected a sampling of the latest data and is waiting for IES to review it for validity and reliability. The hope is for IES to complete their review of the information by late-November. We recognize that Education has collected data for several years, however, it is unclear whether the program office has developed any plans for conducting an evaluation using these data.
Recommendation: In order to improve grantees' planning and implementation efforts, increase the effectiveness of grantee efforts to integrate and manage resources, and learn more about the program's impact, the Secretary of Education should develop a plan to use the data collected from grantees to conduct a national evaluation of the program.
Agency Affected: Department of Education