This is the accessible text file for GAO report number GAO-14-432 
entitled 'Education Grants: Promise Neighborhoods Promotes 
Collaboration but Needs National Evaluation Plan' which was released 
on June 4, 2014. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to the Chairman, Committee on Education and the Workforce, 
House of Representatives: 

May 2014: 

Education Grants: 

Promise Neighborhoods Promotes Collaboration but Needs National 
Evaluation Plan: 

GAO-14-432: 

GAO Highlights: 

Highlights of GAO-14-432, a report to the Chairman, Committee on 
Education and the Workforce, House of Representatives. 

Why GAO Did This Study: 

Education's Promise Neighborhoods program is a competitive grant 
program with goals to improve educational and developmental outcomes 
for children in distressed neighborhoods. The grants fund community-
based organizations' efforts to work with local partners to develop 
and evaluate a cradle-to-career continuum of services in a designated 
geographic footprint. As it is one of several federal programs using 
this model GAO was asked to review the program. 

This report examines: (1) the extent to which Education's strategy for 
awarding grants aligns with program goals; (2) how Education aligns 
Promise Neighborhoods efforts with other related programs; (3) how 
Education evaluates grantees' efforts; and (4) the extent to which 
grants have enabled collaboration at the local level, and the results 
of such collaboration. 

GAO reviewed Federal Register notices, applications, and guidance; 
surveyed all 48 grantees on the application process, coordination of 
resources, collaboration, and early results; visited 11 grantees 
selected based on geography and grant type; and interviewed Education 
officials and technical assistance providers. 

What GAO Found: 

The Department of Education (Education) used a two-phase strategy for 
awarding Promise Neighborhoods (Promise) grants, and aligned grant 
activities with program goals. Education awarded 1-year planning 
grants to organizations with the potential to effectively align 
services for students in their respective neighborhoods. Planning 
grants were generally intended to enhance the grantees' capacity to 
plan a continuum of services. Through a separate competition, 
Education awarded 5-year implementation grants to organizations that 
demonstrated they were most ready to implement their plans. However, 
Education did not communicate clearly to grantees about its 
expectations for the planning grants and the likelihood of receiving 
implementation grants. As a result, some grantees experienced 
challenges sustaining momentum in the absence or delay of 
implementation grant funding. 

The Promise program coordinates with related federal efforts primarily 
through a White House initiative that brings together neighborhood 
grant programs at five federal agencies. The Promise program's efforts 
are focused on ensuring that grants are mutually reinforcing by 
aligning goals, developing common performance measures, and sharing 
technical assistance resources. While Promise grantees incorporate a 
wide range of federal programs in their local strategies, Education 
coordinates with a more limited number of federal programs. Officials 
told us that they do this to avoid spreading program resources too 
thin. Further, Education did not develop an inventory of the federal 
programs that share Promise goals, a practice that could assist 
grantees; help officials make decisions about interagency 
coordination; and identify potential fragmentation, overlap, and 
duplication. 

Education requires Promise grantees to develop information systems and 
collect extensive data, but it has not developed plans to evaluate the 
program. Specifically, implementation grantees must collect data on 
individuals they serve, services they provide, and related outcomes 
and report annually on multiple indicators. However, Education stated 
it must conduct a systematic examination of the reliability and 
validity of the data to determine whether it will be able to use the 
data for an evaluation. Absent an evaluation, Education cannot 
determine the viability and effectiveness of the Promise program's 
approach. 

The Promise grant enabled grantees and their partners to collaborate 
in ways that align with leading practices GAO previously identified 
for enhancing collaboration among interagency groups including 
establishing common outcomes, leveraging resources, and tracking 
performance. For example, Education required grantees to work with 
partners to develop common goals and a plan to use existing and new 
resources to meet identified needs in target areas. Grantees were also 
required to leverage resources by committing funding from multiple 
sources. Implementation grantees were required to collect and use data 
to track performance. Some planning grantees used a leading 
collaborative strategy not required by Education that produced early 
benefits. For example, several grantees and partners told us they 
completed easily achievable projects during the planning year to help 
build momentum and trust. Grantees told us that collaboration yielded 
benefits, including deeper relationships with partners, such as 
schools, as well as the ability to attract additional funding. 
However, grantees also said they faced some challenges collaborating 
with partners, particularly in overcoming privacy concerns related to 
data collection. 

What GAO Recommends: 

GAO recommends that Education communicate grant expectations more 
clearly, identify federal resources that can contribute to the 
program's goals, and develop a strategy for evaluation. In commenting 
on a draft of this report, Education outlined the steps it will take 
to respond to recommendations. 

View [hyperlink, http://www.gao.gov/products/GAO-14-432]. For more 
information, contact Jacqueline M. Nowicki at 617-788-0580 or 
nowickij@gao.gov. 

[End of section] 

Contents: 

Letter: 

Scope and Methodology: 

Background: 

Education's Grant Strategy Aligns with Promise Goals, but Education 
Has Not Clearly Communicated Expectations for Planning Grants: 

Promise Neighborhoods Program Coordinates with Some Programs with 
Related Goals but Has No Inventory of Related Programs: 

Education Collects Extensive Data but Has Not Determined Whether or 
How to Evaluate the Program: 

Grantees Collaborate Using Leading Practices, Resulting in Benefits 
and Challenges: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Promise Neighborhood Grantees: 

Appendix III: Goals, Results, and Performance Indicators for Promise 
Neighborhood Program: 

Appendix IV: Comments from the Department of Education: 

Appendix V: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Promise Neighborhoods Planning and Implementation Grants 
(2010-2012): 

Table 2: Desired Results of the Promise Neighborhoods Grants: 

Table 3: Successful Implementation Grant Applications from Planning 
Grantees: 

Table 4: Examples of Promise Neighborhoods Program Coordination within 
Education and with NRI: 

Table 5: Federal Resources Leveraged by Promise Neighborhoods 
Implementation Grantees: 

Table 6: Approaches for Establishing Common Outcomes: 

Table 7: Approaches for Identifying and Addressing Needs by Leveraging 
Resources: 

Table 8: Approaches for Tracking Performance and Maintaining 
Accountability: 

Table 9: Selected Promise Neighborhoods Collaborative Efforts and 
Outcomes: 

Table 10: Promise Neighborhoods Program's Design for Achieving Its 
Goals: 

Table 11: Promise Neighborhoods Desired Outcomes and Indicators: 

Figures: 

Figure 1: Sample Continuum of Cradle-to-Career Services: 

Figure 2: Locations of Planning and Implementation Grantees: 

Figure 3: Applicants' Views of the Clarity of Implementation Grant 
Application Requirements and Peer Reviewer Scores and Comments: 

Figure 4: Applicants' Views of Helpfulness of Peer Reviewer Comments: 

Figure 5: Key Efforts Contributing to the White House Neighborhood 
Revitalization Initiative: 

Figure 6: Resources Leveraged by Promise Neighborhoods Grantees: 

Figure 7: Number of Implementation Grantees Who Reported They 
Collaborate With Their Partners to Track Performance and Maintain 
Accountability: 

Abbreviations: 

Education: Department of Education: 

HHS: Department of Health and Human Services: 

HUD: Department of Housing and Urban Development: 

Justice: Department of Justice: 

NRI: White House Neighborhood Revitalization Initiative: 

Promise: Promise Neighborhoods: 

Treasury: Department of the Treasury: 

[End of section] 

United States Government Accountability Office: 
GAO:
441 G St. N.W. 
Washington, DC 20548: 

May 5, 2014: 

The Honorable John Kline: 
Chairman: 
Committee on Education and the Workforce: 
House of Representatives: 

Dear Mr. Chairman: 

The Department of Education's (Education) Promise Neighborhoods 
(Promise) program aims to improve educational and developmental 
outcomes for students in some of the country's most distressed urban, 
rural, and tribal neighborhoods by aligning a suite of cradle-to-
career services in a designated geographic footprint. Since 2010, 
Education has competitively awarded Promise planning and 
implementation grants to 48 community-based organizations, including 
nonprofits, institutions of higher education, and Indian tribes that 
work in partnership with several other organizations, such as schools 
and social service agencies. Promise Neighborhoods grants are one of 
several place-based strategies under the White House Neighborhood 
Revitalization Initiative (NRI), which was also launched in 2010. 
Place-based strategies focus on a specific geographic location and 
provide flexibility to address the area's unique needs. 

The purpose of the grants is to plan for, create, and evaluate a 
continuum of services for children and youth residing in a target 
neighborhood. The Promise program provides both planning and 
implementation grants. In fiscal years 2010 through 2012, Education 
awarded a total of 46 1-year planning grants of up to $500,000. In 
fiscal years 2011 and 2012, Education awarded a total of 12 5-year 
implementation grants of up to $6 million per year. Almost all 
implementation grantees had received planning grants. No new grants 
were awarded in fiscal year 2013, and none have been awarded in fiscal 
year 2014, although Education has continued to fund prior-year 
implementation grants. 

The Promise Neighborhood program was established in fiscal year 2010, 
with $10 million from Education's discretionary Fund for the 
Improvement of Education.[Footnote 1] Since 2011, the program has 
received an appropriation in the annual appropriations acts, in 
amounts ranging from approximately $30 million to $60 million. For 
fiscal year 2015, the President has requested $100 million for the 
program and an additional $200 million under the President's 
Opportunity, Growth, and Security Initiative.[Footnote 2] 

In light of the President's requests for increased appropriations and 
specific statutory authority for the Promise Neighborhoods program and 
the increasing use of place-based strategies, you asked us to examine 
the Promise Neighborhoods program. To do this, we examined (1) the 
extent to which Education's strategy for awarding grants aligns with 
program goals, if at all; (2) how Education aligns Promise 
Neighborhoods efforts with other related programs; (3) how Education 
evaluates grantees' efforts; and (4) the extent to which Promise 
Neighborhoods grants have enabled collaboration at the local level, if 
at all, and the results of such collaboration. 

Scope and Methodology: 

To determine the extent to which the structure of the Promise 
Neighborhoods program aligns with program goals and how Education 
selected grantees, we reviewed relevant Federal Register notices, 
application guidance, and agency information on applicants for fiscal 
year 2011 and 2012 implementation grants.[Footnote 3] To determine how 
Education aligns Promise grant activities with other federal programs, 
we reviewed documentation on Education's alignment efforts. To assess 
Education's approach to evaluating the program, we reviewed its grant 
monitoring reports, performance measures, and guidance for data 
collection. To determine the extent to which Promise grants enabled 
collaboration at the local level, we used GAO's prior work on 
enhancing collaboration in interagency groups as criteria.[Footnote 4] 
We compared the Promise grants' collaboration approaches to certain 
successful approaches used by select interagency groups and reviewed 
implementation grantees' application materials. To learn about 
grantees' experiences with the program, we conducted a web-based 
survey of all planning and implementation grantees nationwide from 
late August to early November 2013. We received responses from all 48 
grantees. We asked grantees to provide information on the application 
and peer review process, coordination of federal resources, 
collaboration with local organizations, and results of the planning 
grants. Because not all respondents answered every question, the 
number of grantees responding to any particular question will be noted 
throughout the report. In addition, we conducted site visits to 11 
planning and implementation grantees. During these visits, we 
interviewed five planning grantees and six implementation grantees. 
Sites were selected based on several factors, such as the type of 
grant awarded, the location of grantees, and whether they were urban 
or rural. For all four objectives, we interviewed Education officials, 
technical assistance providers, and subject matter specialists from 
the Promise Neighborhoods Institute.[Footnote 5](See appendix I for 
more detail on the scope and methodology.) 

We conducted this performance audit from February 2013 to May 2014 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

Background: 

The Promise Neighborhoods program is a place-based program that 
attempts to address the problems of children and youth in a designated 
geographic footprint. The program is designed to identify and address 
the needs of children in low-performing schools in areas of 
concentrated poverty by aligning a cradle-to-career continuum of 
services. The program moves beyond a focus on low-performing schools 
by recognizing the role an entire community plays in a child's 
education (see figure 1). Place-based initiatives provide communities 
the flexibility to address their unique needs and interrelated 
problems by taking into account the unique circumstances, challenges, 
and resources in that particular geographic area. 

Figure 1: Sample Continuum of Cradle-to-Career Services: 

[Refer to PDF for image: illustration] 

Promise Neighborhood residents: 

Promise Neighborhood grantee helps coordinate and enhance wrap-around 
educational services and family and community supports. 

Support such as: health, safety, family engagement, technology and 
services to increase family stability, including housing, employment, 
transportation and social service referrals. 

Coaches, connectors, or advisors help families navigate services and 
transitions. 

Early learning: 
Services may include: Prenatal care, Parent education, Early learning 
network, Home visits, Head Start, Preschool. 

K-12 education: 
Services may include: Teacher training, Mentoring, After-school and 
summer programs, Early warning systems, Recreation, Academic case 
management, Academic enrichment, Dropout and gang prevention. 

College and career: 
Services may include: Internships, Dual enrollment, College 
preparation, Career exploration, Financial aid counseling. 

Source: GAO analysis of applications, grantee depictions, and program 
documentation. 

[End of figure] 

The Promise program is one of several place-based initiatives at the 
federal level, but it is the only one focused on educational issues. 
In addition to Education, the Departments of Justice (Justice), 
Housing and Urban Development (HUD), and Health and Human Services 
(HHS) also have grant programs aimed at impoverished neighborhoods. 
Together, these four agencies and their grant programs form the core 
of the White House Neighborhood Revitalization Initiative. This 
initiative coordinates neighborhood grant programs at the federal 
level across agencies, and identifies and shares best practices. Each 
agency's grant program focuses on its respective agency's core 
mission, but together, they focus on key components of neighborhood 
revitalization, education, housing, crime prevention, and healthcare. 

Promise Neighborhoods Model: 

Generally, the purpose of the Promise grants is to fund individual 
grantees' efforts to plan for and create a cradle-to-career pipeline 
of services based on the specific needs of their communities. The 
grants are focused on improving student outcomes on 15 performance 
indicators, chosen by Education. Along with the grantee, partner 
organizations, funded by federal, state, local, private, or nonprofit 
organizations, are expected to collaborate to provide matching funds 
and services. 

A number of nonprofits and foundations have worked on initiatives to 
address complex problems in a similarly comprehensive way. Their 
approach brings together a group of stakeholders from different 
sectors to collaborate on a common agenda, align their efforts, and 
use common measures of success. This approach has been described as 
the collective impact model.[Footnote 6] The premise of the model is 
that better cross-sector alignment and collaboration creates change 
more effectively than isolated interventions by individual 
organizations. A number of organizations have used this approach to 
address issues such as childhood obesity and water pollution. 

Several other cradle-to-career place-based collective impact programs 
share key characteristics with the Promise program, including 
Cincinnati's Strive program and the Harlem Children's Zone. [Footnote 
7] [Footnote 8] These collective impact initiatives use a centralized 
infrastructure and a structured process, including training, tools, 
and resources, intended to result in a common agenda, shared 
measurement, and mutually-reinforcing activities among all 
participants. This centralized infrastructure requires staff to manage 
technology, communications support, data collection, reporting, and 
administrative details. The Promise grantees' role is to create and 
provide this centralized infrastructure for their communities. 

Types of Grants: 

The Promise program relies on a two-phase strategy for awarding 
grants, which includes both one-year planning grants and three-to five-
year implementation grants.[Footnote 9] (See table 1.) 

Table 1: Promise Neighborhoods Planning and Implementation Grants 
(2010-2012): 

Description: 
Planning grant: Supports a comprehensive needs assessment, a 
prioritization process to identity areas where interventions could be 
most successful, and design of a longitudinal data system to track 
results and student outcomes for children in the neighborhood; 
Implementation grant: Supports implementation of strategies developed 
in planning phase. Funds can go toward a wide range of coordination 
and outreach efforts, building and operating longitudinal data 
systems, and some direct services. 

Length and amount: 
Planning grant: 1-year grants; up to $500,000 each; 
Implementation grant: 5-year grants; up to $6 million per year. 

Number awarded: 
Planning grant: 46; 
Implementation grant: 12. 

Source: GAO analysis of relevant Federal Register notices and 
documents from Education. 

[End of table] 

Among other things, planning grantees are required to conduct a 
comprehensive needs assessment of children and youth in the 
neighborhood and develop a plan to deliver a continuum of solutions 
with the potential to achieve results.[Footnote 10] This effort 
involves building community support for and involvement in developing 
the plan. Planning grantees are also expected to establish effective 
partnerships with organizations for purposes such as providing 
solutions along the continuum and obtaining resources to sustain and 
scale up the activities that work. Finally, planning grantees are 
required to plan to build, adapt, or expand a longitudinal data system 
to provide information and use data for learning, continuous 
improvement, and accountability. 

The implementation grant provides funds to develop the administrative 
capacity to implement the planned continuum of services. Education 
expects implementation grantees to build and strengthen the 
partnerships they developed to provide and sustain services and to 
continue to build their longitudinal data systems. 

Education awarded most of the 2010-2012 grants to non-profit 
organizations (38 of 48), eight to institutions of higher education, 
and two to tribal organizations. Almost all (10 of 12) implementation 
grantees received planning grants, while two did not. (See figure 2 
for locations of grantees.) (See appendix II for a list of grantees 
and year of grant award.) 

Figure 2: Locations of Planning and Implementation Grantees: 

[Refer to PDF for image: illustrated U.S. map] 

Grant type: Planning only: 
Athens, Georgia; 
Atlanta, Georgia; 
Adams County, Wisconsin; 
Baltimore, Maryland; 
Camden, New Jersey; 
Campo, California; 
Charlottesville, Virginia; 
Chicago, Illinois; 
Corning, California; 
Detroit, Michigan; 
Fresno, California; 
Glouster, Ohio; 
Houston, Texas; 
Hudson, New York; 
Langley Park, Maryland; 
Lawrence, Massachusetts; 
Little Rock, Arkansas; 
Los Angeles, California; 
Macon, Georgia; 
Marshalltown, Iowa; 
Meriden, Connecticut; 
Nashville, Tennessee; 
Newark,New Jersey; 
New York City, New York (5); 
Northern Cheyenne Reservation, Montana; 
Ogden, Utah; 
Philadelphia, Pennsylvania; 
River Rouge, Michigan; 
St. Paul, Minnesota; 
Thomaston, Maine; 
Tulsa, Oklahoma; 
Worcester, Massachusetts. 

Grant type: Implementation only: 
Lubbock, Texas; 
Minneapolis, Minnesota. 

Grant type: Planning and implementation: 
Boston, Massachusetts; 
Buffalo, New York; 
Hayward, California; 
Indianola, Mississippi; 
Los Angeles, California; 
Multiple Counties, Kentucky; 
San Antonia, Texas; 
San Diego, California; 
San Francisco, California; 
Washington, D.C. 

Source: GAO analysis of grant award announcements; National Atlas of 
the United States (base map). 

[End of figure] 

Education's Grant Strategy Aligns with Promise Goals, but Education 
Has Not Clearly Communicated Expectations for Planning Grants: 

Grant Requirements Enabled Education to Identify Organizations with 
Potential to Coordinate Student Services: 

The planning and implementation grant activities that Education 
developed for the Promise program generally align with Education's 
goal of significantly improving the educational and developmental 
outcomes of children and youth in the nation's most distressed 
communities. According to Education officials, the planning grant 
award process enabled them to identify community-based organizations 
in distressed neighborhoods with the potential to effectively 
coordinate the continuum of services for students living in the 
neighborhood. The eligibility requirements, which included matching 
funds or in-kind donations and an established relationship with the 
community to be served, helped to ensure that grantees had financial 
and organizational capacity and were representative of the area to be 
served. Education developed criteria to evaluate applications and 
select grantees based on the grantees' ability to describe the need 
for the project; the quality of the project design, including the 
ability to leverage existing resources; the quality of the project 
services; and the quality of the management plan. 

Education's Promise planning grants were intended to enhance the 
capacity of identified organizations to create the cradle-to-career 
continuum. The activities required of planning grantees enable 
grantees and their partners to gain a depth of knowledge about their 
communities and the communities' needs, which can increase their 
capacity to focus on improving educational and developmental outcomes 
for children and youth throughout their neighborhood. 

Through a separate competition, Education identified organizations 
that application reviewers determined were most ready to implement 
their plans. While acknowledging that the implementation grantees are 
best positioned to determine the allocation of grant funds, Education 
expects that grant funds will be used to develop the administrative 
capacity to implement the planned continuum and that the majority of 
resources to provide services to students and families will come from 
other public and private funding sources rather than from the grant 
itself. This expectation gives the Promise strategies a chance to 
extend beyond the 5-year life of the grant. Further, the requirement 
that grantees build a longitudinal data set allows Promise grantees 
and their partners to review and analyze robust data in real time to 
make informed decisions about whether to adjust their strategies. The 
data can also help the grantees and Education learn about the impact 
of the program. 

Education identified 10 desired results from implementation of the 
program, which cover the cradle-to-career age span that Promise 
Neighborhoods are expected to address. A technical assistance provider 
stated that the list of desired results help grantees focus on 
improving educational and developmental outcomes across the entire 
continuum. (See table 2.) (The indicators that measure progress toward 
achieving results are listed in Appendix III.) 

Table 2: Desired Results of the Promise Neighborhoods Grants: 

Focus: Academic; 
Result: 
* Children enter kindergarten ready to succeed in school; 
* Students are proficient in core academic subjects; 
* Students successfully transition from middle school grades to high 
school; 
* Youth graduate from high school; 
* High school graduates obtain a postsecondary degree, certification, 
or credential. 

Focus: Family and community supports; 
Result: 
* Students are healthy; 
* Students feel safe at school and in their community; 
* Students live in stable communities; 
* Families and community members support learning in Promise 
Neighborhood schools; 
* Students have access to 21st century learning tools. 

Source: Relevant Federal Register notices. 

[End of table] 

Education's Two-Phase Grant Strategy Created Challenges for Some 
Grantees: 

Education's grantee selection process was generally clear and 
transparent. However, Education did not communicate clearly to 
planning grantees about the probability of receiving an implementation 
grant and its expectations for grantees to continue their efforts 
without implementation funding. This lack of clarity created 
challenges for some grantees. 

Education's Selection Process: 

Education outlined its selection criteria and how grant applications 
would be scored in its grant announcements and selected peer reviewers 
from outside the organization. According to Education officials, the 
peer reviewers had expertise in various related fields, including 
community development and all levels of education. Education provided 
additional training on the application review process. For the 
planning grant selection, Education divided about 100 peer reviewers 
into panels of three to review packages of about 10 applications. 
Afterward, peer reviewers conferred about scores in a conference call. 

For the first implementation grant selection, Education had a two-
tiered peer review process. During the first tier, peer reviewers were 
divided into panels of three to review approximately seven 
applications. During the second tier review of the 16 highest scoring 
applications, panels of reviewers were adjusted so that different 
reviewers read and scored different applications. For the second 
implementation grant selection, there was only one round of reviews. 
Reviewers were asked to review the applications and submit comments 
before meeting on-site to discuss applications. Education posted the 
results online, including peer reviewer comments for grantees and a 
list of applicants with scores above 80 out of 100 points. 

In our web-based survey of grantees, grantees had mixed views on the 
clarity of application requirements and the helpfulness of peer 
reviewer comments. Specifically, 13 of 18 planning grantees who 
applied unsuccessfully for implementation grants and responded to the 
relevant survey question said the application requirements were very 
clear or extremely clear, while 8 of 19 grantees that responded said 
the same about peer reviewer scores and comments (see figure 3). The 
unsuccessful applicants gave somewhat lower marks to the helpfulness 
of peer reviewer comments in improving their future applications and 
strengthening their current strategies (see figure 4). 

Figure 3: Applicants' Views of the Clarity of Implementation Grant 
Application Requirements and Peer Reviewer Scores and Comments: 

[Refer to PDF for image: horizontal bar graph] 

Application requirements: 
Number of grantees responding: 
Not at all clear: 0; 
Somewhat or moderately clear: 5; 
Very or extremely clear: 13. 

Peer reviewer scores and comments: 
Number of grantees responding: 
Not at all clear: 2; 
Somewhat or moderately clear: 9; 
Very or extremely clear: 8. 

Source: GAO survey of Promise Neighborhoods grantees. 

Note: Not all grantees responded to all questions. 

[End of figure] 

Figure 4: Applicants' Views of Helpfulness of Peer Reviewer Comments: 

[Refer to PDF for image: horizontal bar graph] 

Improve future applications: 
Number of grantees responding: 
Not at all helpful: 1; 
Somewhat or moderately helpful: 11; 
Very or extremely helpful: 7. 

Strengthen/refine existing Promise Neighborhood strategies and efforts: 
Number of grantees responding: 
Not at all helpful: 2; 
Somewhat or moderately helpful: 11; 
Very or extremely helpful: 5. 

Source: GAO survey of Promise Neighborhoods grantees. 

Note: Not all grantees responded to all questions. 

[End of figure] 

Some of the 11 planning and implementation grantees that we 
interviewed raised concerns about specific application guidelines, 
such as how the term "neighborhood" is defined and the length of the 
application. Specifically, two rural grantees said that the grant 
application and materials had a few areas that seemed to be more 
geared to urban or suburban grantees. For example, the term 
"neighborhood" was somewhat difficult for them to interpret in a rural 
context. In fact, two rural grantees included multiple towns or 
counties in their neighborhood footprints. Additionally, two grantees 
we spoke with had concerns about the implementation grant 
application's 50-page recommended maximum for the project narrative. 
Both organizations limited their narratives to 50 pages, but said they 
later learned that most of the successful grant recipients had 
exceeded this limit, often by a large amount. 

The timing of the grant cycles created either an overlap or a long gap 
between the two grants. Grantees who applied for the implementation 
grant in the first cycle after receiving a planning grant had an 
overlap between executing the first grant and applying for the second 
grant. According to Education officials these grantees were unable to 
fully apply the knowledge gained in the planning year to develop their 
implementation applications. For example, one grantee said having to 
apply for the implementation grant during the planning year made it 
difficult to create opportunities for community input into the 
planning process. On the other hand, one of the four grantees that 
received an implementation grant 2 years after receiving a planning 
grant faced challenges sustaining the momentum of its efforts without 
additional funding. Another grantee in the same situation was able to 
sustain momentum with a separate grant from a private foundation. 
Education officials said they became aware of the problems with the 
timing of the implementation applications a few months into the first 
planning grant year. However, they said they did not have much 
flexibility in timing the grant cycles. For example, they said that 
they needed to allow time for public comment on the grant notification 
in the Federal Register. In addition, they said that agency budget 
decisions were delayed that year because the Department was operating 
under a continuing resolution for over 6 months in fiscal year 2011--
the first year implementation grants were awarded. 

Some grantees also said there was a disconnect between the planning 
and the implementation grant application processes. Specifically, two 
officials from the six implementation grantees we visited told us that 
a high-quality planning year was not nearly as important for obtaining 
an implementation grant as having someone who could write a high-
quality federal grant application. For example, one grantee noted that 
writing a good implementation grant application was not heavily 
dependent on information gleaned from the planning process. Another 
grantee said that the implementation grant application was written by 
a completely different person who was not involved in planning grant 
activities. 

Challenges with the Two-Phase Strategy: 

Some grantees who received only planning grants reported in our survey 
and in interviews that they experienced challenges continuing their 
work without implementation funds. In addition, two of the five 
planning grantees we interviewed had concerns with Education's 
strategy of awarding few implementation grants compared with the 
number of planning grants.[Footnote 11] 

Education informed grantees there was a possibility they would not 
receive an implementation grant following the planning grant, but no 
information was provided about the likelihood of whether this would 
occur. We found indications that grantees did not fully appreciate 
that receiving a planning grant would not necessarily result in 
receiving an implementation grant. Three of the five planning grantees 
we interviewed stated that they did not have contingency plans for 
continuing their Promise Neighborhood efforts in the event that they 
did not receive implementation funding.[Footnote 12] The lack of 
contingency planning raises questions about the grantees' 
understanding of the probability of receiving an implementation grant. 
Internal control standards state that management should ensure that 
effective external communications occur with groups that can have a 
serious impact on programs, projects, operations, and other 
activities, including budgeting and financing.[Footnote 13] To date, 
Education has awarded 46 planning grants (21, 15, and 10 in 2010, 
2011, and 2012, respectively) and 12 implementation grants. Even 
though all but two implementation grants were awarded to planning 
grantees, fewer than one-quarter of planning grantees received 
implementation funding.[Footnote 14] (See table 3.) 

Table 3: Successful Implementation Grant Applications from Planning 
Grantees: 

Year planning grant received: Number of recipients; 
2010: 21; 
2011: 15. 

Year planning grant received: Number that applied for 2011 
implementation grants; 
2010: 19; 
2011: n/a. 

Year planning grant received: Number awarded 2011 implementation 
grants; 
2010: 4; 
2011: n/a. 

Year planning grant received: Number that applied for 2012 
implementation grants; 
2010: 10; 
2011: 8. 

Year planning grant received: Number awarded a 2012 implementation 
grants; 
2010: 4; 
2011: 2. 

Source: GAO analysis of applicant lists from Education. 

Note: In both 2011 and 2012, there was also one successful 
implementation grant applicant that had not previously received a 
planning grant. 

[End of table] 

Education officials provided several reasons for separating the 
planning and implementation grants and for not awarding implementation 
grants to all planning grantees who applied. Officials said that when 
they awarded the first planning grants, they were not sure which 
neighborhoods had potential grantees with the capacity to implement a 
Promise plan. In their view, the planning grants allowed them to 
invest in the capacity of communities to take on this work, while the 
implementation grants were only awarded to those that demonstrated 
they were ready for implementation. Education officials said it was 
important that grantees demonstrate they have an implementation plan 
in place before receiving such a large sum of money. In addition, 
after the first round of implementation grants were awarded, they 
noted that some applicants did not receive implementation grants 
because they were not yet competitive--in part because they had 
applied for the implementation grants before their planning efforts 
were complete. Finally, in commenting on a draft of this report, 
Education officials said that in several years, Congress appropriated 
less funds than were requested, which, they said, affected the number 
of implementation grants Education awarded. 

In 2010, both Education's Federal Register Notice Inviting 
Applications for planning grants and a related frequently asked 
questions document informed organizations receiving planning grants 
that they should not necessarily plan on automatically receiving 
implementation grants.[Footnote 15] The frequently asked questions 
guidance noted that the two types of grants could stand alone. For 
example, an applicant could receive just a planning grant, consecutive 
planning and implementation grants, or--if the applicant was further 
along in the planning process--just an implementation grant. Education 
officials told us that they viewed the planning grant activities as 
useful in themselves. For example, they told us that the planning 
process offers rich data and begins the process of bringing together 
partners and breaking down silos. They expected that planning grantees 
that applied for but did not receive implementation funding could 
continue their efforts without implementation grant funding, using 
their partners' pledged matching funds to implement their plans on a 
smaller scale. They noted that the requirement to develop memoranda of 
understanding with partners should have signaled that the obligations 
of the partner organizations were not to be contingent upon receipt of 
an implementation grant. However, Education did not require grantees 
to have matching funds in-hand before submitting their applications. 
Especially in light of the difficult fiscal climate that federal 
agencies will likely continue to face in the future, we believe that 
it is important for Education to clearly communicate to grantees 
regarding expectations for planning and implementation grants. Clear 
communication and expectations can also help promote more realistic 
expectations among grantees about future funding opportunities given 
the fiscal realities of the Promise program over the past 5 years. 

Grantees who had not received implementation grants were trying to 
continue their efforts and most reported significant challenges in 
sustaining momentum. According to our survey, since the end of the 
planning grant, most planning grantees who did not receive an 
implementation grant (17 out of 29 that answered the related question) 
found it very or extremely challenging to maintain funding, 12 out of 
29 planning grantees felt that maintaining key leadership positions 
was very or extremely challenging, and 13 out of 29 planning grantees 
found that hiring staff was very or extremely challenging. 

Four of the five planning grantees we interviewed who had not received 
implementation grants told us that they need to determine how to 
implement scaled-down versions of programs and services identified in 
their implementation grant applications. They described challenges 
continuing their work without implementation funding. For example, 
three grantees noted that partners had pledged funding as a match for 
federal dollars in their implementation grant proposal. Without the 
leverage of implementation grant funds, it was difficult to maintain 
the proposed funding streams.[Footnote 16] All of the five grantees we 
interviewed that had received only planning grants said the planning 
process was very helpful in building connections and trust and 
deepening communication among partners, and between partners and the 
community. Four grantees were concerned, however, that the trust and 
momentum they had built might dissipate if they were not able to carry 
out their plans without an implementation grant. 

Promise Neighborhoods Program Coordinates with Some Programs with 
Related Goals but Has No Inventory of Related Programs: 

Promise Program Coordinates Closely with Some Related Federal Programs 
to Align Grants: 

In an effort to target its resources and align the Promise program 
goals with those of other place-based initiatives, the Promise program 
coordinates closely with a limited number of federal programs within 
Education and with other federal programs as part of the White House 
Neighborhood Revitalization Initiative (NRI).The NRI is an interagency 
coordinating body that aligns place-based programs run by HUD, HHS, 
Justice, and the Department of the Treasury (Treasury) (see figure 5). 
Coordination through NRI is more structured than internal coordination 
within Education, which, according to Promise program officials, 
occurs as needed. Liaisons from each grant program meet at biweekly 
and monthly NRI meetings. They have formed a program integration 
workgroup to coordinate program development, monitoring, and technical 
assistance for the grant programs included. For example, they 
conducted a joint monitoring trip to a neighborhood in San Antonio, 
Texas that has Promise, HUD's Choice Neighborhood, and Justice's Byrne 
Criminal Justice Innovation grants. 

Figure 5: Key Efforts Contributing to the White House Neighborhood 
Revitalization Initiative: 

[Refer to PDF for image: illustration] 

White House Neighborhood Revitalization Initiative: 

Department of Health and Human Services: 
Community Health Centers: Comprehensive preventive and primary health 
care to underserved urban and rural communities. 

Department of Education: 
Promise Neighborhoods: Supports projects to create a comprehensive 
continuum of education programs and family and community supports that 
improve the educational and developmental outcomes of children and 
youth from birth through college and career. 

Department of Justice: 
Byrne Criminal Justice Innovation: Fosters partnerships between law 
enforcement agencies and community-based organizations to control and 
prevent violent crime, drug abuse, and gang activity in designated 
high crime neighborhoods. Partnerships in the program balance targeted 
enforcement with prevention, intervention, and neighborhood 
restoration. 

Department of the Treasury: 
Community Development Financial Institutions (CDFI) Fund: Promotes 
economic revitalization in low-income communities. The fund provides 
funds to CDFIs for Financial Assistance and Technical Assistance to 
develop businesses, affordable housing, basic banking services, among 
other things. 

Department of Housing and Urban Development: 
Choice Neighborhoods: Supports transformation of distressed public and 
assisted housing into sustainable mixed-income housing. 

Source: GAO analysis of White House Neighborhood Revitalization 
Initiative and agency documents and interviews with participating 
agencies. 

Note: HHS' Community Health Centers have been in existence since the 
1960s. Treasury's Community Development Financial Institutions Fund 
was established in 1994 by the Riegle Community Development and 
Regulatory Improvement Act of 1994. HUD's Choice Neighborhood program 
was launched in 2010 to replace HUD's HOPE VI program. DOJ's Byrne 
Criminal Justice Innovation program made its first grants in 2012. 

[End of figure] 

In coordinating within Education and with NRI, Education's efforts are 
focused on ensuring that grants are mutually reinforcing. These 
coordination activities include aligning goals, developing common 
performance measures where there are common purposes, and sharing 
technical assistance resources in areas where programs address similar 
issues or fund similar activities. (See table 4.) 

Table 4: Examples of Promise Neighborhoods Program Coordination within 
Education and with NRI: 

Coordination activity: Align grant language or activities and 
establish competitive preference priority[A]; 
Example of coordination within the Department of Education: Education 
aligned the Promise program and the Race to the Top grant program to 
ensure grant activities were mutually reinforcing by including a 
competitive preference priority in the Race to the Top program.[B] 
Applicants could receive points under this priority for doing work 
similar to that funded by the Promise program, such as increasing the 
schools' core resources by providing additional student and family 
services. This allows districts to further some of the aims of the 
Promise program when carrying out Race to the Top activities, even if 
they did not have Promise grantees in their districts. Promise 
officials were also involved in the peer review process for selection 
of Race to the Top Early Learning Challenge grantees[C]; 
Example of coordination within NRI: Education and HUD have coordinated 
to establish competitive preference priorities. The Choice 
Neighborhoods planning grant application included a categorical 
preference for Promise planning grantees who meet Choice planning 
grant requirements. Similarly, the Promise Program includes a 
competitive preference priority for Choice grantees. There are some 
neighborhoods with both Choice and Promise grants, but, according to 
Education, most Promise grantees do not have distressed public housing 
in their geographic footprints. 

Coordination activity: Align goals and share performance measures; 
Example of coordination within the Department of Education: The 
Promise program coordinated with Education's Office of Safe and 
Healthy Students to align the student safety performance measure with 
one that was being used by the Safe Schools/Healthy Students 
program[D]; 
Example of coordination within NRI: To align goals and measures, the 
agencies use common language in grant applications and share 
performance indicators where there are mutual interests. For example, 
one of the indicators of need included in the Promise planning grant 
notice is high rates of vacant or substandard homes, including 
distressed public and assisted housing, which is aligned with the 
HUD's Choice grant program. 

Coordination activity: Share technical assistance resources; 
Example of coordination within the Department of Education: The 
Promise office hosted a joint webinar with School Improvement Grant 
recipients[E]; 
Example of coordination within NRI: The integration workgroup is 
exploring ways to share technical assistance with the agencies' 
respective grantees. For example, Education officials stated they are 
trying to find a way to use the technical assistance provided to Byrne 
grantees to help Promise grantees who have public safety concerns. In 
addition, Education has invited other NRI programs to annual grantee 
conferences to conduct workshops on integrating the grants. 

Source: GAO analysis based on interviews with Education officials, NRI 
documentation and grant announcements. 

[A] Under a competitive preference priority, preference is given to an 
application by (1) awarding additional points, depending on the extent 
to which the application meets the priority or (2) selecting an 
application that meets the priority over an application of comparable 
merit that does not meet the priority. 34 C.F.R. § 75.105(c)(2). For 
example, in the Race to the Top District program, an applicant 
received points under this priority based on the extent to which it 
integrated public and private resources to augment the schools' core 
resources by providing additional student and family supports, such as 
addressing the social-emotional, behavioral, and other needs of 
participating students giving highest priority to those in high-needs 
schools. 

[B] Education created Race to the Top under the American Recovery and 
Reinvestment Act of 2009 to provide incentives for states to reform K-
12 education in areas such as improving the lowest performing schools 
and developing effective teachers and leaders. 

[C] Race to the Top Early Learning Challenge grants are meant to 
support states' efforts to increase the number and percentage of low-
income and disadvantaged children enrolled in high-quality early 
learning programs and to design and implement an integrated system of 
high-quality early learning programs and services. The program is run 
jointly by Education and HHS. 

[D] The Office of Safe and Healthy Students' mission is to support the 
creation of safe, disciplined, drug-free and healthy learning 
environments for students. 

[E] The School Improvement Grants program funds reforms in the 
country's lowest-performing schools with the goal of improving student 
outcomes, such as standardized test scores and graduation rates. 

[End of table] 

The Promise program has also participated in another place-based 
program led out of the White House Domestic Policy Council: the Strong 
Cities, Strong Communities initiative.[Footnote 17] This program sends 
teams of federal officials to work with distressed cities, providing 
them expertise to more efficiently and effectively use the federal 
funds they already receive. Education's Promise program participates 
in initial on-site assessments of communities. Education staff 
assisted two of the participating communities by providing education 
expertise at their request. 

As of 2013, Education also began taking part in the Promise Zone 
initiative, a new program intended to accelerate and expand the work 
taking place under NRI grant programs such as Promise Neighborhoods. 
Unlike the Promise Neighborhoods program, which focuses primarily on 
the educational and developmental needs of children and youth, the 
Promise Zones program addresses economic security, jobs, education, 
affordable housing, and public safety in high-poverty areas usually 
larger than a neighborhood.[Footnote 18] The Promise Zone initiative, 
launched in 2013, is lead by HUD and takes an approach that is similar 
to that used by Strong Cities, Strong Communities in that it brings 
together expertise from multiple agencies. It is a place-based 
initiative that does not receive direct federal funding but uses 
existing resources from programs within the NRI agencies and five 
other agencies[Footnote 19] in partnership with state and local 
governments, businesses, and non-profit organizations.[Footnote 20] 
Only areas that already had certain NRI grants or a similar rural or 
tribal grant were eligible to apply in the first round. As of January 
2014, three Promise Neighborhoods implementation sites in San Antonio, 
Los Angeles, and Southeastern Kentucky were located in designated 
Promise Zones, which provide additional opportunity for coordination 
at the federal and local level. 

The Promise Neighborhoods program does, on occasion, coordinate with 
other individual federal agencies and programs outside of the NRI, but 
officials stated that the program is focused on deepening and 
broadening the communication it has with the five named NRI programs 
and Promise Zones. Promise Neighborhoods officials explained that they 
had concerns about spreading their coordination efforts too thinly 
given the large number of programs grantees may include in their 
strategies. 

Education Does Not Have an Inventory of Federal Programs with Similar 
Goals: 

In addition to Promise grants from Education, individual Promise 
Neighborhoods have access to a broad range of federal programs from 
other agencies, including many programs that are not part of NRI. 
However, Education has not developed an inventory of federal programs 
that could contribute to Promise program goals that it could share 
with planning and implementation grantees and use to make its own 
decisions about coordination across agencies. In recent work examining 
approaches used by interagency groups that successfully collaborated, 
we found that an inventory of resources related to reaching 
interagency goals can be used to promote an understanding of related 
governmentwide programs.[Footnote 21] Such inventories are useful in 
making decisions about coordinating related programs across agency 
lines and between levels of government, according to officials. We 
have also found that creating a comprehensive list of programs is a 
first step in identifying potential fragmentation, overlap, or 
duplication among federal programs or activities.[Footnote 22] 

As shown in table 5, the 12 implementation grantees we surveyed stated 
that they included a variety of federal resources in their Promise 
Neighborhoods strategies. AmeriCorps was included in 9 out of 11 
implementation grantees' strategies,[Footnote 23] followed by Head 
Start (8 of 12) and Education's School Improvement Grants (6 of 11). 
None of these are part of NRI. Few grantees said that NRI programs 
were part of their Promise strategies. For example, four grantees said 
that a Choice Neighborhood grant was part of their Promise strategy, 
and three grantees stated that DOJ's Byrne program was part of their 
strategy. Education officials attributed the small number of grantees 
that use HUD's Choice program to the fact that few grantees have 
distressed public housing within their footprint that is eligible for 
this funding. 

Table 5: Federal Resources Leveraged by Promise Neighborhoods 
Implementation Grantees: 

Agency: HUD; 

Federal grants/programs: Choice Neighborhood grants; [NRI program] 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 4. 

Federal grants/programs: Other HUD grants; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 4. 

Agency: DOJ; 

Federal grants/programs: Byrne grants; [NRI program] 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 3. 

Federal grants/programs: Other DOJ grants; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 2. 

Agency: HHS; 

Federal grants/programs: Community Health Center; [NRI program]
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 4. 

Federal grants/programs: Head Start; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 8. 

Federal grants/programs: Early Head Start; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 5. 

Federal grants/programs: Other HHS Grants; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 3. 

Agency: Education; 

Federal grants/programs: Race to the Top; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 3. 

Federal grants/programs: School Improvement Grants; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 6. 

Federal grants/programs: Investing in Innovation; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 3. 

Federal grants/programs: 21st Century Learning; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 4. 

Federal grants/programs: Full Service Community Schools; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 2. 

Federal grants/programs: Federal TRIO Programs; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 5. 

Federal grants/programs: GEAR UP; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 5. 

Federal grants/programs: Early Reading First; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 1. 

Federal grants/programs: Other Education grants; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 0. 

Agency: Corporation for National and Community Service; 

Federal grants/programs: AmeriCorps; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 9. 

Federal grants/programs: Social Innovation Fund; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 1. 

Agency: DOL; 

Federal grants/programs: YouthBuild; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 3. 

Agency: USDA; 

Federal grants/programs: Nutrition Programs; 
Number of grantees who said program was included in a Promise 
Neighborhood strategy: 3. 

Source: GAO analysis of survey of Promise Neighborhood grantees. 

[End of table] 

Although Promise grantees conduct their own inventories of the 
existing federal and other resources in their neighborhoods in order 
to develop their strategies, two grantees we spoke with were unaware 
of some of the other federal programs that could contribute towards 
their strategies. For example, one implementation grantee we spoke to 
with concerns about school safety was unaware of DOJ's Byrne Criminal 
Justice Innovation grant program. Another planning grantee who 
completed our survey commented that a list of related federal programs 
like the one in our survey would be especially useful to grantees who 
did not receive implementation grants. Education officials with the 
Promise program told us that sometimes grantees are unaware that the 
community is benefiting from certain federal programs because programs 
are renamed as they filter down through the state or local levels. 
Education officials said they emphasize to grantees the importance of 
reaching out to key partners to ensure they are aware of other 
federally funded programs in the neighborhood because their partners 
may be more knowledgeable about other sources of federal funding. 
While encouraging grantees to reach out to key partners is helpful, 
Education, through its coordination with other federal agencies, would 
likely have more knowledge about existing federal resources. 

Without a federal level inventory, Education is not well-positioned to 
support grantee efforts to identify other federal programs that could 
contribute to Promise program goals. Further, Education lacks complete 
information to inform decisions about future federal coordination 
efforts and identify potential fragmentation, overlap, and duplication. 

Education Collects Extensive Data but Has Not Determined Whether or 
How to Evaluate the Program: 

While Education is collecting a large amount of data from Promise 
grantees that was intended, in part, to be used to evaluate the 
program, the Education offices responsible for program evaluation--the 
Institute for Educational Sciences (IES) and Office of Planning, 
Evaluation, and Policy Development (OPEPD)--have not yet determined 
whether or how they will evaluate the program. 

One of Education's primary goals for the Promise program, as described 
in the Federal Register, is to learn about the overall impact of the 
program through a rigorous program evaluation. Applicants are required 
to describe their commitment to work with a national evaluator for 
Promise Neighborhoods to ensure that data collection and program 
design are consistent with plans to conduct a rigorous national 
evaluation of the program and the specific solutions and strategies 
pursued by individual grantees. We have found that federal program 
evaluation studies provide external accountability for the use of 
public resources. Evaluation can help to determine the "value added" 
of the expenditure of federal resources or to learn how to improve 
performance--or both. Evaluation can play a key role in strategic 
planning and in program management, informing both program design and 
execution.[Footnote 24] 

Education requires implementation grantees to report annually on their 
performance using 15 indicators. The indicators include graduation 
rates, attendance, academic proficiency, student mobility, physical 
activity, and perceptions of safety. (See table 11 in appendix III.) 
Education contracted with the Urban Institute to provide guidance on 
how to collect data on the indicators, including data sources and 
survey techniques. According to Urban Institute officials, they used 
existing, validated measures whenever possible to ensure comparability 
across programs.[Footnote 25] Seven of twelve implementation grantees 
we surveyed said the guidance documents were extremely or very 
helpful, while four found it moderately helpful and one somewhat 
helpful. 

The Urban Institute has analyzed the data on the indicators for the 
first implementation year (the baseline), but Education has not 
decided whether it will make the first year's data public because it 
was not collected in a consistent manner and not all grantees were 
able to collect all of the necessary data. According to Promise 
program officials there were inconsistencies in data collection 
because guidance was not available until February 2013, 13 months 
after 2011 implementation grants were awarded and over 1 month after 
2012 implementation grants were awarded. 

Promise officials stated that they will use the performance data to 
target their technical assistance. They are still working with 
grantees to develop meaningful targets for the second implementation 
year. Urban Institute officials noted that these 15 indicators help 
grantees focus their efforts on the outcomes they are trying to 
achieve. 

In addition, Promise grantees are required to develop a longitudinal 
data system to collect information on the individuals served, services 
provided in the cradle-to-career continuum, and the related 
outcomes.[Footnote 26] Grantees are expected to use the longitudinal 
data to evaluate their programs on an ongoing basis and make 
adjustments to their strategies and services, as discussed later in 
this report. 

Grantees are also required to provide the longitudinal data to 
Education, which Education officials said they may use to create a 
restricted-use data set. However, Education currently does not have a 
plan for analyzing the data. In commenting on a draft of this report, 
Education stated it must first conduct a systematic examination of the 
reliability and validity of the data to determine whether it can be 
used for a descriptive study and a restricted-use data set. Education 
further stated that the restricted-use data set would only be made 
available to external researchers after Education determines that the 
data quality is adequate and appropriate for research; analyzes the 
data, taking into account privacy concerns; and determines whether to 
release its own report. In addition, officials from IES and OPEPD 
cited limitations and challenges to using the longitudinal data for 
program evaluation. 

* An official from IES, the entity responsible for all impact 
evaluations conducted by Education, told us that it is not feasible to 
conduct an impact evaluation of individual program pieces or an 
overall evaluation of the Promise approach. The official offered three 
options for evaluation. IES' preferred option is to conduct a rigorous 
impact evaluation with a control group obtained through randomized 
assignment to the program. However, Promise Neighborhoods are not 
designed to create such a control group. Another option would be for 
IES to use students or families who were not chosen to participate in 
an oversubscribed program as a control group, but an informal poll 
that IES took at a Promise Neighborhoods conference suggested that 
there were not a sufficient number of oversubscribed programs. A third 
option was to develop a comparison group of neighborhoods that did not 
receive a Promise Neighborhood grant. However, IES officials question 
whether such an approach would enable them to match neighborhoods that 
were comparable to Promise neighborhoods at the beginning of the grant 
period. Finally, IES noted that collecting additional data for a 
control group could be expensive. 

* Education's OPEPD is responsible for conducting other types of 
program evaluations. According to Education officials, it could 
conduct a more limited evaluation focused on outcomes without 
demonstrating that they are a direct result of the Promise program, 
but they have no specific plans to do so. An OPEPD official stated 
OPEPD is reluctant to commit to a plan because they have not yet seen 
the data and do not know how reliable or complete it will be. In 
addition, the official said that OPEPD is unsure about funding and 
that any comprehensive evaluations are expensive to carry out. By 
creating a restricted-use data set, OPEPD hopes that other researchers 
may have the funding to use the data to reach some conclusions about 
the program. The OPEPD official further explained that no one has ever 
evaluated a community-based approach like this one and that they hope 
researchers may have some ideas about how to do so. 

Researchers at the Urban Institute and within the Promise grantee 
community have proposed other options for evaluating the program. A 
researcher at the Urban Institute[Footnote 27] noted that random 
assignment is not the right approach for evaluating place-based 
programs.[Footnote 28] Instead, the researcher recommends a variety of 
other options for evaluating such programs, including approaches that 
estimate a single site's effect on outcomes and aggregating those 
outcomes. This differs from the traditional program evaluation 
approach, which IES has considered, of isolating the effects of an 
intervention so that its effects can be measured separately from other 
interventions. 

While Education recognizes the importance of evaluating the Promise 
program, they lack a plan to do so. If an evaluation is not conducted, 
Education will have limited information about the Promise program's 
success or the viability of the program's collaborative approach. 

Grantees Collaborate Using Leading Practices, Resulting in Benefits 
and Challenges: 

Promise Grant Framework Encourages Collaboration: 

The Promise program generally requires grantees to use collaborative 
approaches. We found that grantees are following approaches consistent 
with those we have recognized as enhancing and sustaining 
collaboration with partners.[Footnote 29] The approaches we have 
previously identified include: 

* Establishing common outcomes: Establishing common outcomes helps 
collaborating agencies develop a clear and compelling rationale to 
work together. 

* Addressing needs by leveraging resources: Leveraging the various 
human, information technology, physical and financial resources 
available from agencies in a collaborative group allows the group to 
obtain benefits that would not be available if they worked separately. 

* Tracking performance and maintaining accountability: Tracking 
performance and other mechanisms for maintaining accountability are 
consistent with our prior work, which has shown that performance 
information can be used to improve results by setting priorities and 
allocating resources to take corrective actions to solve program 
problems. 

The approaches are discussed below and in Tables 6 through 8. 

Establishing Common Outcomes: 

Table 6: Approaches for Establishing Common Outcomes: 

Select collaboration approaches identified by GAO: 
* Start group with the most directly affected participants and 
gradually broaden to others; 
* Conduct outreach to stakeholders to identify shared interests; 
* Identify early wins for the group to accomplish; 
* Develop outcomes that represent the collective interests of 
participants; 
Grantee collaboration activities required by Education: 
* Establish, build and strengthen partnerships for purposes such as 
providing services and obtaining resources. Document partnerships with 
memoranda of understanding which explain how partner organizations' 
desired results align with the Promise program; 
* Develop a plan to deliver the continuum of services during the 
implementation years and describe how the grantee has built community 
support for and involvement in the development of the plan; 
* Propose clear and measurable annual goals as well as short and long 
term goals, as appropriate, for the implementation grant; 
Examples of grantee and partner collaboration: 
* 44 of 46 grantees that received planning grants reported that they 
brought service providers together to break down silos; 
* One grantee partnered with several organizations on a key goal for 
their Promise Neighborhood--building an early childhood foundation and 
early learning network. Three of the grantees' partners identified 
their own long term goals that align with the early childhood 
foundation in the memoranda of understanding. 

Source: GAO-14-220 and GAO analysis of relevant Federal Register 
notices. 

[End of table] 

A significant majority of the grantees we interviewed stated that 
working with partners to establish common outcomes and strategies 
helped foster closer relationships between stakeholders in the 
community. One grantee stated that in some cases, organizations were 
doing work in the neighborhood in isolation from each other. The 
Promise grant helped bring these organizations together. Similarly, 
another grantee stated that Promise is distinct from other federal 
programs in that it provides funds for organizations to create 
partnerships and facilitate collaboration. 

Almost all grantees stated that working collaboratively to meet the 
requirements of the grant helped break down barriers with local 
schools. Specifically, our survey results indicated that for 44 out of 
46 grantees that received planning grants, the grants enabled or 
enhanced their existing efforts to build relationships with schools. 
Three of the 11 grantees we interviewed also stated that one of the 
benefits of working collaboratively was building relationships between 
school administrators. For example, one Promise grantee stated that 
bringing partners together to establish common goals and strategies 
enabled the grantee to introduce the local elementary school principal 
to the principal of the secondary school where students typically 
transition for the next phase of their education. Another grantee 
reports being able to strengthen relationships among local school 
leaders. For example, the grantee stated that the schools are 
developing plans to use the same student identification code for the 
duration of each student's attendance in the schools. 

Four grantees and partners we interviewed stated that one of the 
lessons learned from their planning year was that undertaking 
practical, achievable collaborative projects that can be completed in 
the short term helped build momentum in their Promise efforts. 
Grantees stated that the grant does not designate funds for Promise 
sites to identify and achieve early tangible benefits for the 
community, and it is unclear whether grantees we did not interview 
also set aside funds for this purpose. However, the several grantees 
that raised outside funds during the planning grant year for projects 
such as building playgrounds and other safe, outdoor community spaces 
stated that these early accomplishments helped build trust and 
goodwill with the community. Our recent work on collaboration found 
that identifying such "early wins" can enable collaborative groups to 
practice working together and allows participants to build upon recent 
experiences, working relationships, improved knowledge of related 
programs, and team structures to coordinate group activities.[Footnote 
30] Given that many grantees do not receive implementation funding, 
some planning grantees may be missing an opportunity to demonstrate 
early in their Promise efforts what can be achieved when they work 
collaboratively. Such demonstrations can encourage grantees and their 
partners to continue their efforts even without implementation funding. 

Addressing Needs by Leveraging Resources: 

Table 7: Approaches for Identifying and Addressing Needs by Leveraging 
Resources: 

Select collaboration approaches identified by GAO: 
* Create an inventory of resources dedicated towards interagency 
outcomes; 
* Leverage related agency resources towards the group's outcomes; 
* Pilot test new collaborative ideas, programs or policies before 
investing resources; 
Grantee collaboration activities required by Education: 
* Complete a needs assessment and prioritization analysis during the 
planning year to identify and serve children with the highest needs[A]; 
* Planning and implementation grantees must obtain funds or in-kind 
donations to match a percentage of the Promise program funding they 
will use to support their project; 
* Establish clear, annual goals for evaluating progress in leveraging 
resources for the implementation grant; 
Examples of grantee and partner collaboration: 
* Several grantees stated that studying the needs of the community and 
mapping neighborhood assets during the planning year helped them more 
appropriately target resources. For example, during the planning year 
one grantee found that the community lacked afterschool programming 
options and residents lacked knowledge of existing services. The 
grantee partnered with two local schools to create a resource hub 
which now offers academic support, referrals, family literacy, and 
parent engagement opportunities. 

Source: GAO-14-220 and GAO analysis of relevant Federal Register 
notices. 

[A] Education refers to the prioritization analysis as a segmentation 
analysis. 

[End of table] 

Implementation grantees reported that the grant enhanced their ability 
to access resources. Specifically, our survey results indicated that 
the grants enabled 41 out of 46 grantees that received planning grants 
to build new efforts or enhance existing efforts for identifying 
service providers. A majority of grantees also reported that the grant 
enabled new efforts or enhanced existing efforts to secure funding 
from non-profits or federal, state and local government (see figure 6). 

Figure 6: Resources Leveraged by Promise Neighborhoods Grantees: 

[Refer to PDF for image: horizontal bar graph] 

Identify service providers: 
Created new effort: 7; 
Enhanced existing effort: 34. 

Identify and access state/local funds: 
Created new effort: 11; 
Enhanced existing effort: 23. 

Identify and access non-profit funds: 
Created new effort: 11; 
Enhanced existing effort: 26. 

Identify and access for-profit funds: 
Created new effort: 12; 
Enhanced existing effort: 16. 

Identify and access other federal funds: 
Created new effort: 13; 
Enhanced existing effort: 18. 

Source: GAO survey of Promise Neighborhoods grantees. 

[End of figure] 

Several grantees we interviewed stated that identifying and leveraging 
resources to address needs helped them identify gaps in services and 
identify resources that were not being adequately used. Additionally, 
implementation grantees we surveyed reported that they were able to 
decrease overlapping services. Specifically, 7 of 12 implementation 
grantees reported reduced overlap of activities and resources as a 
result of collaborating with their Promise partners. Grantees reduced 
overlap by subcontracting services from partner organizations rather 
than providing similar services in-house, consolidating outreach to 
community members for specific programs, proactively identifying 
children who are likely to attend a variety of programs to ensure that 
resources are optimized, and reducing case management services for 
families being served by multiple agencies. 

Four of the six implementation grantees and partners we interviewed 
found that the Promise implementation grant helped or may help them 
attract funding. For example, officials from one Promise Neighborhood 
stated their efforts increased funders' confidence in their capacity 
to implement other initiatives. As a result, the grantee received a 
$500,000 donation from a funder to pilot a family economic mobility 
initiative. 

Tracking Performance and Maintaining Accountability: 

Table 8: Approaches for Tracking Performance and Maintaining 
Accountability: 

Select collaboration approaches identified by GAO: 
* Develop performance measures and tie measures to shared outcomes; 
* Identify and share relevant agency performance data; 
* Develop methods to report on progress; 
Grantee collaboration activities required by Education: 
* Link specific activities and solutions to measures that can be used 
to track progress; 
* Develop data systems that integrate data from multiple sources; 
* Use data to manage program performance and to report progress; 
Examples of grantee and partner collaboration: 
* All six of the implementation grantees we interviewed stated that 
they are either in the process of building or deploying longitudinal 
data systems or generally collecting and tracking data. Additionally, 
8 of 12 implementation grantees surveyed reported that they currently 
share access to their data systems with their partners. 

Source: GAO-14-220 and GAO analysis of relevant Federal Register 
notices. 

[End of table] 

Four of the 11 grantees we interviewed mentioned that they have 
mechanisms in place to ensure that partners are committed to achieving 
their expected outcomes, such as score cards.[Footnote 31] Further, a 
majority of the implementation grantees we surveyed reported that they 
currently work or plan to work with their partners to share access to 
data systems and establish performance measures, among other things 
(see figure 7). 

Figure 7: Number of Implementation Grantees Who Reported They 
Collaborate With Their Partners to Track Performance and Maintain 
Accountability: 

[Refer to PDF for image: horizontal bar graph] 

Establish common goals and measures: 
Number of grantees responding: 
No, this is not part of our strategy: 0; 
No, but we plan to do so: 1; 
Yes, we're using this method: 11. 

Joint evaluations and reports: 
Number of grantees responding: 
No, this is not part of our strategy: 0; 
No, but we plan to do so: 4; 
Yes, we're using this method: 8. 

Sharing data systems: 
Number of grantees responding: 
No, this is not part of our strategy: 1; 
No, but we plan to do so: 3; 
Yes, we're using this method: 8. 

Sharing access to data systems: 
Number of grantees responding: 
No, this is not part of our strategy: 1; 
No, but we plan to do so: 3; 
Yes, we're using this method: 8. 

Source: GAO survey of Promise Neighborhoods grantees. 

[End of figure] 

A majority of the grantees we surveyed stated that they had conducted 
or were planning to conduct an evaluation of their efforts at the 
local level. Fourteen of 44 respondents said they had conducted a 
formal evaluation, while 17 said they were planning to do so. Eighteen 
said they had conducted a formal process evaluation, and another 18 
said they planned to do so.[Footnote 32] 

All of the implementation grantees we interviewed stated that they are 
still in the process of developing their data collection and 
monitoring efforts and several are addressing significant challenges 
during this process. Additionally, 16 of 39 grantees reported in our 
survey that since the end of the planning grant, they have found it 
very or extremely challenging to develop their data collection 
infrastructure. Specifically: 

* Two of the six implementation grantees we interviewed stated that 
because the Urban Institute's guidance for collecting data and 
reporting results was not released until after they received 
implementation funds,[Footnote 33] they began their implementation 
efforts with inadequate knowledge of the work required and resources 
they would need to meet the data requirements for the program. For 
example, officials from one Promise Neighborhood stated that Promise 
funding for evaluation is limited and that planning how to spend funds 
for evaluation may be difficult without an idea of how resource 
intensive it may be to meet the program's data requirements. According 
to these officials, Education's technical assistance providers stated 
that it should take two staff members (one full-time and one half-
time) to build and maintain the data system. However, officials from 
the neighborhood only dedicated one staff member to manage their data 
system. Further, officials from two Promise Neighborhoods stated that 
some of the Promise Neighborhood's partners who are supposed to submit 
data to the longitudinal database do not have the capacity or 
technical ability to use the longitudinal database software provided 
by the Promise Neighborhoods Institute. Officials stated that they 
have to provide help to their partners in order for them to navigate 
the database. 

* Grantees in collaboration with their partners are expected to enroll 
children and families in a data system and collect, store and use 
identified individual-level data and other personally identifiable 
information in order to monitor Promise efforts. However, federal laws 
and regulations restrict how grantees may access and share this data 
in order to protect the privacy of individual-level data. Students' 
school records are covered by the Family Educational Rights and 
Privacy Act of 1974 (FERPA)[Footnote 34] and medical records are 
covered by the Health Insurance Portability and Accountability Act of 
1996 (HIPAA).[Footnote 35] Promise grantees must obtain written 
consent in order to access and share individual-level data from 
schools for students in their target areas, in certain circumstances. 
Six of the 11 grantees we spoke with stated that one challenge they 
faced was getting consent to access and share students' records. For 
example, officials from one Promise Neighborhood stated they sent 
separate consent forms for sharing private data as part of the package 
of paperwork that parents are normally required to sign before the 
start of the school year. However, officials noted concern over 
whether they will be able to get signed consent forms from all 
families in the target area. Further, officials stated that they are 
concerned that the students and families that need Promise programs 
and services the most may not be signing consent forms and granting 
access to their information. Similarly, officials from another Promise 
Neighborhood stated that they have 2,100 children targeted in their 
geographic footprint but only received consent forms to collect and 
share individual-level data on 80 percent of those children. Officials 
stated that they are considering using aggregate data to report 
outcomes for the children that they do not receive consent forms for. 

Grantees Collaborate to Deliver a Range of Services: 

Grantees and partners provided examples of how they have collaborated 
through the Promise grant to deliver services and supports that are 
intended to improve educational and developmental outcomes. Grantees 
and their partners focused on delivering services at various steps 
along the cradle-to-career pipeline, including: 

* Early learning supports: programs or services designed to improve 
outcomes and ensure that young children enter kindergarten and 
progress through early elementary school grades demonstrating age-
appropriate functioning. 

* K-12 supports: programs, including policies and personnel, linked to 
improving educational outcomes for children in pre-school through 12th 
grade. These include developing effective teachers and principals, 
facilitating the use of data on student achievement and student growth 
to inform decision-making, supporting a well-rounded curriculum, and 
creating multiple pathways for students to earn high school diplomas. 

* College and career supports: programs preparing students for college 
and career success. These include partnering with higher education 
institutions to help instill a college-going culture in the 
neighborhood, providing dual-enrollment opportunities for students to 
gain college credit while in high school, and providing access to 
career and technical education programs. 

* Family and community supports: these include child and youth 
physical, mental, behavioral and emotional health programs, safety 
programs such as those to prevent or reduce gang activity and programs 
that expand access to quality affordable housing. 

For examples of the services delivered and outcomes reported by 
grantees for each part of the cradle-to-career pipeline, see table 9 
below. 

Table 9: Selected Promise Neighborhoods Collaborative Efforts and 
Outcomes: 

Support type: Early learning supports; 
Neighborhood location: Boston, MA; 
Service activity: Established a workgroup to create and implement 
school readiness and quality early learning strategies for children 
birth to age 5; 
Reported outcome: Began forming an Early Learning Network of 
neighborhood center-based and home-based providers to develop an early 
childhood system of care. 

Support type: Early learning supports; 
Neighborhood location: San Diego, CA; 
Service activity: Grantee worked with local elementary school partner 
to implement a transition to kindergarten program; 
Reported outcome: Helped children develop early social-emotional 
skills, improved their behavior and kindergarten attendance rates, 
enabled faculty and parents to interact, and allowed parents to become 
familiar with school facilities. 

Support type: K-12 supports; 
Neighborhood location: Los Angeles, CA; 
Service activity: Co-located grantee staff as onsite coordinators and 
advisors to school administrators to identify and manage community 
services; 
Reported outcome: Gave the student counselor more time to focus her 
efforts on academics rather than on managing agencies and outside 
resources to support students' non-academic needs. 

Support type: K-12 supports; 
Neighborhood location: Boston, MA; 
Service activity: Partnered with a local organization that provides 
residency-based teacher education programs to write the charter for a 
Pre-K-5th grade charter school; 
Reported outcome: Opened a new charter school in the neighborhood to 
provide families with more school choice. 

Support type: K-12 supports; 
Neighborhood location: Berea, KY; 
Service activity: Identified a need to help at-risk students graduate 
from high school through partnerships with local schools; 
Reported outcome: Held an intensive summer school program for at-risk 
high school students, which helped four participating students 
graduate in August and avoid a 5th year in high school. 

Support type: College and career supports; 
Neighborhood location: Indianola, MS; 
Service activity: Grantee partnered with the school district led by 
the district conservator to develop a college preparatory program; 
Reported outcome: Modified the school day across all schools to 
include an extra period at least twice a week. This extended learning 
time is used to improve students' retention of school subject areas 
and provide students with ACT preparation, among other things. 

Support type: College and career supports; 
Neighborhood location: San Diego, CA; 
Service activity: Conducted a needs assessment and confirmed the need 
for more support to help families and students understand and discuss 
opportunities for college education; 
Reported outcome: Partnered with a local college support organization 
to develop a college-prep program for students grades 2-5 that 
provides academic advising for students and parents. 

Support type: Family and community supports; 
Neighborhood location: Boston, MA; 
Service activity: Worked with a local school and partners to launch a 
homelessness prevention initiative that identifies students and 
families at risk of losing their homes and helps them avoid eviction; 
Reported outcome: Created a safety net for approximately 50 homeless 
students at its partner schools. 

Support type: Family and community supports; 
Neighborhood location: Los Angeles, CA; 
Service activity: Used funds from Department of Labor programs to 
develop youth source centers; 
Reported outcome: Provides residents with tools such as computer labs 
for computer literacy classes, youth services, and has space available 
for local schools to use for presentations and workshops. 

Source: GAO analysis of site visits with Promise Neighborhoods 
grantees. 

[End of table] 

Conclusions: 

The Promise program has energized the 48 planning and implementation 
grantees and their partners to tackle the complex challenges facing 
impoverished neighborhoods together. While grantees said they will 
continue their efforts to build their Promise Neighborhoods, planning 
grantees faced challenges in sustaining their work over the long term 
without implementation grants. Planning grantees, especially those 
concerned about building trust with their communities and partners, 
may have been better served if Education had provided a more 
transparent, realistic picture of the fiscal reality of the Promise 
program and its potential impact on implementation grant funding. Lack 
of clear communication about the expectations Education had for 
planning grantees who did not receive implementation funding made it 
difficult for these grantees to develop specific plans to continue 
their efforts without future Promise funds. However, the reported 
small, yet tangible benefits that some communities pursued during the 
planning year--such as a safe place for children to play--increased 
momentum and built trust with community members. Encouraging such 
"early wins" could help all grantees and their partners build upon and 
improve their efforts, especially since implementation funding has 
proven scarce. 

Additionally, much of the information grantees use about what existing 
federal, state, and local programs and resources to incorporate into 
their strategies is gleaned through their needs assessment at the 
local level. Education has not provided grantees with comprehensive 
information about other federal resources that may be available to use 
in their Promise strategies. Education is best positioned to develop 
and share such an inventory of federal programs that relate to the 
goals of the Promise program. Without such an inventory, Education may 
be missing opportunities to better support grantees, find other 
federal programs for future coordination efforts, and identify 
potential fragmentation, overlap, and duplication at the federal level. 

One of the Promise program's primary goals is to identify the overall 
impact of its approach and the relationship between particular 
strategies and student outcomes. Grantees are investing significant 
time and resources to collect data to assess the program, but 
Education lacks a clear plan for using it. Without evaluating program, 
it will be difficult for Education to determine whether it is 
successfully addressing the complex problem of poor student outcomes 
in impoverished neighborhoods. Finally, the Promise program is one of 
several place-based and collective impact programs being implemented 
across many federal agencies. Given the number of these initiatives, 
not evaluating the program limits Education and other agencies from 
learning about the extent to which model is effective and should be 
replicated. 

Recommendations for Executive Action: 

In order to improve grantees' planning and implementation efforts, 
increase the effectiveness of grantee efforts to integrate and manage 
resources, and learn more about the program's impact, we recommend 
that the Secretary of Education take the following three actions: 

1. Clarify program guidance about planning and implementation grants 
to provide reasonable assurance that planning grantees are better 
prepared to continue their efforts in the absence of implementation 
funding. Additional guidance could include encouraging grantees to set 
aside a small amount of the grant to identify and deliver early, 
tangible benefits to their neighborhoods. 

2. Develop and disseminate to grantees on an ongoing basis an 
inventory of federal programs and resources that can contribute to the 
Promise Neighborhoods program's goal to better support coordination 
across agency lines. 

3. Develop a plan to use the data collected from grantees to conduct a 
national evaluation of the program. 

Agency Comments: 

We provided a draft of this report to the Department of Education for 
review and comment. Education's comments are reproduced in appendix IV 
and are summarized below. Education also provided technical comments, 
which we incorporated into the final report as appropriate. Education 
outlined the steps it would take to implement our three 
recommendations, and provided its perspective on communicating 
expectations to grantees regarding future funding. Education did not 
explicitly agree or disagree with our findings. 

Regarding our finding that Education did not communicate clearly to 
planning grantees about its expectations for the grants, Education 
stated that in any given year it does not know and therefore cannot 
communicate the amount of funding available or the number of grant 
awards anticipated in the following year. We agree, and have clarified 
our finding in the report accordingly. 

Education stated that an early assessment of planning grantees' 
likelihood of receiving implementation funding would have been 
premature. Education noted that although Congress has funded the 
Promise program for the past 5 years, in 4 of those 5 years it 
appropriated far less than the President requested, and for the last 3 
years the program has essentially been level funded. Education further 
stated that this underscores the limited control that the program had 
over the number of implementation grants made. We recognize that 
federal agencies have faced a difficult fiscal climate over the past 
few years, particularly for discretionary programs. For that reason--
and especially given the level at which the Promise program has been 
funded for the past 3 years--we believe it is even more important that 
Education be clear and transparent with planning grantees about 
historical fiscal realities of the Promise program and the 
implications this may have on future implementation grants. We also 
believe this situation highlights the need for planning grantees to 
have contingency plans, especially given Education's expectations that 
grantees continue their efforts even in the absence of implementation 
funding. We further believe that this also underscores the importance 
of "early wins" to demonstrate what can be achieved when grantees and 
their partners work collaboratively, as such demonstrations can 
encourage them to continue their efforts even without implementation 
funding. 

In discussing its perspective on communicating expectations to 
grantees regarding future funding, Education stated that its 
Notifications Inviting Applications indicated that future funding was 
contingent on the availability of funds and that the program's 
frequently asked questions document noted that implementation funding 
was not guaranteed and that planning grantees would have to compete 
for implementation grants. We believe that our report adequately 
reflects these communication efforts. However, as we reported, 
Education did not communicate to planning grantees that it expected 
them to continue their efforts even in the absence of implementation 
funding. Nor did Education communicate to implementation grant 
applicants that it expected them to be able to use their partners' 
pledged matching funds even if they did not receive implementation 
grants. This lack of communication was evidenced by planning grantees' 
lack of contingency plans and challenges they faced accessing the 
pledged matching funds, according to the grantees we interviewed. 

In response to our first recommendation, Education stated that it 
would continue to communicate to planning grant applicants that 
implementation funding is contingent on the availability of funds, and 
that it would provide more targeted technical assistance to planning 
grant recipients regarding strategies for continuing grantees' efforts 
absent implementation funding. Education also stated that it would 
clarify to grantees that planning grant funds could be used to achieve 
early, tangible benefits. 

Regarding our second recommendation, Education stated that it would 
work with its technical assistance providers to create a mechanism to 
distribute a comprehensive list of external funding opportunities, 
programs and resources on a regular basis to better support the 
grantees' implementation efforts. 

With regard to our final recommendation, Education stated that it will 
consider options for how and whether it can use the data collected 
from grantees to conduct a national evaluation. Education stated that 
as a first step it will conduct a systematic evaluation of the 
reliability and validity of the data, given issues that we and 
Education noted about inconsistencies in data collection and privacy 
concerns. In addition, Education stated that to date, it has not 
received sufficient funding to support a national evaluation. We agree 
that conducting evaluations can be costly. However, given that one of 
Education's primary goals is to learn about the overall impact of the 
program through a rigorous program evaluation, we continue to believe 
that absent an evaluation, it will be difficult for Education to 
determine whether it is successfully addressing the complex problem of 
poor student outcomes in impoverished neighborhoods--one of its stated 
goals. Further, developing an evaluation plan would provide critical 
information about the resources required to conduct an evaluation, and 
could better inform future funding requests for such an evaluation. 

As agreed with your office, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies of this report 
to the Secretary of Education and other interested congressional 
committees. In addition, the report will be available at no charge on 
GAO's website at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions regarding this report, please 
contact me at 617-788-0580 or NowickiJ@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made major contributions 
to this report are in appendix V. 

Sincerely yours, 

Signed by: 

Jacqueline M. Nowicki: 
Acting Director, Education, Workforce, and Income Security: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

To better understand grantees' experiences with the Promise 
Neighborhoods program, we conducted a web-based survey of all 48 
planning and implementation grantees. The survey was conducted from 
August 23, 2013 through November 7, 2013. We received completed 
surveys from all 48 grantees for a 100 percent response rate. The 
survey included questions about the clarity and helpfulness of the 
application and peer review process, challenges sustaining efforts 
after the end of the planning grant, coordination of federal 
resources, collaboration with local organizations and associated 
challenges, the extent to which local coordination reduced 
duplication, overlap and fragmentation, if at all, the mechanisms 
organizations use to track the results of their efforts, the results 
of the grants, and the helpfulness of Education's guidance and 
resources for the program. 

Because this was not a sample survey, there are no sampling errors. 
However, the practical difficulties of conducting any survey may 
introduce nonsampling errors, such as variations in how respondents 
interpret questions and their willingness to offer accurate responses. 
We took steps to minimize nonsampling errors, including pretesting 
draft instruments and using a web-based administration system. 
Specifically, during survey development, we pretested draft 
instruments with five grantees that received planning and/or 
implementation grants. In the pretests, we were generally interested 
in the clarity, precision, and objectivity of the questions, as well 
as the flow and layout of the survey. For example, we wanted to ensure 
definitions used in the surveys were clear and known to the 
respondents, categories provided in closed-ended questions were 
complete and exclusive, and the ordering of survey sections and the 
questions within each section was appropriate. We revised the final 
survey based on pretest results. We took another step to minimize 
nonsampling errors by using a web-based survey. This allowed 
respondents to enter their responses directly into an electronic 
instrument and created a record for each respondent in a data file--
eliminating the need for manual data entry and its associated errors. 
To further minimize errors, programs used to analyze the survey data 
were independently verified to ensure the accuracy of this work. 
Because not all respondents answered every question, we reported the 
number of grantees responding to particular questions throughout the 
report. 

In addition, we conducted site visits to 11 Promise grantees. We 
selected sites based on several factors, such as the type of grant 
awarded, the location of the grantees, and whether the Promise 
Neighborhood was urban or rural. The site visits provided 
opportunities to collect more in-depth information on the program and 
highlighted different types of grantees and approaches. We visited six 
implementation grantees in Boston, Massachusetts; Berea, Kentucky; 
Chula Vista, California; Indianola, Mississippi; Los Angeles, 
California; and Washington, DC. We visited five planning grantees in 
Campo, California; Lawrence, Massachusetts; Los Angeles, California; 
Nashville, Tennessee; and Worcester, Massachusetts. These include one 
tribal and two rural grantees. We also interviewed Education officials 
and technical assistance providers, as well as other experts who have 
worked with Promise grant applicants, such as the Promise 
Neighborhoods Institute. 

To determine how well the structure of Education's Promise 
Neighborhoods grant program aligns with program goals and how 
Education selected grantees, using Education's goals for the Promise 
program as criteria, we reviewed Education reports on place-based 
strategies; relevant Federal Register notices; and application 
guidance and training materials, including both the guidance available 
to applicants and to the peer reviewers regarding the technical 
evaluation/grant selection process. We reviewed agency information on 
applicants for implementation grants in the fiscal year 2011 and 2012 
cycles, as fiscal years 2011 and 2012 were the only years in which 
Education awarded implementation grants. For both cycles, we analyzed 
application materials and technical evaluation documentation for a 
subset of implementation grant applicants--those that received 
planning grants in prior years. We compared the scores in each 
component of the application for both successful and unsuccessful 
applicants to identify criteria or factors that accounted for 
significant variation in total scores. We conducted a limited review 
of selected peer reviewer comments to gain more insight into the 
reasons for any differences. We interviewed Education officials about 
the process that the department used for the selection of both 
planning and implementation grantees. 

To determine how the Promise Neighborhoods program coordinated with 
other Education programs and with the other federal agencies, 
including those involved in the White House Neighborhood 
Revitalization Initiative (NRI), we reviewed documentation of the 
NRI's efforts and interviewed agency officials participating in the 
NRI. We also interviewed cognizant officials at other agencies 
participating in the NRI. To assess Education's approach to evaluating 
the success of the grants, we reviewed grant monitoring reports, 
Education's performance measures, and related guidance for data 
collection for this program and interviewed agency officials 
responsible for evaluation, including technical assistance providers. 

To determine the extent to which Promise grants enabled collaboration 
at the local level, we used GAO's prior work on implementing 
interagency collaborative mechanisms as criteria.[Footnote 36] We 
compared the Promise grants' collaboration mechanisms to certain 
successful approaches used by select interagency groups and reviewed 
implementation grantees' application materials. Our 11 site visits 
provided additional insight into how selected grantees align services 
supported by multiple funding streams and delivered by multiple 
providers. Using survey responses from all planning grantees, we 
determined whether they have continued their efforts, whether they 
have implemented any of their strategies, and what, if any, interim 
results they have identified, regardless of whether they received 
implementation grants. Site visits provided illustrative examples of 
interim benefits and challenges. 

We conducted this performance audit from February 2013 to May 2014 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on our audit objectives. 

[End of section] 

Appendix II: Promise Neighborhood Grantees: 

Organization: Abyssinian Development Corporation; 
Location: New York, NY (Harlem); 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: Amherst H. Wilder Foundation; 
Location: St. Paul, MN; 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: Athens Clarke County Family Connection Inc.; 
Location: Athens, GA; 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: Berea College; 
Location: Clay, Jackson, and Owsley Counties, KY; 
Year planning grant received: 2010; 
Year implementation grant received: 2011. 

Organization: Boys & Girls Club of the Northern Cheyenne Nation; 
Location: Northern Cheyenne Reservation, MT; 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: California State University East Bay; 
Location: Hayward, CA; 
Year planning grant received: 2010; 
Year implementation grant received: 2011. 

Organization: DC Promise Neighborhood, Inc.; 
Location: Washington, DC; 
Year planning grant received: 2010; 
Year implementation grant received: 2012. 

Organization: Community Day Care Center of Lawrence, Inc.; 
Location: Lawrence, MA; 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: Delta Health Alliance, Inc.; 
Location: Indianola, MS; 
Year planning grant received: 2010; 
Year implementation grant received: 2012. 

Organization: Dudley Street Neighborhood Initiative; 
Location: Boston, MA; 
Year planning grant received: 2010; 
Year implementation grant received: 2012. 

Organization: Lutheran Family Health Centers/Lutheran Medical Center; 
Location: New York , NY (Brooklyn); 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: Morehouse School of Medicine, Inc.; 
Location: Atlanta, GA; 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: Neighborhood Centers, Inc.; 
Location: Houston, TX; 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: Proyecto Pasotoral at Dolores Mission; 
Location: Los Angeles, CA; 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: The Guidance Center; 
Location: River Rouge, MI; 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: United Way of Central Massachusetts, Inc.; 
Location: Worcester, MA; 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: United Way of San Antonio & Bexar County, Inc; 
Location: San Antonio, TX; 
Year planning grant received: 2010; 
Year implementation grant received: 2011. 

Organization: Universal Community Homes; 
Location: Philadelphia, PA; 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: University of Arkansas at Little Rock; 
Location: Little Rock, AR; 
Year planning grant received: 2010; 
Year implementation grant received: [Empty]. 

Organization: Westminster Foundation; 
Location: Buffalo, NY; 
Year planning grant received: 2010; 
Year implementation grant received: 2011. 

Organization: Youth Policy Institute; 
Location: Los Angeles, CA; 
Year planning grant received: 2010; 
Year implementation grant received: 2012. 

Organization: Black Family Development; 
Location: Detroit , MI; 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: CAMBA; 
Location: New York, NY (Brooklyn); 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: Campo Band of Mission Indians; 
Location: Campo, CA; 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: Catholic Diocese of Albany; 
Location: Greenport, NY and Hudson, NY; 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: Children Youth and Family Services; 
Location: Charlottesville, VA; 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: Community Action Project of Tulsa; 
Location: Tulsa, OK; 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: Thomas and Jeanne Elmezzi Foundation; 
Location: New York, NY (Queens); 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: Martha O'Bryan Center; 
Location: Nashville, TN; 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: Mercer University; 
Location: Macon, GA; 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: Meriden Children First; 
Location: Meriden, CT; 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: Mission Economic Development Agency; 
Location: San Francisco, CA; 
Year planning grant received: 2011; 
Year implementation grant received: 2012. 

Organization: Northside Achievement Zone; 
Location: Minneapolis, MN; 
Year planning grant received: [Empty]; 
Year implementation grant received: 2011. 

Organization: Ohio University; 
Location: Glouster, OH; 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: Reading and Beyond; 
Location: Fresno, CA; 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: SGA Youth and Family Services; 
Location: Chicago, IL; 
Year planning grant received: 2011; 
Year implementation grant received: [Empty]. 

Organization: South Bay Community Services; 
Location: Chula Vista, CA; 
Year planning grant received: 2011; 
Year implementation grant received: 2012. 

Organization: CASA de Maryland, Inc.; 
Location: Langley Park, MD; 
Year planning grant received: 2012; 
Year implementation grant received: [Empty]. 

Organization: Center for Family Services, Inc.; 
Location: Camden, NJ; 
Year planning grant received: 2012; 
Year implementation grant received: [Empty]. 

Organization: Cypress Hills Local Development Corporation; 
Location: New York, NY (Brooklyn); 
Year planning grant received: 2012; 
Year implementation grant received: [Empty]. 

Organization: Penquis C.A.P., Inc.; 
Location: Rockland, Cushing, Owls Head, St. George, Thomaston, and 
South Thomaston, ME; 
Year planning grant received: 2012; 
Year implementation grant received: [Empty]. 

Organization: Mid-Iowa Community Action; 
Location: Marshalltown, IA; 
Year planning grant received: 2012; 
Year implementation grant received: [Empty]. 

Organization: Paskenta Band of Nomlaki Indians; 
Location: Corning, CA; 
Year planning grant received: 2012; 
Year implementation grant received: [Empty]. 

Organization: Rutgers, The State University; 
Location: Newark, NJ; 
Year planning grant received: 2012; 
Year implementation grant received: [Empty]. 

Organization: Texas Tech University; 
Location: Lubbock, TX; 
Year planning grant received: [Empty]; 
Year implementation grant received: 2012. 

Organization: United Way of Northern Utah; 
Location: Ogden, UT; 
Year planning grant received: 2012; 
Year implementation grant received: [Empty]. 

Organization: University of Maryland Baltimore; 
Location: Baltimore, MD; 
Year planning grant received: 2012; 
Year implementation grant received: [Empty]. 

Organization: Renewal Unlimited, Inc.; 
Location: Adams County, WI; 
Year planning grant received: 2012; 
Year implementation grant received: [Empty]. 

Source: Department of Education data. 

[End of table] 

[End of section] 

Appendix III: Goals, Results, and Performance Indicators for Promise 
Neighborhood Program: 

Table 10: Promise Neighborhoods Program's Design for Achieving Its 
Goals: 

How Promise Neighborhoods will achieve its goal: Identify and increase 
the capacity of eligible organizations focused on achieving results 
for children and youth throughout an entire neighborhood; 
Grant requirements supporting method of achieving goal: 
* Applicants must be a nonprofit, institution of higher education, or 
Indian tribe; 
* Applicants must be representative of the area to be served; 
* Applicant must already be providing at least one of the programs or 
services proposed in the cradle-to-career continuum in the 
neighborhood. 

How Promise Neighborhoods will achieve its goal: Building a complete 
continuum of cradle-through-college-to-career solutions of both 
educational programs and family and community supports with great 
schools at the center; 
Grant requirements supporting method of achieving goal: 
* Implementation grant applicants must develop an implementation plan 
based on a comprehensive needs assessment and prioritization analysis. 
Applicant must also describe how it built community support for and 
involvement in the development of the plan; 
* While grants are awarded to organizations already providing services 
in the community, the applications require a partnership with a target 
school serving the neighborhood that is either a persistently lowest-
achieving or a low-performing school. 

How Promise Neighborhoods will achieve its goal: Integrating programs 
and breaking down agency "silos" so that solutions are implemented 
effectively and efficiently across agencies; 
Grant requirements supporting method of achieving goal: 
* Applicants must identify effective partner organizations and 
formalize these partnerships with memoranda of understanding; 
* Grant recipients are required to have a structure to hold partners 
accountable. 

How Promise Neighborhoods will achieve its goal: Developing the local 
infrastructure of systems and resources needed to sustain and scale up 
proven, effective solutions across the broader region beyond the 
initial neighborhood; 
Grant requirements supporting method of achieving goal: 
* Implementation grant applicants must include a plan to collect data 
on indicators at least annually and use data for learning, continuous 
improvement, and accountability. A longitudinal data system may 
provide some of this data; 
* Applicants must submit a list of proposed funds and in-kind 
donations to match the federal investment that it proposes to use to 
support its Promise Neighborhoods project. 

How Promise Neighborhoods will achieve its goal: Learning about the 
overall impact of the Promise Neighborhoods program and about the 
relationship between particular strategies in Promise Neighborhoods 
and student outcomes, including through a rigorous evaluation of the 
program; 
Grant requirements supporting method of achieving goal: 
* Implementation grant recipients must provide performance on 
indicators at least annually and submit longitudinal data to 
Education, which will be used to evaluate the program; 
* Grant recipients must also participate in a community of practice to 
solve persistent problems or improve practices among Promise 
Neighborhoods. 

Source: GAO analysis of relevant Federal Register notices and other 
program documentation. 

[End of table] 

Table 11: Promise Neighborhoods Desired Outcomes and Indicators: 

Outcome: Children enter kindergarten ready to succeed in school; 

Indicator: Number and percent of children, from birth to kindergarten 
entry, who have a place where they usually go, other than an emergency 
room, when they are sick or in need of advice about their health. 
Indicator: Number and percent of three-year-olds and children in 
kindergarten who demonstrate at the beginning of the program or school 
year age-appropriate functioning across multiple domains of early 
learning as determined using developmentally-appropriate early 
learning measures. 
Indicator: Number and percent of children, from birth to kindergarten 
entry, participating in center-based or formal home-based early 
learning settings or programs, which may include Early Head Start, 
Head Start, child care, or preschool. 

Outcome: Students are proficient in core academic subjects; 
Indicator: Number and percent of students at or above grade level 
according to state mathematics and reading or language arts 
assessments in at least the grades required by the Elementary and 
Secondary Education Act (3rd through 8th and once in high school). 

Outcome: Students successfully transition from middle school grades to 
high school; 
Indicator: Attendance rate of students in 6th, 7th, 8th, and 9th grade. 

Outcome: Youth graduate from high school; 
Indicator: Graduation rate. 

Outcome: High school graduates obtain a postsecondary degree, 
certification, or credential; 
Indicator: Number and percent of Promise Neighborhood students who 
graduate with a regular high school diploma and obtain postsecondary 
degrees, vocational certificates, or other industry-recognized 
certifications or credentials without the need for remediation. 

Outcome: Students are healthy; 
Indicator: Number and percent of children who participate in at least 
60 minutes of moderate to vigorous physical activity daily. 

Indicator: Students feel safe at school and in their community: Number 
and percent of children who consume five or more servings of fruits 
and vegetables daily. 

Outcome: Students feel safe at school and in their community; 
Indicator: Number and percent of students who feel safe at school and 
traveling to and from school, as measured by a school climate needs 
assessment. 

Outcome: Students live in stable communities; 
Indicator: Student mobility rate. 

Outcome: Families and community members support learning in Promise 
Neighborhood schools; 
Indicator: For children birth to kindergarten entry, the number and 
percent of parents or family members who report reading to their 
children three or more times a week. 

Indicator: Outcome: For children in kindergarten through 8th grades, 
the number and percent of parents or family members who report 
encouraging their children to read books outside of school. 

Indicator: Students have access to 21st century learning tools: For 
children in the 9th to 12th grades, the number and percent of parents 
or family members who report talking with their child about the 
importance of college and career. 

Outcome: Students have access to 21st century learning tools; 
Indicator: Number and percent of students who have school and home 
access (and percent of the day they have access) to broadband internet 
and a connected computing device. 

Source: Relevant Federal Register notices. 

[End of table] 

[End of section] 

Appendix IV: Comments from the Department of Education: 

United States Department of Education: 
Office of Innovation and Improvement: 
400 Maryland Ave. S.W. 
Washington, DC 20202: 
[hyperlink, http://www.ed.gov] 

April 25, 2014: 

Ms. Jacqueline M. Nowicki: 
Acting Director: 
Education, Workforce, and Income Security Issues: 
Government Accountability Office: 
Washington, DC 20548: 

Dear Ms. Nowicki: 

Thank you for the opportunity to comment on the Government 
Accountability Office (GAO) draft report entitled, "Promise 
Neighborhoods Promotes Collaboration, but Needs National
Evaluation Plan." While the Department of Education (the Department) 
appreciates the positive aspects of the Promise Neighborhoods program 
highlighted in the report, GAO fails to provide critical information 
needed to better understand the Department's response and approach to 
the recommended Executive Actions. Before responding to GAO's 
Recommendations for Executive Action, we would like to offer 
additional information on the background of the Promise Neighborhoods 
program and the communication of expectations to the grantees related 
to future funding. 

I. Background on the Promise Neighborhoods Program: 

The Promise Neighborhoods program is a place-based initiative intended 
to significantly transform neighborhoods of concentrated poverty into 
neighborhoods of promise by improving the educational and 
developmental outcomes of the children and families living in these
communities. The Promise Neighborhoods program provides critical 
support for the planning and implementation of comprehensive services 
ranging from early learning, K-12 education, to college and career 
support. This includes programs to improve the health, safety. and 
stability of neighborhoods, as well as to boost family engagement in 
student learning and improve access to learning technology. By 
providing support, resources, and guidance to the Promise 
Neighborhoods grantees with their planning and implementation efforts, 
the Department has demonstrated its commitment to ensuring that all 
children and youth growing up in Promise Neighborhoods have access to 
excellent schools and strong systems of family and community support. 

Through the Promise Neighborhoods program, the Department has 
established a meaningful approach to supporting, improving, and 
transforming the educational and developmental outcomes of children 
and youth in our most distressed communities. In the 12 communities with
funds to implement their plans, we are seeing the acceleration of the 
coordination and alignment of early childhood services. These 
communities are expanding and enhancing services by leveraging the 
resources and capacity of partners to improve both the quality of and 
access to services. Examples include the additional offering of 
parenting classes; an increase in the number of slots for early 
childhood education; and enhancements to adult literacy programs. In
the report, GAO acknowledges that the Promise Neighborhoods program 
embodies, through its grant requirements, some of the leading 
practices (e.g., establishing common outcomes, leveraging resources, 
and tracking performance) and approaches previously identified by GAO
as necessary to enhancing and sustaining collaboration among partners. 
GAO also states in the report that the Promise Neighborhoods program 
is distinct from other federal programs in that it provides funds for 
organizations to create partnerships, facilitate collaboration, and 
use federal grant funds in a coordinated manner to improve services 
and reduce overlap between programs. Like GAO, we also believe these 
partnerships are critical to scaling up and sustaining change in
Promise Neighborhoods. 

II. Areas for Improvement Identified in The Report: 

GAO identifies many of the strengths of the Promise Neighborhoods 
program in the report. Several areas where improvement can be made are 
also identified. We will consider these areas for improvement 
carefully as we respond to the report, continue to support grantee 
efforts, and administer the program. One area in particular where the 
Department would like to offer additional information is our 
communication to grantees on future funding. 

The report does not provide a complete and accurate picture of the 
Department's efforts to convey clear expectations to the Promise 
Neighborhoods grantees, nor does it discuss the funding history for 
this program and its impact on the number and timing of the grant awards
made. GAO states in the report that the planning grantees may have 
been better served had the Department provided an earlier and more 
transparent assessment of their chances to receive funding for 
implementation. However, an early assessment would have been premature 
and unfair, and grantees knew that there was no guarantee of further 
funding. 

Congress, beginning in fiscal year (FY) 2010, has funded the Promise 
Neighborhoods program for the past five years. However, with the 
exception of FY 2010, the Promise Neighborhoods program's annual 
appropriation has been much lower than the President's budget request 
for the program. For the past three years, this program has been level-
funded, receiving $57-$60 million annually, which is only sufficient 
to cover current implementation grantees' continuation costs and to 
provide technical assistance to support the existing planning and 
implementation grantees. In FY 2014, for example, the President's 
budget requested $300 million for the program to make grant awards 
(both planning and implementation grants); to provide technical
assistance to the grantees; and to support a national evaluation of 
the program. Yet in FY 2014, the program only received $57 million. 
Due solely to a lack of available funds, no new planning or 
implementation grants have been awarded since 2012. 

This financial context is important and underscores the limited 
control the Department has over various issues noted in the GAO 
report, including: the total number of grants made in any given
year; the amount of funding available to award grants; the amount of 
funding available to provide technical assistance and support a 
national program evaluation; and the timing of the grant awards. Due 
to these factors, in any given year in which grant awards are made, the
Department does not know--and therefore cannot communicate to 
potential applicants--the amount of funding available or the number of 
grant awards anticipated in the following year. 

The Department, beginning with the first Promise Neighborhoods 
planning competition in FY 2010, and in subsequent years, has 
consistently and clearly communicated to grantees about the
uncertainties associated with future funding under this program. The 
Department's efforts to communicate these expectations are noted in 
all grant competition documents created for potential applicants 
beginning with the FY 2010 Notice Inviting Applications (NIA) and the
frequently Asked Questions (FAQs) Guidance. These documents, coupled 
with the preapplication webinar series, are the earliest vehicles the 
Department has to communicate with potential applicants (including 
current grantees that intend to submit an application). 

Since FY 2010, each Promise Neighborhoods NIA has included an accurate 
estimate of the amount of funding available. the average size of 
awards, and the number of awards to be made. Furthermore, all NIAs for 
this program (both planning and implementation) address the issue of
future funding and contain language that states, "In subsequent years, 
contingent on the availability of funds," the Department intends to 
conduct competitions or may make additional awards from the list of 
unfunded applicants. The Department further elaborates on this by
including a section in the FAQs Guidance entitled, "Future 
Competitions," which states that planning grantees will not 
automatically receive implementation grants. It also clarifies that
planning grantees will be required to compete for implementation 
grants alongside applicants who did not compete for or receive a 
planning grant. 

To further demonstrate the Department's effort to communicate 
expectations to potential planning grantees regarding their chances of 
receiving implementation funding, the Department also required the 
planning grantees to develop memoranda of understanding (MOUs) with key
partners. Since this was a requirement in the planning phase, all 
planning grantees understood that the obligation of the partner 
organizations to sign the MOUs was not contingent upon the receipt of 
federal funding for all the reasons cited earlier regarding the 
uncertainty of future federal funding. 

III. Comments on Specific Recommendations: 

Clarify program guidance about planning and implementation grants to 
provide reasonable assurance that planning grantees are better 
prepared to continue their efforts in the absence of implementation 
funding. Additional guidance could include encouraging grantees to set
aside a small amount of the grant to identify and deliver early, 
tangible benefits to their neighborhoods. 

As funding becomes available to support new planning and 
implementation grants, the Department will continue to emphasize in 
its communications to all interested planning applicants through the 
vehicles discussed earlier (i.e., NIA, FAQs Guidance, pre-application
webinars, MOU requirement) that federal funding to support continued 
implementation efforts is contingent upon the availability of funds. 
In addition, during the post-award phase, Department staff will 
provide more targeted technical assistance and guidance to the 
planning grantees regarding strategies to consider for continuing 
their efforts absent implementation funding. 

GAO also notes that the Department could include additional guidance 
that encourages grantees to set aside funding to identify and deliver 
early, tangible benefits to their neighborhoods. Currently, grantees 
are free to use Promise Neighborhoods funds to support some of these 
early benefits described in the GAO report as long as these efforts 
are: directly related to either the planning process or the execution 
of an implementation plan; and are an allowable use of funds under 
federal law. In fact, many of the grantees have opted to use their 
matching funds to support these types of activities. While the 
Department will continue to work with the grantees to clarify 
allowable uses or Promise Neighborhoods funds, we believe, and have 
stated in the NIA for the implementation competition, that grantees 
are best positioned to determine the allocation of their grant funds. 

Develop and disseminate to grantees on an ongoing basis an inventory 
of programs and resources that can contribute to the Promise 
Neighborhoods program's goal to better support coordination across 
federal agency lines. 

Currently, Department staff along with the Promise Neighborhoods 
program's technical assistance providers, disseminates resources to 
the Promise Neighborhoods grantees that could enhance or expand their 
current efforts, including external funding opportunities. In addition,
Department staff will work with the Promise Neighborhoods program's 
technical assistance providers to create a mechanism to distribute, on 
a regular basis, a comprehensive list of external funding 
opportunities, programs, and resources to better support the grantees' 
implementation efforts. 

Develop a plan to use the data collected from grantees to conduct a 
national evaluation of the program. 

The Department will consider options for how and whether it can use 
the data collected from grantees to conduct a national evaluation. As 
GAO acknowledges, there have been some challenges in the data 
collection, and grantees have reported some difficulties in 
collaborating with partners about data, particularly because of 
privacy concerns related to some of the data collection. As a first 
step in determining whether the data can be used for a descriptive 
study and to create a restricted use data set, the Department will 
move ahead with a systematic examination of the reliability and 
validity of this data. A data set would only be made available to 
researchers after the Department has determined that the data quality 
is adequate and appropriate for research; analyzed the data, taking 
into account the privacy concerns; and considered the release of its 
own report. 

Additionally, the Department would need to receive funding to support 
an evaluation. Thus far, sufficient funds have not been secured to 
allow us to fund grantees, provide technical assistance, and conduct 
an evaluation. If additional funding becomes available and the issues 
noted above can be resolved, the Department will move to develop plans 
for a national evaluation of the Promise Neighborhoods Program. 

IV. Conclusion: 

Thank you for the opportunity to provide a written response to the 
draft report. We strongly encourage GAO to consider the information 
provided in this response when preparing the final report. As 
indicated, we will use the information in this report to continue to 
improve the Department's oversight to help ensure the success of the 
Promise Neighborhoods program. 

Sincerely, 

Signed by: 

Nadya Chinoy Dabby: 
Assistant Deputy Secretary for Innovation and Improvement: 

[End of section] 

Appendix V: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Jacqueline M. Nowicki (617) 788-0580: 

Staff Acknowledgments: 

In addition to the contact named above, Elizabeth Sirois, Assistant 
Director; Jacques Arsenault; Aimee Elivert; and Lara Laufer made key 
contributions to this report. Also contributing to this report were 
James Bennett, Deborah Bland, Mallory Barg Bulman, Holly Dye, Alex 
Galuten, Jean McSween, Matthew Saradjian, and Sarah Veale. 

[End of section] 

Footnotes: 

[1] The conference report accompanying H.R. 3288 included $10 million 
for Promise Neighborhoods. H.R. Rep. No. 111-366, at 1051 (2009) 
(Conf. Rep.). 

[2] Office of Management and Budget, Budget of the United States 
Government, Fiscal Year 2015 (Washington, D.C.: March 2014). 

[3] Fiscal years 2011 and 2012 were the only years in which Education 
awarded implementation grants. 

[4] GAO, Managing for Results: Implementation Approaches Used to 
Enhance Collaboration in Interagency Groups, [hyperlink, 
http://www.gao.gov/products/GAO-14-220] (Washington, D.C.: Feb. 14, 
2014). To identify the approaches we selected four interagency groups 
that use key practices for enhancing and sustaining collaboration. To 
identify successful approaches, we reviewed agency documents and 
interviewed agency officials who participated in these groups. 

[5] The Promise Neighborhoods Institute is a nonprofit independent 
organization affiliated with PolicyLink. The institute supports 
Promise Neighborhoods grantees and aspiring Promise Neighborhoods by 
providing technical assistance with its partners, the Harlem 
Children's Zone and the Center for the Study of Social Policy. The 
Center for the Study of Social Policy is one of Education's contracted 
technical assistance providers. 

[6] John Kania and Mark Kramer, "Collective Impact," Stanford Social 
Innovation Review, (Winter 2011). 

[7] Cincinnati's Strive program brings together representatives from 
private and corporate foundations, city governments, school districts, 
universities and community colleges, and education-related nonprofit 
and advocacy groups to coordinate improvements at every stage of a 
young person's life, from "cradle to career." Within 4 years Strive 
reported positive trends on almost two-thirds of its success 
indicators, including high school graduation rates and fourth grade 
reading and math scores. 

[8] The Harlem Children's Zone provides a range of support services to 
address problems in a 100-block area in New York City, including poor 
housing, failing schools, violent crime, and chronic health problems. 
The program provides parenting workshops, preschool, health 
initiatives, charter schools, in-school programs, and after-school 
programs, among other services. 

[9] According to Education officials, all implementation grants 
awarded to date have been for 5-year projects. 

[10] For both planning and implementation grants, Education considers 
only applications that meet the criteria for an absolute priority. 
Under these priorities, applicants are required to describe in detail 
about how they will carry out the project. Federal regulations require 
grantees to carry out the project as specified in their approved 
applications. 34 C.F.R. § 75.700. As a result, throughout this report 
we refer to the numerous aspects of the application required under an 
absolute priority as "requirements," as the grantees are required to 
carry out the project as described in their applications. 

[11] Because we asked grantees to generally discuss what suggestions 
they had for improving the program, not all of the grantees we 
interviewed commented specifically about Education's two-phase 
strategy for awarding grants. 

[12] One of the five planning grantees we interviewed had not yet 
applied for an implementation grant. 

[13] GAO, Internal Control: Standards for Internal Control in the 
Federal Government, [hyperlink, 
http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: 
November 1999). 

[14] Of the 21 grantees who received planning grants in 2010, 8 
received implementation grants in either 2011 or 2012. Of the 15 2011 
planning grantees, two have received implementation grants. Two 
implementation grants were awarded to grantees who had not received 
planning grants. Education did not award any additional planning or 
implementation grants for 2013 or 2014. 

[15] The 2010 Federal Register Notice Inviting Applications for 
planning grants noted that in subsequent years, contingent on the 
availability of funds, the Department intended to conduct competitions 
for implementation grants, as well as competitions for new Promise 
Neighborhoods planning grants. The 2010 Frequently Asked Questions 
issued by Education also stated that planning grantees would not 
automatically receive implementation grants and would compete for 
implementation grants alongside applicants who did not compete for or 
receive a planning grant. 

[16] Because we asked grantees to generally discuss how they sustained 
momentum in the absence of implementation funding, not all grantees 
provided information about whether they were able to keep pledged 
matching funds. 

[17] This city-focused, interagency program began in 2011. 

[18] Promise Zones had to meet a number of requirements, including 
meeting certain poverty thresholds and having certain population 
levels. 

[19] Applicants are usually municipalities or non-profit organizations 
working with the municipality. Designations allow the zones to receive 
on-the-ground technical assistance from federal agencies to solve 
problems and navigate federal programs. In addition they will be 
eligible for competitive preference priority in applying for certain 
grants from the 10 federal agencies involved in the effort. Designated 
Promise Zones use existing resources to make new investments in the 
community with their local business and non-profit partners and have 
identified related outcomes they will track. The administration has 
also proposed employer tax incentives to create jobs for Zone 
residents and attract private capital to disinvested places. The first 
five of twenty Zones were designated in January 2014: San Antonio, 
Philadelphia, Los Angeles, Southeastern Kentucky, and the Choctaw 
Nation of Oklahoma. 

[20] The five other agencies are the Departments of Commerce, Labor, 
and Agriculture, the Small Business Administration, and the 
Corporation for National and Community Service. 

[21] GAO, Managing for Results: Implementation Approaches Used to 
Enhance Collaboration in Interagency Groups, [hyperlink, 
http://www.gao.gov/products/GAO-14-220] (Washington, D.C.: February 
14, 2014). The GPRA Modernization Act of 2010 requires the Director of 
the Office of Management and Budget to compile and make publicly 
available a comprehensive list of all federal programs identified by 
federal agencies. Pub. L. No. 111-352, § 7, 124 Stat. 3866, 3876. 

[22] GAO, 2013 Annual Report Actions Needed to Reduce Fragmentation, 
Overlap, and Duplication and Achieve Other Financial Benefits, 
[hyperlink, http://www.gao.gov/products/GAO-13-279SP] (Washington, 
D.C.: Apr. 9. 2013). This report and previous GAO annual reports on 
this topic have highlighted the challenges associated with the lack of 
a comprehensive list of federal programs and funding information. 

[23] Not all respondents answered every question. 

[24] GAO, Designing Evaluations: 2012 Revision, [hyperlink, 
http://www.gao.gov/products/GAO-12-208G] (Washington, D.C.: January 
2012). 

[25] Urban Institute, Measuring Performance: A Guidance Document for 
Promise Neighborhoods on Collecting Data and Reporting Results, 
prepared for the U.S. Department of Education (Washington, D.C.: 
February 2013). 

[26] The Promise Neighborhoods Institute works closely with the 
technical assistance provider and provides assistance to high-scoring 
Promise Neighborhoods applicants and both implementation and planning 
grantees. It has made longitudinal database software available to all 
grantees free of charge. 

[27] The researcher proposed this option in his capacity as an 
independent researcher, not as part of the Urban Institute's contract 
for technical assistance. 

[28] Nichols, Austin, Evaluation of Community-Wide Interventions, 
(Washington, D.C.: July 2013). Random assignment is an evaluation 
design in which participants are either randomly selected to receive a 
service or assigned to a control group. According to Nichols, the 
traditional program evaluation approach does not lend itself to 
evaluation of the Promise program because the Promise program aims to 
produce change by affecting the entire community, not just those who 
receive direct services. In addition, place-based programs are not the 
same from one place to another. These programs offer different 
interventions based on the communities' and the individuals' unique 
needs, and interventions may change as outcome data is analyzed. 

[29] GAO, Managing for Results: Implementation Approaches Used to 
Enhance Collaboration in Interagency Groups, [hyperlink, 
http://www.gao.gov/products/GAO-14-220] (Washington, D.C.: Feb. 14, 
2014). 

[30] [hyperlink, http://www.gao.gov/products/GAO-14-220]. 

[31] Because we asked grantees to generally discuss how they were 
using data, not all of the grantees we interviewed specifically 
discussed the mechanisms they have in place to ensure that their 
partners are achieving expected outcomes. 

[32] A process evaluation examines whether a program is operating as 
intended rather than whether it achieved the outcomes it is intended 
to achieve. 

[33] The Urban Institute released its guidance for grantees on 
collecting and reporting data in February 2013. 

[34] FERPA protects the privacy of student education records. 20 
U.S.C. § 1232g. The law applies to all schools that receive funds 
under an applicable program of the U.S. Department of Education. Under 
FERPA, schools generally must have written consent from the parents or 
eligible student in order to release information from a student's 
education record. 

[35] HIPAA and its implementing regulations provide federal 
protections for individually identifiable health information and give 
patients an array of rights with respect to that information. Pub. L. 
No. 104-191, 110 Stat. 1936 (1996); 45 C.F.R. Parts 160, 164. These 
provisions generally require that health insurers and providers must 
have consumers' permission before their health information can be used 
or shared for certain purposes, such as marketing. 

[36] GAO, Managing for Results: Implementation Approaches Used to 
Enhance Collaboration in Interagency Groups, [hyperlink, 
http://www.gao.gov/products/GAO-14-220] (Washington, D.C.: Feb. 14, 
2014). To identify the approaches we selected four interagency groups 
that use key practices for enhancing and sustaining collaboration. To 
identify successful approaches, we reviewed agency documents and 
interviewed agency officials that participated in these groups. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO's commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
"E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO's actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO's 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]