This is the accessible text file for GAO report number GAO-13-581 
entitled 'Rental Housing Assistance: HUD Data on Self-Sufficiency 
Programs Should Be Improved' which was released on August 8, 2013. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Congressional Requesters: 

July 2013: 

Rental Housing Assistance: 

HUD Data on Self-Sufficiency Programs Should Be Improved: 

GAO-13-581: 

GAO Highlights: 

Highlights of GAO-13-581, a report to congressional requesters. 

Why GAO Did This Study: 

HUD reported in 2011 that nearly 8.5 million lower-income families 
paid more than half their monthly income for rent, lived in 
substandard housing, or both. As the number of those needing 
assistance is greater than existing federal programs can serve, if 
families were able to increase their income and no longer require 
housing assistance, spaces could become available for other needy 
families. HUD offers several competitive grants that PHAs can use to 
hire staff who link residents to services or implement programs that 
encourage self-sufficiency. GAO was asked to examine the effectiveness 
of HUD’s efforts to promote self-sufficiency among residents. Among 
its objectives, this report describes (1) costs and resident 
participation in HUD grant programs for PHAs that encourage work and 
self-sufficiency and (2) available information on the programs’ 
effects on residents. 

GAO reviewed HUD’s goals for encouraging self-sufficiency, program 
descriptions, and regulations; analyzed grant award data for fiscal 
years 2006-2011 and available outcome information; and interviewed HUD 
and PHA officials. 

What GAO Found: 

The Department of Housing and Urban Development (HUD) funds five key 
grant programs that encourage resident self-sufficiency. In fiscal 
year 2011, HUD awarded $113 million to the Housing Choice Voucher 
Family Self-Sufficiency (FSS), Public Housing FSS, and Resident 
Opportunity and Self-Sufficiency Service Coordinators (ROSS SC) 
programs. Public housing agencies (PHA) with HOPE VI grants or 
designated as Moving to Work (MTW) agencies spent a portion of their 
funds on activities that encourage self-sufficiency, but the amounts 
MTW agencies spent are not known for the program as a whole. 
Additionally, data on resident participation in the five programs were 
limited. The number of families that participated in the FSS programs 
and ROSS SC cannot be reliably assessed due to missing start dates, 
end dates, and annual updates, and a lack of reporting guidance. HOPE 
VI data on residents’ participation does not include information on 
the elderly or persons with disabilities. Programwide MTW data on 
participation generally were unavailable. Internal control standards 
for the federal government state that program managers need 
operational data to determine whether they are meeting goals for 
accountability (effective and efficient use of resources). Without 
complete participation data, HUD lacks key information to effectively 
manage and evaluate its programs and Congress lacks data needed to 
oversee the programs. 

HUD lacks a strategy for using data it requires of PHAs to expand what 
is known about outcomes in four of the programs. HUD has performed 
limited analysis of the data related to self-sufficiency outcomes for 
both types of FSS grants reported into its information systems. HUD 
has not analyzed similar data reported for ROSS SC and MTW activities. 
However, for HOPE VI HUD collects consistent, outcome-based measures 
for participation in self-sufficiency activities and uses the data to 
track residents’ progress towards self-sufficiency. Internal control 
standards underline the importance not only of collecting but also 
using information to achieve programmatic goals. Also, the GPRA 
Modernization Act of 2010 (GPRAMA) emphasizes the need for information 
on the effectiveness of federal programs to improve congressional 
decision making. A strategy for using these data could inform overall 
management review, congressional oversight, and planning for these 
programs. Using such data could help HUD identify from which PHAs to 
draw lessons to help improve HUD management of the programs as well as 
PHA management of self-sufficiency-related activities. GAO’s analysis 
of available data on residents who participated in the FSS programs 
suggests positive changes for those who completed the programs, but 
the results are not conclusive because data indicating whether a 
family exited FSS or subsidized housing were missing for 35 percent of 
families that started an FSS program in 2006. 

What GAO Recommends: 

For three of its self-sufficiency programs, HUD should develop 
processes and program-specific reporting guidance to better ensure 
required data on participation and outcomes are complete. HUD agreed 
with three recommendations but disagreed that it should analyze data 
for the ROSS SC program. GAO believes that analysis of program data is 
critical for assessing outcomes. 

View GAO-13-581. For more information, contact Mathew Scirè at (202) 
512-8678 or ScireMJ@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

Participation Information for Self-Sufficiency Programs Was Limited: 

HUD Has Not Fully Utilized the Data It Collects on Self-Sufficiency 
Programs: 

HUD Encourages PHAs to Coordinate with Local Service Providers: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Comments from the Department of Housing and Urban 
Development: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Program Features for the Family Self-Sufficiency and Service 
Coordinators Programs: 

Table 2: Total Amounts Awarded to PHAs through FSS and ROSS SC 
Programs, Fiscal Years 2006-2011 (dollars in millions): 

Table 3: FSS Escrow Disbursements, Fiscal Years 2006-2011 (dollars in 
millions): 

Table 4: Proportion of HOPE VI Revitalization Grant Funds Spent on 
Community and Supportive Services, for Grants Awarded in Fiscal Years 
2006-2010 (dollars in millions): 

Table 5: FSS Participation Data Posted by HUD: 

Table 6: Data Reported by HOPE VI Grantees on the Number of Residents 
Who Enrolled in and Completed Community and Supportive Service 
Programs, as of December 2012: 

Table 7: HUD Information on Experiences of a Subset of Families That 
Started FSS in Fiscal Year 2006 and for Which Data Were Available on 
Program Completion, Exit, or Extension, as of December 2012: 

Table 8: Income and Employment Changes for Families that Began FSS in 
Fiscal Year 2006 and Had Completed It by Fiscal Year 2011: 

Table 9: Examples of Ways That HUD Encourages or Requires PHAs to 
Coordinate with Local Agencies to Improve Resident Self-Sufficiency: 

Figure: 

Figure 1: FSS Escrow Account Example: 

Abbreviations: 

FSS: Family Self-Sufficiency: 

GPRA: Government Performance and Results Act: 

GPRAMA: GPRA Modernization Act of 2010: 

HCV FSS: Housing Choice Voucher Family Self-Sufficiency program: 

HUD: Department of Housing and Urban Development: 

MOU: Memorandum of Understanding: 

MTW: Moving to Work: 

OMB: Office of Management and Budget: 

NOFA: notice of funding availability: 

PH FSS: Public Housing Family Self-Sufficiency program: 

PHA: public housing agency: 

PIC: Public and Indian Housing Information Center: 

QHWRA: Quality Housing and Work Responsibility Act of 1998: 

ROSS SC: Resident Opportunity and Self-Sufficiency Service 
Coordinators program: 

WIA: Workforce Investment Act: 

[End of section] 

GAO:
United States Government Accountability Office: 
441 G St. N.W. 
Washington, DC 20548: 

July 9, 2013: 

The Honorable Shelley Moore Capito: 
Chairman: 
Subcommittee on Financial Institutions and Consumer Credit: 
Committee on Financial Services: 
House of Representatives: 

The Honorable Randy Neugebauer: 
Chairman: 
Subcommittee on Housing and Insurance: 
Committee on Financial Services: 
House of Representatives: 

The Department of Housing and Urban Development (HUD) provided more 
than $33 billion in rental assistance in fiscal year 2012 to millions 
of low-income households through a variety of programs. These 
programs--primarily housing choice vouchers, project-based rental 
assistance, and public housing--serve an estimated 5.4 million low-
income households.[Footnote 1] However, HUD reported in 2011 that 
nearly 8.5 million lower-income families paid more than half their 
monthly income for rent, lived in severely substandard housing, or 
both. As the number of those needing assistance is greater than can be 
served by existing federal programs, if families were able to increase 
their income and move out of assisted housing, spaces could become 
available to assist other needy families. 

The Quality Housing and Work Responsibility Act of 1998 (QHWRA) 
requires public housing agencies (PHA), which administer public 
housing and voucher programs, to establish policies and practices that 
encourage all but elderly residents and residents with disabilities to 
work and become more economically self-sufficient.[Footnote 2] For 
example, QHWRA requires PHAs to establish minimum rent policies (with 
exceptions for hardship circumstances). For the public housing 
program, it also requires nonelderly, nondisabled residents to work or 
participate in community service or a self-sufficiency program for 8 
hours a month; requires PHAs to establish rent policies that encourage 
and reward employment and economic self-sufficiency; and allows PHAs 
to phase-in rent increases 1 year after public housing residents find 
employment. To encourage resident self-sufficiency, PHAs can apply for 
several HUD grants. Some of these grants provide funds for PHA staff 
to link residents to the services that they need to become 
increasingly self-sufficient. Other grant programs enable selected 
PHAs to establish their own policies--such as rent policies--that 
encourage work and self-sufficiency. 

You asked us to review the effectiveness of HUD programs that 
encourage resident self-sufficiency. This report (1) discusses what is 
known about the costs of and residents' participation in HUD grant 
programs that encourage work and self-sufficiency, (2) determines what 
is known about the effect on residents of HUD's grant programs, and 
(3) describes steps HUD has taken to coordinate with other federal 
agencies and increase residents' access to non-HUD programs that 
encourage work and self-sufficiency. 

To describe what is known about the costs of HUD grant programs that 
encourage self-sufficiency, we reviewed documentation of HUD's 
programs and determined whether and how they defined and addressed 
resident self-sufficiency. Because HUD did not have an agencywide 
definition of self-sufficiency, we established criteria for what 
constitutes a self-sufficiency program, and identified the Housing 
Choice Voucher Family Self-Sufficiency (FSS), Public Housing Family 
Self-Sufficiency, Resident Opportunity and Self-Sufficiency Service 
Coordinators (ROSS SC), Moving to Work (MTW), and HOPE VI programs. 
HUD officials agreed that we had identified the key grant programs 
that encourage work and self-sufficiency. We then interviewed HUD 
officials about costs associated with the five programs, and reviewed 
documentation of grant awards for the programs. To assess resident 
participation, we determined how HUD collects program participation 
data and analyzed data from HUD's Public and Indian Housing 
Information Center (PIC) system and the HOPE VI Quarterly Reporting 
System. To determine what is known about the effects of the five 
programs on residents' self-sufficiency, we determined how HUD 
collects data on indicators of self-sufficiency for each of the five 
programs and interviewed HUD officials about how they use the data. 
For the Family Self-Sufficiency (FSS) programs, we analyzed changes in 
indicators of self-sufficiency for a cohort of participants who 
completed the programs. We also summarized available research on self-
sufficiency-related outcomes for four programs. To describe how HUD 
has coordinated with other federal agencies to increase residents' 
access to non-HUD programs, we reviewed HUD regulations and policies 
related to such coordination and interviewed officials from PHAs 
administering each HUD program in our review. For all three 
objectives, we interviewed HUD officials and officials from a sample 
of five PHAs and research organizations. We determined, based on 
interviews with HUD officials and our own analyses, that the fiscal 
years 2006 through 2011 data were not sufficiently complete to enable 
us to reliably analyze the total number of families participating over 
time and their self-sufficiency-related outcomes. Further, for those 
families that began either of the FSS programs in 2006, we described 
outcomes for those families for which HUD's data indicated that they 
successfully completed the program. HUD's data for approximately one-
third of these families did not indicate whether the family 
successfully completed the program or otherwise exited the public 
housing or voucher programs. Additionally, we determined based on 
interviews with HUD officials and our own analysis that HUD data on 
the extent to which MTW families participated in FSS was not reliable 
prior to 2011. Appendix I provides further details on our scope and 
methodology. 

We conducted this performance audit from August 2012 through July 2013 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on the audit objectives. 

Background: 

HUD housing serves populations that include persons in a position to 
increase their self-sufficiency and those who need long-term support 
(for example, the elderly and persons with severe disabilities). To 
assist those who can improve their self-sufficiency, HUD permits PHAs 
to allocate space in its public housing to offer training programs and 
information sessions. It also awards grants to PHAs to encourage them 
to work with local social service providers in offering education, job 
training, and other supportive services to residents of public housing 
and recipients of vouchers. The purpose of these programs is to 
increase the earned incomes of residents and reduce their reliance on 
cash assistance and housing assistance. 

HUD-Funded Self-Sufficiency Programs: 

For this report, we examined five HUD programs that fund self-
sufficiency efforts in whole or in part. Three grant programs only 
award grants to support self-sufficiency activities: HCV FSS, PH FSS, 
and ROSS SC.[Footnote 3] FSS was authorized in 1990 to help families 
receiving vouchers or in public housing become self-sufficient through 
referrals to education, training, and other supportive services and 
case management.[Footnote 4] PHAs use grant funds to pay program 
coordinators who link residents to the supportive services they need 
to achieve self-sufficiency.[Footnote 5] Families in either FSS 
program sign a contract with their PHA, requiring that all family 
members become independent of cash welfare assistance and the head of 
the family seek and maintain employment.[Footnote 6] Both programs 
feature case management, referrals to supportive services, and an 
escrow account that accumulates balances, or savings, for the tenant 
based on increases in tenant contributions toward rent. The ROSS SC 
program provides funding to hire service coordinators to assess the 
needs of public housing residents and coordinate available resources 
in the community to meet those needs.[Footnote 7] 

Table 1: Program Features for the Family Self-Sufficiency and Service 
Coordinators Programs: 

Program: Housing Choice Voucher FSS; 
Features that encourage self-sufficiency: 
Escrow account; 
Family generally must remain independent from welfare for at least 1 
year to obtain funds in escrow account; 
Eligible recipients: Public housing agencies that administer housing 
choice vouchers; 
Intended beneficiaries: Housing choice voucher recipients; 
Funding terms and uses: One-year funding for coordinator salaries. 

Program: Public Housing FSS; 
Features that encourage self-sufficiency: 
Family generally must remain independent from welfare for at least 1 
year to obtain funds in escrow account; 
Family head must become employed; 
Case management; 
Eligible recipients: Public housing agencies that administer public 
housing; Tribes/Tribally Designated Housing Entities; 
Intended beneficiaries: Public housing and Indian housing residents; 
Funding terms and uses: One-year funding for coordinator salaries. 

Program: Resident Opportunity and Self-Sufficiency Service 
Coordinators; 
Features that encourage self-sufficiency: Referrals to services that 
enable families to increase earned income, reduce or eliminate need 
for welfare assistance, and make progress towards economic 
independence, or the case of the elderly or disabled, live 
independently; 
Eligible recipients: Public housing agencies; Tribes/Tribally 
Designated Housing Entities; Resident associations; Nonprofit 
organizations; 
Intended beneficiaries: Public housing and Indian housing residents; 
Funding terms and uses: Three-year funding for coordinator salary, 
training, and administrative expenses; Requires matching funds, in-
kind donations totaling 25 percent of the requested grant amount, or 
both. 

Source: GAO analysis of HUD information. 

[End of table] 

For the FSS programs, the escrow accounts are incentives to increase 
work effort and earnings. Specifically, when participants have to pay 
a higher rent after their earned income increases, the PHA calculates 
an escrow credit that is deposited each month into an interest-bearing 
account (see figure 1). Families that successfully complete their 
contract for either FSS program receive their accrued escrow funds. In 
general, in order to complete an FSS contract, the family head must 
certify that no member of the family has received welfare for at least 
one year, and the family head must be employed.[Footnote 8] 

Figure 1: FSS Escrow Account Example: 

[Refer to PDF for image: illustration] 

Monthly payments while tenant has no earned income: 

Payments: 
Tenant: $100; (Tenant has no earned income); 
PHA: $400 (subsidy); 
Rent owed: $500. 

Monthly payments after tenant finds employment and has earned income: 

Payments: 
Tenant: $300; (Tenant finds employment and has earned income); 
PHA: $400 (subsidy); 
Rent owed: $500. 

Escrow account: 
No monthly deposits in escrow while payments equal rent; 
Amount in excess of rent deposited monthly in escrow. 

Escrow outcome possibilities: 
* Withdrawn by resident during program (subject to approval); 
* Disbursed to resident at end of program; 
* Forfeited by resident due to noncompletion of program. 

Source: GAO. 

Note: The figure presents a hypothetical example. Individual FSS 
participant experiences may vary based on factors, including level of 
income at program enrollment and changes in employment and income 
during program participation. 

[End of figure] 

HUD has two other programs--MTW and HOPE VI Revitalization Grants 
(HOPE VI)--that allow participating PHAs to determine how they will 
encourage self-sufficiency.[Footnote 9] The purposes of HUD's MTW 
demonstration program include providing PHAs the flexibility to design 
and test policies that give incentives to families with children to 
become economically self-sufficient.[Footnote 10] For example, PHAs in 
MTW can alter eligibility and rent policies. Through the HOPE VI 
program, participating PHAs (HOPE VI agencies) can use HOPE VI grants 
to demolish, rehabilitate, or replace severely distressed public 
housing and also provide community and supportive services to help 
residents achieve self-sufficiency.[Footnote 11] HOPE VI agencies can 
spend up to 15 percent of their revitalization grant funds for 
community and supportive services activities, which HUD defines as any 
activity designed to promote upward mobility, self-sufficiency, and 
improved quality of life for residents of the public housing project 
involved.[Footnote 12] 

HUD Indicators of Self-Sufficiency: 

For the two FSS programs, HUD regulations state that self-sufficiency 
means that a family is no longer receiving housing assistance or 
welfare assistance.[Footnote 13] However, the regulations also 
indicate that while achieving self-sufficiency is an FSS objective, no 
longer receiving housing assistance is not a requirement for 
completing the program and receiving escrow funds.[Footnote 14] HUD 
does not define self-sufficiency for the other three programs we 
reviewed. Thus, HUD does not have a uniform measure of self-
sufficiency across the programs. HUD officials with responsibility for 
administering all five programs noted that a single definition of self-
sufficiency would not be useful, particularly because levels of self-
sufficiency could differ for different groups. However, officials 
noted that the concept is commonly understood to indicate that a 
family does not rely on government programs that are intended to 
address poverty. 

Other HUD documents suggest additional indicators for self-
sufficiency. For instance, HUD's fiscal years 2010 through 2015 
strategic plan emphasizes the importance of increasing economic 
security and self-sufficiency, but does not define these terms. 
[Footnote 15] The plan notes that increasing the average income of HUD-
assisted households is an indicator of success towards these goals, 
and that increasing self-sufficiency requires access to life-skills 
training, wealth-creation and asset-building opportunities, job 
training, and career services. HUD program information, such as NOFAs, 
also describes indicators of progress towards self-sufficiency. These 
indicators include the number of families with members who obtained 
their first job, or obtained higher-paying jobs; no longer needed 
benefits under one or more welfare programs; obtained a high school 
diploma or higher education degree; or accomplished similar goals that 
would assist the family in obtaining economic independence. To 
graduate from either FSS program and obtain any escrowed funds, 
families generally must remain independent from welfare assistance for 
at least 1 year. Similarly, for the ROSS SC program, NOFAs state that 
the services provided to participating families should help them 
increase earned income, reduce or eliminate the need for welfare 
assistance, or progress toward economic independence and housing self-
sufficiency. In the case of elderly residents or residents with 
disabilities, services should help improve living conditions and 
enable residents to age in-place. For the HOPE VI program, indicators 
of progress towards self-sufficiency include the number of residents 
who obtained a high school or equivalent education, obtained new 
employment, and completed a job training program. 

In contrast, MTW agencies determine their own measures of residents' 
progress towards self-sufficiency. In a 2012 report on the MTW 
program, we reported that HUD had not defined key program terms, 
including the statutory purpose of encouraging employment and self-
sufficiency, and recommended that HUD issue guidance that clarifies 
such terms.[Footnote 16] We also noted the limited usefulness of 
having MTW agencies devise their own metrics, particularly when they 
are not outcome-oriented. We recommended that HUD improve its guidance 
to MTW agencies by requiring that they provide performance information 
that is quantifiable and outcome-oriented. HUD has taken steps to 
revise its reporting guidance, and the Office of Management and Budget 
(OMB) approved revised guidance on May 31, 2013. 

Data Collection: 

HUD requires all PHAs to collect detailed data from their residents. 
PHAs must submit these data electronically through the PIC at least 
annually for each household that participates in assisted housing 
programs. According to HUD guidance, PIC can be used to create 
reports, analyze programs, monitor PHAs, and provide information about 
those that live in HUD-subsidized housing. The data collected include 
amounts and sources of income; the amount of rent paid; and the 
presence of household heads and members who are elderly or have 
disabilities. MTW agencies can submit some tenant-related data into a 
separate module in PIC called MTW-PIC, which was created in 2007 to 
better accommodate some of the activities allowed under MTW. Most MTW 
agencies had transitioned to it by 2008. 

PHAs that receive either of the FSS grants, including MTW agencies, 
must report additional data about participating households into a 
section of the PIC and MTW-PIC systems called the FSS Addendum. The 
addendum captures data on each family when it enters either of the two 
FSS programs and should be updated annually. Specifically, the 
addendum includes fields for PHAs to report whether the head of the 
household works full-time, part-time, or is unemployed; the highest 
grade of education for the head of the household; and the sources of 
assistance received by the family, such as cash or food assistance. 

Participation Information for Self-Sufficiency Programs Was Limited: 

Available Data Indicate That Annual Amounts of Grant Awards in Three 
Self-Sufficiency Programs Increased: 

The annual totals for grant awards in the two FSS and ROSS SC programs 
have increased in recent years. As shown in table 2, the amount HUD 
awarded to PHAs in constant 2013 dollars through the two FSS programs 
increased from about $64 million in fiscal year 2006 to about $78 
million in fiscal year 2011, with the majority of funds being awarded 
through the HCV FSS program. For the ROSS SC program, the total amount 
awarded, in constant 2013 dollars, increased from $30 million in 
fiscal year 2008 to $35 million in fiscal year 2011.[Footnote 17] 

Table 2: Total Amounts Awarded to PHAs through FSS and ROSS SC 
Programs, Fiscal Years 2006-2011: 

Program: HCV FSS; 
2006: $54 million; 
2007: $52 million; 
2008: $53 million; 
2009: $53 million; 
2010: $63 million; 
2011: $62 million; 
Total: $337 million. 

Program: PH FSS; 
2006: $10 million; 
2007: $12 million; 
2008: $13 million; 
2009: $13 million; 
2010: $16 million; 
2011: $16 million; 
Total: $80 million. 

Program: ROSS SC; 
2006: n/a; 
2007: n/a; 
2008: $30 million; 
2009: $34 million; 
2010: $34 million; 
2011: $35 million; 
Total: $133 million. 

Source: HUD. 

Notes: Award amounts for PH FSS and ROSS SC may include amounts drawn 
from previous years' appropriations. All dollar amounts are adjusted 
to fiscal year 2013 dollars using the chain-weighted Gross Domestic 
Product price index and rounded to the nearest million. We used "n/a" 
(not applicable) for the 2006 and 2007 data for ROSS SC because while 
HUD awarded funds for a different version of the program in 2006 and 
2007, ROSS SC was implemented in its current form in 2008. 

[End of table] 

Escrow payments to households represent additional program 
expenditures associated with the two FSS programs. As previously 
described, PHAs establish escrow accounts for households that 
participate in FSS. PHAs generally disburse the amount in the escrow 
account (the excess of any amount owed to the PHA by the family) to 
the head of the family when the contract of participation has been 
completed.[Footnote 18] As previously noted, in order to successfully 
complete an FSS contract, the head of the family must be employed and 
no members may be receiving welfare. According to HUD data adjusted to 
2013 dollars, the agency disbursed nearly $82 million to FSS 
participants between fiscal years 2006 and 2011 (see table 3). 

Table 3: FSS Escrow Disbursements, Fiscal Years 2006-2011: 

Program: HCV FSS amount disbursed; 
2006: $14 million; 
2007: $14 million; 
2008: $14 million; 
2009: $12 million; 
2010: $11 million; 
2011: $10 million; 
Total amount disbursed: $76 million. 

Program: PH FSS amount disbursed; 
2006: $1 million; 
2007: $1 million; 
2008: $1 million; 
2009: $1 million; 
2010: $1 million; 
2011: $1 million; 
Total amount disbursed: $6 million. 

Program: Total; 
2006: $15 million; 
2007: $15 million; 
2008: $15 million; 
2009: $13 million; 
2010: $12 million; 
2011: $11 million; 
Total amount disbursed: $82 million. 

Source: GAO analysis of HUD data. 

Notes: All dollar amounts are adjusted to fiscal year 2013 dollars 
using the chain-weighted Gross Domestic Product price index, and 
rounded to the nearest million. Sum of disbursements may not equal the 
total column due to rounding. This analysis excludes MTW agencies. 

[End of table] 

Expenditures for community and supportive services, adjusted to 2013 
dollars, represented 7 percent or less of HOPE VI grant awards during 
fiscal years 2006-2010. HUD data indicate that the 29 PHAs that were 
awarded HOPE VI revitalization grants between fiscal years 2006 and 
2010 had spent about 5 percent of these funds on community and 
supportive services as of December 2012 (see table 4). Of the 29 
grants, 4 were closed and 25 were open. HOPE VI agencies with open 
grants could spend more of their revitalization grant on supportive 
services, up to 15 percent. HOPE VI agencies also can leverage other 
sources of funding for community and supportive services. 

Table 4: Proportion of HOPE VI Revitalization Grant Funds Spent on 
Community and Supportive Services, for Grants Awarded in Fiscal Years 
2006-2010: 

Fiscal year: 2006; 
Number of revitalization grants (status as of December 2012): 
Closed grants: 4; 
Open grants: n/a; 
Total actual expenditures for community and supportive services (as of 
December 2012): $6 million; 
Total grant award amount: $82 million; 
Proportion of award amounts expended on community and supportive 
services (as of December 2012): 7%. 

Fiscal year: 2007; 
Number of revitalization grants (status as of December 2012): 
Closed grants: n/a; 
Open grants: 5; 
Total actual expenditures for community and supportive services (as of 
December 2012): $7 million; 
Total grant award amount: $98 million; 
Proportion of award amounts expended on community and supportive 
services (as of December 2012): 7%. 

Fiscal year: 2008; 
Number of revitalization grants (status as of December 2012): 
Closed grants: n/a; 
Open grants: 6; 
Total actual expenditures for community and supportive services (as of 
December 2012): $5 million; 
Total grant award amount: $105 million; 
Proportion of award amounts expended on community and supportive 
services (as of December 2012): 5%. 

Fiscal year: 2009; 
Number of revitalization grants (status as of December 2012): 
Closed grants: n/a; 
Open grants: 6; 
Total actual expenditures for community and supportive services (as of 
December 2012): $5 million; 
Total grant award amount: $121 million; 
Proportion of award amounts expended on community and supportive 
services (as of December 2012): 4%. 

Fiscal year: 2010; 
Number of revitalization grants (status as of December 2012): 
Closed grants: n/a; 
Open grants: 8; 
Total actual expenditures for community and supportive services (as of 
December 2012): $3 million; 
Total grant award amount: $161 million; 
Proportion of award amounts expended on community and supportive 
services (as of December 2012): 2%. 

Fiscal year: Total; 
Number of revitalization grants (status as of December 2012): 
Closed grants: 4; 
Open grants: 25; 
Total actual expenditures for community and supportive services (as of 
December 2012): $26 million; 
Total grant award amount: $568 million; 
Proportion of award amounts expended on community and supportive 
services (as of December 2012): 5%. 

Source: GAO analysis of HUD data. 

Notes: The last HOPE VI revitalization grant NOFA was for fiscal year 
2010. All dollar amounts are adjusted to fiscal year 2013 dollars 
using the chain-weighted Gross Domestic Product price index and 
rounded to the nearest million. 

[End of table] 

The amounts that MTW agencies spend on activities intended to increase 
resident self-sufficiency are not known for the program as a whole. 
MTW agencies are not required to expend a specific proportion of their 
HUD funds on activities that encourage work and self-sufficiency. 
However, they must implement activities that address the program's 
statutory purposes (which include encouraging work and self-
sufficiency), and annually submit written reports to HUD with 
information on each activity they undertake and its linkage to the 
program's statutory purposes. MTW agencies had been required to 
annually report to HUD financial data on the sources and uses of their 
funds. A HUD official with responsibility for administering the MTW 
program stated that the agency had not analyzed this information and 
that MTW agencies had reported it using varying formats. According to 
HUD, with the implementation of revised MTW reporting requirements, 
MTW agencies will be required to report data on the costs of self-
sufficiency activities using standardized metrics. 

Program Participation Not Comprehensively Known Because of Data 
Limitations: 

Residents' participation in the five self-sufficiency programs was not 
comprehensively known because the data were not reliable, aggregated 
programwide, or collected for all participants. Internal control 
standards state that transactions should be promptly recorded to 
maintain their relevance and value to management in controlling 
operations and making decisions.[Footnote 19] This applies to the 
entire life cycle of a transaction or event. They also state that 
program managers need operational data to determine whether they are 
meeting their agencies' strategic plans and meeting their goals for 
the effective and efficient use of resources. 

The Two FSS Programs: 

The total number of families in either of the two FSS programs cannot 
be reliably assessed based on available PIC data for fiscal years 2006-
2011 because of missing program start dates, exit dates, and annual 
updates.[Footnote 20] According to HUD guidance, PHAs that receive FSS 
grants must update information about each participating family at 
least annually. As a part of these updates, PHAs are supposed to 
indicate whether the family is active or has exited the program. For 
those who exited the program, PHAs are supposed to indicate whether 
they completed the program or stopped participating for other reasons. 
These data must be reported in the FSS addendum of PIC or MTW-PIC. 
NOFAs consistently have cited PIC as a data source for FSS program 
participation counts, which HUD uses in part to determine eligibility 
for funding. For some years, the notices also stated that applicants 
for grant funding could use other documentation for participation 
counts, such as a separate HUD form or MTW documents. [Footnote 21] 

However, we observed that many PIC records lacked information on when 
and why the participating family exited the program before completion. 
For example, while HUD's PIC data indicate that 14,690 families began 
participating in the two FSS programs in fiscal year 2006, data on 
exits for these 5-year programs were missing for nearly half of these 
families. For 11 percent of the families that began the FSS programs 
in 2006, HUD data do not indicate the families had exited the 
programs, although other HUD data indicated that the families no 
longer received housing assistance. Similarly, HUD data also indicate 
that 11,596 families exited the two programs in fiscal year 2011 (some 
completing the program and others leaving before completion), but did 
not indicate a start date for about one quarter of these families. 
[Footnote 22] Because both FSS programs are intended to be 5-year 
programs, it is reasonable to expect that most of the families that 
began participating in fiscal year 2006 would have exited or completed 
the program by fiscal year 2011.[Footnote 23] 

HUD began posting FSS participation data from the PIC system online 
beginning with the fiscal year 2009 funding announcement for HCV FSS 
and the fiscal year 2012 funding announcement for PH FSS. According to 
these data, the number of families participating in HCV FSS has 
increased in recent years (see table 5). 

Table 5: FSS Participation Data Posted by HUD: 

Fiscal year of NOFA: 2009; 
HCV FSS participants: 54,972; 
PH FSS participants: n/a. 

Fiscal year of NOFA: 2010; 
HCV FSS participants: 56,491; 
PH FSS participants: n/a. 

Fiscal year of NOFA: 2011; 
HCV FSS participants: 56,915; 
PH FSS participants: n/a. 

Fiscal year of NOFA: 2012; 
HCV FSS participants: 68,548; 
PH FSS participants: 14,386. 

Source: HUD. 

Notes: According to HUD officials, the posted participation data for 
fiscal years 2009 through 2011 do not consistently include MTW 
agencies' data due to the data reliability issues discussed in this 
report. We used "n/a" (not applicable) for the 2009-2011 data for PH 
FSS because HUD did not post participation online for the PH FSS 
program until the fiscal year 2012 funding round. 

[End of table] 

HUD officials told us that posting participation data online has 
emphasized to PHAs the importance of ensuring the data they submit 
into PIC are accurate. If a PHA believed the participation number HUD 
posted was incorrect and that it would be underfunded or ineligible, 
that PHA could submit documentation to confirm a higher number. 
[Footnote 24] However, as a part of the NOFA process, neither HUD nor 
the PHA were required to make corrections to the PIC system.[Footnote 
25] Officials from one of the PHAs with whom we met stated that the 
enrollment data in PIC generally are inconsistent with their internal 
records. The staff stated that they have worked with HUD to correct 
PIC, but noted that once some errors were fixed, new errors emerged. 
Staff from another PHA, which is also a MTW agency, stated that their 
FSS enrollment data do not appear in PIC; thus, they must create ad 
hoc reports using their own internal systems. 

According to HUD officials with responsibility for implementing the 
FSS programs, the FSS records could be incomplete or incorrect for 
several reasons. For example, if a participating family left public 
housing or the voucher program, the responsible PHA might not update 
the FSS portion of PIC to reflect the departure. PHAs are supposed to 
enter the exit date, whether the family completed their contract of 
participation, or one of five primary reasons for leaving the program 
without completing it. According to these officials, no HUD staff have 
specific responsibility for monitoring the completeness of FSS 
participants' records in PIC. HUD officials also stated that there are 
challenges with the PIC system. They were aware of cases in which PHAs 
have entered data into PIC, but either the entries were not saved or 
they overwrote previously entered data. PHA officials with whom we 
met, as well as HUD staff with responsibility for the PIC system, 
stated that PHAs cannot readily run reports that show FSS 
participation data, a feature which would enable them to identify and 
correct missing or incorrect data. While PHAs can run ad hoc reports 
from PIC, the functionality of the system is limited, and records 
older than 18 months cannot be accessed. We previously reported on the 
weaknesses associated with HUD's antiquated technologies.[Footnote 26] 
Without complete information on families' participation in either FSS 
program (including a reliable count of program participants, 
participants' length of time in the program, and reasons participants 
do not complete the programs), HUD lacks accurate information to make 
grant funding decisions. Moreover, by not analyzing the extent to 
which PHAs have reported required data into PIC (and MTW-PIC), HUD's 
ability to effectively oversee the program is limited. 

ROSS SC: 

Participation data for the ROSS SC program from fiscal years 2008 (the 
year the current version of this program started) through 2011 also 
were limited, primarily because of the lack of reporting guidance for 
the program and the difficulty of aggregating the data. HUD does not 
require PHAs to report whether a resident received services through 
this program in its information systems; rather, HUD collects ROSS SC 
participation data from PHAs through individual Excel-based reporting 
tools.[Footnote 27] According to HUD officials with responsibility for 
the PIC and MTW-PIC systems, these systems were intended to collect 
resident characteristics and not to enable PHAs to report data on 
residents' participation in specific programs, such as ROSS SC. PHAs 
must use the tool to apply for grant funding, inserting projections 
for the number of residents or households they intend to serve. Grant 
recipients then must submit annual updates, reporting on the number of 
residents or households actually assisted. Internal control standards 
state that transactions should be promptly recorded to maintain their 
relevance and value to management in controlling operations and making 
decisions. They also state that program managers need operational data 
to determine whether they have been meeting their agencies' strategic 
plans and meeting their goals for the effective and efficient use of 
resources. As we previously reported, according to OMB, being able to 
track and measure specific program data can help agencies diagnose 
problems, identify drivers of future performance, evaluate risk, 
support collaboration, and inform follow-up actions. Analyses of 
patterns and anomalies in program information also can help agencies 
discover ways to achieve more value for the taxpayers' money.[Footnote 
28] 

According to HUD officials, the data that individual PHAs report using 
the Excel-based reporting tool cannot be easily or reliably aggregated 
to provide a count of resident participation across all PHAs. HUD 
officials who manage the ROSS SC program did not attempt to aggregate 
data reported through individual reporting tools until 2013 in 
response to our review. While HUD provided us with annual program 
participation data based on the tools, we determined that the data 
were not sufficiently reliable for these purposes. Specifically, we 
found duplicate records; differences in the time periods for which 
PHAs were reporting data; and different results for the "number of 
persons receiving services," the "number of persons served," and the 
sum of counts of persons served by age category. HUD officials told us 
that there are no line-level instructions for reporting these data. 
Thus, recipients of ROSS SC grants have not been given any formal 
guidance on what should be reported into these fields. Because of 
these limitations, HUD does not use the reporting tools to prepare 
official counts of residents assisted through the ROSS SC program. 
Rather, HUD multiplies the number of coordinators funded each year by 
50 (the minimum number of residents who must be assisted under the 
terms of the grant). Based on this formula, they can report on the 
minimum number of residents who likely were assisted. Using this 
approach, HUD estimated that from 6,450 to 7,800 residents received 
services through the ROSS SC program annually from fiscal years 2008 
through 2011. While this estimate provides information on the minimum 
number of residents who likely received assistance, it is not based on 
a count of the number of residents who actually received assistance. 

HUD officials who manage the ROSS SC program stated that the current 
process for collecting data from the reporting tools was supposed to 
be temporary, and a planned web-based replacement was never developed. 
Additionally, the officials stated that the data PHAs report into the 
tool were not always comparable because PHAs have varying 
interpretations of what they should report, and HUD never developed 
program-specific reporting guidance. Without developing a reliable 
process for collecting and analyzing data on the number of residents 
assisted through the ROSS SC program, HUD lacks basic information 
needed to manage the program. 

HOPE VI: 

According to HUD participation data for HOPE VI (available on a 
cumulative basis), at least 73,000 working-age, nondisabled residents 
have participated in the community and supportive services component 
of the HOPE VI program since the program began in 1993.[Footnote 29] 
For the 280 revitalization grants awarded through the program since 
its inception, HOPE VI agencies self-reported that about 55,000 
original residents (those who lived at the site before revitalization) 
participated in a program or service designed to help them progress 
towards self-sufficiency.[Footnote 30] In addition, HOPE VI agencies 
self-reported that nearly 18,000 additional individuals who became 
residents at the revitalized HOPE VI sites also participated in 
community and supportive services. 

MTW: 

Programwide data on residents' participation in MTW activities related 
to increasing self-sufficiency from fiscal years 2006 through 2011 
generally were unavailable. While MTW-PIC was created in 2007 to 
better fit the needs of MTW agencies, this system was not designed to 
collect activity-level data (unless a household was participating in 
one of the two FSS programs). Moreover, HUD officials do not consider 
the data in MTW-PIC to be reliable prior to 2011 because some MTW 
agencies were still transitioning to it through the end of 2010. As a 
result, officials with responsibility for administering the MTW 
program have not used MTW-PIC as a tool for analyzing residents' 
participation in activities related to increasing self-sufficiency. In 
addition, HUD does not analyze the data that MTW agencies provide in 
their annual MTW reports, including data on residents' participation 
in activities related to self-sufficiency, because reporting 
requirements do not call for the reporting of standardized data, such 
as the number of residents who found employment. 

We previously recommended that HUD develop and implement a plan for 
quantitatively assessing the effectiveness of similar activities, 
which would include activities related to encouraging self-
sufficiency.[Footnote 31] HUD agreed that quantitatively assessing the 
effectiveness of similar activities was an important step. HUD has 
made revisions to its reporting requirements, which were approved by 
OMB in May 2013. 

According to our analysis of available HUD data, 338,900 households 
received rental housing assistance from an MTW agency in fiscal year 
2011. Because HUD officials do not believe their PIC or MTW-PIC 
systems contain reliable data from MTW agencies prior to 2011, we did 
not attempt to analyze these data. Of the MTW households, 14,314 were 
reported as participating in the two FSS programs in fiscal year 2011. 
According to HUD officials, most MTW agencies participate in FSS, but 
the quality of their FSS data reporting is unknown. 

HUD Has Not Fully Utilized the Data It Collects on Self-Sufficiency 
Programs: 

HUD Collects Data on Indicators of Self-Sufficiency: 

HUD requirements for collecting data on indicators of self-sufficiency 
vary by program. HUD requires PHAs to collect and report into PIC or 
MTW-PIC certain types of detailed information on every resident of HUD-
assisted rental housing. For instance, PHAs collect data on the amount 
and sources of residents' income, including whether it is earned or 
provided through disability payments, Temporary Assistance for Needy 
Families, or other sources. PHAs are supposed to collect and report 
this type of information into HUD's PIC system at least annually. For 
residents affiliated with the MTW program, this information is 
reported into MTW-PIC, a separate module in the PIC system.[Footnote 
32] 

In addition to reporting basic demographic and income data on each 
resident, HUD requires PHAs that implement the five programs to report 
additional indicators of residents' progress towards self-sufficiency 
using information systems, an Excel-based reporting tool, and 
narrative reports. 

* FSS programs and ROSS SC. As discussed earlier in this report, HUD 
requires PHAs that receive FSS grants to enter additional data in the 
FSS Addendum section of PIC on each family upon entry into an FSS 
program and update this information annually. For example, PHAs enter 
whether the head of the household works full-time, part-time, or is 
unemployed; and the highest grade of education for the head of the 
household. For the two FSS programs and the ROSS SC program, HUD also 
requires PHAs to annually enter summary output and outcome data 
related to their residents' progress towards self-sufficiency into the 
previously described Excel-based reporting tool. Data fields in the 
tool include the number of households that increased their income, 
moved to nonsubsidized housing, and the number of residents that 
obtained a high school diploma. 

* HOPE VI. HUD requires participating agencies to submit quarterly 
reports of summary data on residents' progress towards self-
sufficiency into the HOPE VI Quarterly Reporting System. For example, 
HOPE VI agencies must report the total number of residents who 
participated in activities that facilitate self-sufficiency, including 
the numbers enrolled in counseling programs, job training, and General 
Education Development classes. 

* MTW. HUD requires MTW agencies to submit annual reports containing 
summary information about the impact of the activities intended to 
encourage resident self-sufficiency they have been implementing. The 
agencies were able to use metrics of their choosing at the time of our 
review. 

HUD Has Lacked a Strategy for Using Collected Data to Evaluate the 
Impact of Four Programs: 

HUD has performed limited analyses--to assess outcomes for the 
programs as a whole--of data related to self-sufficiency outcomes that 
FSS grant recipients must report into its information systems. In 
addition, HUD has not analyzed similar data that FSS, ROSS SC, and MTW 
agencies must report through other mechanisms to assess each program 
as a whole. Standards for internal control emphasize the need for 
federal agencies both to collect reliable information with which to 
manage their programs and review the integrity of performance 
measures. Moreover, these standards emphasize the need for program 
managers to collect operational data to determine whether they have 
been meeting their goals for the efficient and effective use of 
resources.[Footnote 33] Additionally, the GPRA Modernization Act of 
2010 (GPRAMA) emphasizes the need for information on the effectiveness 
of federal programs to improve congressional decision making.[Footnote 
34] 

FSS and the ROSS SC Programs: 

Based on the information that PHAs must report in PIC and the FSS 
Addendum in PIC, HUD has performed limited analyses of the data 
related to self-sufficiency outcomes for the two FSS programs. As 
previously discussed, FSS participation data were sometimes incomplete 
and therefore of questionable reliability. Specifically, HUD's data 
lacked start dates and annual updates for some FSS participants. 
Despite these limitations, HUD has used these data in its 
Congressional Budget Justifications. For example, in the 2014 
Congressional Budget Justification HUD reported that as of March 30, 
2012, a total of 57,087 families were enrolled in the two FSS 
programs. Without annually updated records for each family, HUD cannot 
reliably determine whether families were still active program 
participants. 

Additionally, HUD has not analyzed and reported on the experiences of 
all families that start the FSS programs, including the extent to 
which they completed the program, the primary reasons they exited the 
program without completing it, and the extent to which required data 
are missing. According to HUD, they have not conducted such analysis 
due their own concerns about the usefulness of available data. As 
described previously, internal control standards state that such 
information can help agencies determine whether they have been meeting 
operational goals and using resources effectively. According to HUD 
officials with responsibility for administering the two FSS programs, 
they were aware that some PHAs did a better job of reporting into PIC 
than others, which affected the completeness and reliability of the 
data. We acknowledge that analyzing this information and summarizing 
overall changes in indicators of self-sufficiency among participants 
would not yield definitive results on the impact of FSS on resident 
self-sufficiency. But by not analyzing the data that FSS grant 
recipients must report in HUD's information systems more thoroughly, 
HUD has been missing an opportunity to gain valuable information about 
the results of FSS programs for certain agencies. That is, for those 
agencies that have submitted complete data (all annual updates as well 
as information on whether and how the participant exited the program), 
HUD could review available information, such as changes in income and 
employment. Also, HUD has been missing an opportunity to identify PHAs 
with notably effective or ineffective FSS programs (or data reporting) 
and learn from their experiences. Ultimately, HUD's lack of complete 
data on FSS participants limits the usefulness of analysis reported to 
Congress. 

In part due to the data's weaknesses, HUD also has not assessed the 
data related to self-sufficiency outcomes that it requires FSS and 
ROSS-SC grant recipients to report through the Excel-based reporting 
tool for each program as a whole. According to HUD officials, HUD 
field office staff review the outcomes data that individual PHAs 
report, but HUD had no process in place for assessing the outcomes 
reported through this tool programwide. Standards for internal control 
emphasize the need for federal agencies to collect reliable 
information with which to manage their programs and to review the 
integrity of performance measures.[Footnote 35] HUD officials with 
responsibility for these programs told us that headquarters staff did 
not use the reporting tools to assess the effectiveness of the FSS or 
ROSS SC programs because a system had not been developed to do such an 
assessment and because the data submitted were sometimes incomplete, 
not comparable, and unreliable. HUD officials also stated that PHAs 
likely vary in their interpretations of what to report because HUD 
never developed program-specific guidance. In 2013, in response to our 
review, HUD had its contractor (which collects data PHAs report using 
the tool) aggregate the outcomes data reported for the PH FSS and ROSS 
SC programs for fiscal years 2008 through 2011. However, HUD staff 
found that the results did not appear to be accurate or reliable. 

While HUD's effort to aggregate the performance data that they require 
grant recipients to report was a step in the right direction, the 
agency lacks a strategy for better ensuring that the outcomes data it 
collects are reliable and permit comparison across PHAs. Without a 
plan for helping to ensure that the outcome data FSS and ROSS SC grant 
recipients report are comparable and reliable, HUD will be unable to 
fully use the data it requires PHAs to report. 

MTW Program: 

HUD has not assessed the effectiveness of the MTW program using the 
information that it requires MTW agencies to submit in their annual 
performance reports on the impact of their MTW activities, including 
activities related to increasing resident self-sufficiency. MTW 
agencies generally have devised their own metrics for activities and 
reporting performance information, so the usefulness of this 
information for assessing programwide results is limited. That is, 
because the data are not consistent across agencies, they cannot be 
used to assess the performance of similar activities across MTW 
agencies. Additionally, in some cases the information is not outcome-
oriented and thus cannot be effectively used to assess performance. 

We previously recommended that HUD (1) improve its guidance to MTW 
agencies on providing information in their performance reports by 
requiring that such information be quantifiable and outcome-oriented 
to the extent possible; and (2) develop and implement a plan for 
quantitatively assessing the program as a whole, including the 
identification of standard performance data needed.[Footnote 36] HUD 
generally agreed with our recommendations, and in May 2013 OMB 
approved the revised guidance. Additionally, according to HUD, the 
agency continues to seek funding for a full evaluation that will 
better analyze the MTW information that is already collected and 
assess the effectiveness of the program, including self-sufficiency 
activities. 

HOPE VI: 

Unlike the previously discussed programs, for the HOPE VI program, HUD 
collects consistent, outcome-based measures for residents' 
participation in major types of community and supportive services, and 
uses the data to track residents' progress towards self-sufficiency. 
[Footnote 37] According to cumulative data on the community and 
supportive services as of the end of 2012, 72,278 working-age adults 
had received services related to improving their level of self-
sufficiency. According to these data, many residents enrolled in 
services related to education and job training, and from one-third to 
half of those who enrolled completed the activity (see table 6). 
However, these outputs may not be directly attributable to the 
community and supportive services program and are self-reported by 
HOPE VI agencies. HUD officials with responsibility for collecting 
these data noted that it would be excessively time and resource-
intensive to verify the accuracy of these data. HUD staff also noted 
that if a HOPE VI agency's community and supportive services data 
appeared to be inconsistent with past trends, HUD staff would follow 
up with the agency. 

Table 96: Data Reported by HOPE VI Grantees on the Number of Residents 
Who Enrolled in and Completed Community and Supportive Service 
Programs, as of December 2012: 

Activity: Job skills training; 
Enrollment: 34,098; 
Completion: 18,115; 
Percentage: 53%. 

Activity: High school or equivalent education; 
Enrollment: 17,944; 
Completion: 5,225; 
Percentage: 29%. 

Activity: Entrepreneurship training; 
Enrollment: 3,704; 
Completion: 1,626; 
Percentage: 44%. 

Activity: Homeownership counseling; 
Enrollment: 16,277; 
Completion: 7,004; 
Percentage: 43%. 

Source: HUD. 

Note: These figures include the last reported data for HOPE VI 
grantees with closed revitalization grants, and the most recent 
updates for grantees with ongoing revitalization grants. 

[End of table] 

Our Analysis of Available Data Is Not Conclusive: 

Where HUD had data, the data suggest positive changes in income and 
employment for families that participated in the two FSS programs, but 
these results are not conclusive. Data on program completion were 
missing for nearly half of the records we evaluated. More 
specifically, the FSS Addendum data in PIC lacked exit, completion, or 
extension data on 6,819 (46 percent) of 14,690 families that started 
either of the programs in 2006. Of these families, the "family report" 
section of PIC indicated that 1,671 (25 percent) had left subsidized 
housing. Thus, after comparing both the FSS Addendum and the main 
"family report" data, we found that HUD's systems lacked information 
on whether 5,148--35 percent--of the families that started either of 
the FSS programs in 2006 exited the FSS programs or subsidized housing. 

Of the subset of families for which exit, completion, or extension 
data were available in the FSS Addendum of PIC (that is, 54 percent of 
program participants), about 60 percent exited the program without 
completing it. In contrast, 25 percent of participants completed the 
program in 5 years or less (see table 7). 

Table 7: HUD Information on Experiences of a Subset of Families That 
Started FSS in Fiscal Year 2006 and for Which Data Were Available on 
Program Completion, Exit, or Extension, as of December 2012: 

Exited without completing: 
HCV FSS: 4,215; 
PH FSS: 497; 
Total FSS: 4,712; 
Percentage: 60%. 

Completed in 5 years or less: 
HCV FSS: 1,731; 
PH FSS: 206; 
Total FSS: 1,937; 
Percentage: 25. 

Completed in 6 years:
HCV FSS: 189; 
PH FSS: 31; 
Total FSS: 220; 
Percentage: 3. 

Granted an extension past 2011: 
HCV FSS: 915; 
PH FSS: 87; 
Total FSS: 1,002; 
Percentage: 13. 

Source: GAO analysis of HUD data. 

Notes: These data exclude MTW households and data on families for 
which HUD's FSS Addendum in PIC lacked completion, exit, or extension 
dates--46 percent of families that started either of the FSS programs 
in their fiscal year 2006. Percentages do not total to 100 percent due 
to rounding. 

[End of table] 

Considering only those families for which HUD had complete data, we 
observed positive changes in income and employment. We observed these 
changes for the families that started an FSS program in 2006 and 
completed it in 5 years or less.[Footnote 38] As noted above, this 
information is based on the 25 percent of families for which HUD data 
indicated completed the programs in 5 years or less. But, these 
findings do not take into account other factors that may have affected 
the families' progress towards self-sufficiency. Specifically, of the 
1,937 families that started either of the two FSS programs in fiscal 
year 2006 and completed them within 5 years, HUD's data suggest 
positive changes in income and employment (see table 8). For example, 
median income increased from about $17,000 per year to about $25,000 
per year. Families with total incomes of $35,000 or more when they 
started the program (ninetieth percentile) experienced income gains of 
34 percent, compared with income gains of 106 percent for families 
that had total incomes of about $3,000 or less (tenth percentile). 
These families also experienced positive changes in employment. For 
example, full-time employment among these program graduates increased 
76 percent. The proportion of these graduates who were working part-
time decreased 30 percent and the proportion not employed decreased 61 
percent. 

Table 8: Income and Employment Changes for Families that Began FSS in 
Fiscal Year 2006 and Had Completed It by Fiscal Year 2011: 

Median income[B]; 
Program start: 
HCV FSS: $17,153; 
PH FSS: $11,507; 
Total FSS: $16,684; 
Program completion[A]: 
HCV FSS: $25,804; 
PH FSS: $19,267; 
Total FSS: $25,323; 
Percent age change for total FSS: 52%. 

Mean income; 
Program start: 
HCV FSS: $18,551; 
PH FSS: $14,983; 
Total FSS: $18,172; 
Program completion[A]: 
HCV FSS: $27,286; 
PH FSS: $ 21,889; 
Total FSS: $26,712; 
Percent age change for total FSS: 47%. 

Income at tenth percentile; 
Program start: 
HCV FSS: $3,411; 
PH FSS: $0; 
Total FSS: $3,075; 
Program completion[A]: 
HCV FSS: $7,513; 
PH FSS: $2,402; 
Total FSS: $6,345; 
Percent age change for total FSS: 106%. 

Income at ninetieth percentile; 
Program start: 
HCV FSS: $$35,264; 
PH FSS: $32,072; 
Total FSS: $35,179; 
Program completion[A]: 
HCV FSS: $47,232; 
PH FSS: $45,502; 
Total FSS: $47,141; 
Percent age change for total FSS: 34%. 

Employed full-time[C]; 
Program start: 
HCV FSS: $664; 
PH FSS: $84; 
Total FSS: $748; 
Program completion[A]: 
HCV FSS: $1,181; 
PH FSS: $135; 
Total FSS: $1,316; 
Percent age change for total FSS: 76%. 

Employed part-time[C]; 
Program start: 
HCV FSS: $442; 
PH FSS: $46; 
Total FSS: $488; 
Program completion[A]: 
HCV FSS: $310; 
PH FSS: $34; 
Total FSS: $344; 
Percent age change for total FSS: -30%. 

Not employed[C]; 
Program start: 
HCV FSS: $625; 
PH FSS: $76; 
Total FSS: $701; 
Program completion[A]:
HCV FSS: $ 234; 
PH FSS: $37; 
Total FSS: $271; 
Percent age change for total FSS: -61%. 

Source: GAO analysis of HUD data. 

Notes: We used HUD's FSS Addendum data to determine the number of 
families. All dollar amounts are adjusted to fiscal year 2013 dollars 
using the chain-weighted Gross Domestic Product price index. This 
analysis excludes MTW agencies. Six of the HCV FSS records were 
missing information on employment status at program completion. 

[A] Successfully graduated from the program. 

[B] All income figures are based on the total household income. 

[C] Head of household. 

[End of table] 

About half of the 1,937 families that started one of the FSS programs 
in fiscal year 2006 and completed it received disbursements of escrow 
funds, according to PIC data. More specifically, 945 of the 1,937 
families that completed either of the two FSS programs in 5 years or 
less received a total of $5.9 million. The average amount disbursed 
ranged from about $2,400 in fiscal year 2007 to about $7,700 in fiscal 
year 2011.[Footnote 39] 

We also analyzed HUD data to determine the extent to which those who 
completed either of the two FSS programs stopped receiving rental 
housing assistance from HUD. As noted above, 1,937 families that began 
either of the two FSS programs in 2006 completed them within 5 years. 
Of these families, HUD data suggest that 473, or about 25 percent, had 
stopped receiving rental housing assistance from HUD within 1 year of 
program completion. 

Other Research on the Impact of Self-Sufficiency Programs Also 
Produced Inconclusive Results: 

HUD and a variety of researchers, including GAO, have reported on what 
is known about the impact on residents of the two FSS programs, HOPE 
VI, and MTW.[Footnote 40] These reports have shown some positive 
outcomes related to indicators of self-sufficiency but are mostly 
inconclusive because of methodological limitations. For example, 
evaluations of HUD's FSS programs indicate some positive impacts on 
participating residents but are not conclusive. 

* A 2004 study compared HCV FSS participants with nonparticipants at 
seven PHAs.[Footnote 41] The study reported that HCV FSS participants 
achieved higher incomes than nonparticipants and were more successful 
in reducing their reliance on welfare. But the study did not control 
for potentially significant differences between program participants 
and the comparison group, such as educational attainment and level of 
motivation. 

* A 2011 study of HCV FSS tracked 181 participants from 14 PHAs. 
[Footnote 42] This evaluation concluded that program graduates 
achieved substantial financial gains. It also found that residents who 
started the program with higher levels of education at enrollment 
achieved better results. This study is also inconclusive, because it 
did not employ a control group, participating residents were not 
representative of the overall HCV FSS population, and several PHAs 
declined to participate. 

* In addition, a 2005 academic study of a single PHA's implementation 
of both FSS programs found that residents who completed the programs 
attained significantly higher annual household incomes than those who 
did not and were able to find housing in the private market.[Footnote 
43] This study is not generalizable because the sample size of program 
participants was small and it examined one PHA. 

According to HUD, in March 2012 it began work on the first study of 
FSS to use random-assignment methods to evaluate the effectiveness of 
the programs. The study will be conducted over at least 5 years. 

HUD, GAO, and independent researchers have described the impact of the 
HOPE VI program on residents, but none of these studies has isolated 
the impact of community and supportive services on residents' self-
sufficiency. 

* HUD's 2003 study of 15 HOPE VI sites described changes in the 
characteristics of the residents who occupied the sites before and 
after revitalization and described the types of supportive services 
provided.[Footnote 44] The report noted that the characteristics of 
the resident population at HOPE VI sites might be different after 
revitalization for several reasons. For example, the revitalized 
developments may draw higher-income residents, and new on-site 
services may draw people interested in receiving services. However, 
this study did not include an assessment of the impact of community 
and supportive services on residents. 

* In 2004, we described limited HUD data on 165 HOPE VI grants awarded 
through fiscal year 2001, which indicated that the community and 
supportive services component of HOPE VI contributed towards positive 
outcomes.[Footnote 45] This description was based on HUD's HOPE VI 
Quarterly Reporting System. Through this system, HOPE VI grantees 
reported on the numbers of residents who participated in and graduated 
from various types of services. We noted the outcomes could not always 
be attributed to participation in HOPE VI. 

* The Urban Institute reported in 2007 that, based on its work in 
tracking residents who had been relocated from HOPE VI sites in 2001, 
there was no evidence that community and supportive services improved 
residents' self-sufficiency.[Footnote 46] According to the institute's 
analysis, while many residents reported living in better housing 
located in safer neighborhoods, unemployment rates had not improved. 

* A 2009 update to the Urban Institute's 2007 tracking effort 
similarly found that unemployment and self-sufficiency had not 
improved for residents who moved into private-market housing or for 
those remaining in public housing.[Footnote 47] 

Although HUD issued a report on the MTW program in 2010, little is 
known about the program's impact on the statutory goal of increasing 
resident self-sufficiency. A comprehensive evaluation of the MTW 
program is lacking, in part because HUD does not have a plan for 
identifying and analyzing standard performance data and has not 
established performance indicators for the MTW program as a whole. The 
2010 HUD report to Congress found that the effects of many MTW 
activities, especially as they related to residents, could not be 
conclusively identified because of the variety of and differences in 
the activities and metrics that MTW agencies were implementing. 
[Footnote 48] The Urban Institute, which evaluated the program in 
2004, noted that the lack of consistent data on resident 
characteristics, incomes, and rent payments prevented the authors from 
being able to determine whether individual MTW agencies were able to 
achieve the goal of increasing self-sufficiency.[Footnote 49] HUD's 
MTW Office has developed a statement of work for an evaluation of the 
program that is intended to assess self-sufficiency and other 
outcomes, but HUD officials confirmed that the proposed study was on 
hold pending funding. Officials also noted that newly admitted MTW 
agencies must implement and study rent reform activities, which are 
designed as a strategy to promote work and therefore self-sufficiency. 
[Footnote 50] These agencies have entered into partnerships with the 
Urban Institute and local universities to evaluate their rent reform 
activities. 

HUD Encourages PHAs to Coordinate with Local Service Providers: 

HUD's Strategic Plan and Guidance Encourage Coordination, and Some 
Programs Require It: 

HUD aims to improve self-sufficiency among residents of HUD-assisted 
rental housing by encouraging coordination between PHAs and agencies 
offering services that promote work and self-sufficiency. HUD's 2010-
2015 strategic plan presents strategies for increasing resident self-
sufficiency, which include coordination with federal, state, and local 
programs to increase access to job training, career services, and work 
support.[Footnote 51] 

Consistent with this strategy, HUD formed a partnership with the 
Department of Labor (Labor) to improve residents' access to Labor 
programs and services. Residents of HUD-assisted rental housing may be 
eligible for a variety of Labor programs and services, including those 
funded through the Workforce Investment Act (WIA). These services are 
delivered locally through American Job Centers, also known as one-stop 
centers, which provide education and career training, job search 
tools, and assistance with developing resumes and interview skills. 
Since 2009, HUD officials have held meetings with Labor's Employment 
and Training Administration to share information about their 
respective programs and seek opportunities to collaborate. One product 
of these meetings has been a joint toolkit for PHAs and local 
workforce agencies intended to improve HUD-assisted residents' access 
to employment services offered through the workforce agencies' one-
stop centers, intended for release in 2013. The draft toolkit offers 
PHAs and workforce agencies a baseline understanding of each other's 
functions, examples and lessons learned from successful partnerships 
currently in place, information on available online resources, and 
sample Memorandums of Understanding (MOU), or partnership agreements. 

HUD guidance also encourages PHAs to coordinate with local entities 
such as social service agencies and job training providers to increase 
residents' self-sufficiency. For example, HUD issued a notice to all 
PHAs in 2011 promoting partnerships with such agencies.[Footnote 52] 
The notice describes the benefits of collaboration, provides examples 
of possible partnerships and strategies for partnership development, 
and includes model MOUs. The notice states that HUD encourages such 
partnerships and recognizes that they benefit both PHAs and the 
households they serve. In addition, HUD's guidance to PHAs 
implementing the self-sufficiency programs we reviewed emphasizes the 
value of coordinating with local agencies. For example, HUD developed 
a resource website for HOPE VI grantees that provides information on 
interagency coordination and examples of PHA practices.[Footnote 53] 
Also, HUD's training materials for ROSS SC grantees include resources 
that PHAs can use to form partnerships, such as a sample partner 
outreach letter. While this guidance is directed to HOPE VI and ROSS 
SC grantees, it is publicly available on HUD's website and can be 
accessed by all PHAs. 

HUD's program requirements for four of the self-sufficiency programs 
we reviewed call for PHAs to coordinate with local service providers 
(see table 9).[Footnote 54] For example, grant funds for the two FSS 
programs and the ROSS SC program cannot be used to fund direct 
services. Instead, they must be used to hire coordinators who refer 
residents to local service providers, making coordination a key 
component of these programs' designs. Furthermore, applicants for ROSS 
SC and HOPE VI grants must demonstrate financial or in-kind support 
from partner organizations or agencies. Additionally, supportive 
services funded through HOPE VI grants must be coordinated with other 
service providers, including state and local programs. Finally, all 
four programs require participating PHAs to form coordinating 
committees to help ensure that residents are linked to the services 
that they need. For example, PHAs that receive either of the two FSS 
grants must establish a Program Coordinating Committee, which is 
charged with securing commitments of public or private resources for 
the operation of self-sufficiency programs. PHAs are encouraged to 
include local service providers, including welfare and workforce 
agencies, on the committees. 

Table 9: Examples of Ways That HUD Encourages or Requires PHAs to 
Coordinate with Local Agencies to Improve Resident Self-Sufficiency: 

For PHAs implementing: HUD notice, "Promoting Partnerships," offers 
examples, strategies, and model agreements for coordinating with local 
agencies[B]; 
FSS[A]: [Check]; 
ROSS SC: [Check]; 
HOPE VI: [Check]; 
MTW: [Check]. 

For PHAs implementing: Grant funds are used to pay a coordinator or 
case manager responsible for connecting residents with available 
community resources; 
FSS[A]: [Check]; 
ROSS SC: [Check]; 
HOPE VI: [Check]; 
MTW: [Empty]. 

For PHAs implementing: Grant funds may be used to contract with local 
agencies to provide services to residents[C]; 
FSS[A]: [Empty]; 
ROSS SC: [Empty]; 
HOPE VI: [Check]; 
MTW: [Check]. 

For PHAs implementing: Grant applications include expectations for 
PHAs to coordinate with local agencies; 
FSS[A]: [Check]; 
ROSS SC: [Check]; 
HOPE VI: [Check]; 
MTW: [Empty]. 

For PHAs implementing: Grant applications ask PHAs to demonstrate 
monetary or in-kind support from local agencies; 
FSS[A]: [Empty]; 
ROSS SC: [Check]; 
HOPE VI: [Check]; 
MTW: [Empty]. 

For PHAs implementing: Program requirements encourage PHAs to include 
local agencies on coordinating committees; 
FSS[A]: [Check]; 
ROSS SC: [Check]; 
HOPE VI: [Check]; 
MTW: [Empty]. 

Source: GAO analysis of HUD information. 

Notes: HUD regulations implementing a provision in the Quality Housing 
and Work Responsibility Act of 1998 require that all PHAs (not only 
those implementing one of HUD's self-sufficiency programs) make their 
best effort to enter into cooperation agreements with local welfare 
agencies to target services and share information. However, this 
requirement has not been monitored and HUD officials were not aware 
whether PHAs generally had such agreements in place. 

[A] Applies to both HCV FSS and PH FSS. 

[B] This notice applies to all PHAs; it is not limited to those PHAs 
that implement one of HUD's self-sufficiency programs. 

[C] MTW agencies do not receive special funding allocations. Rather, 
they receive funds from the three primary funding sources (public 
housing capital funds, public housing operating funds, and Housing 
Choice Vouchers). 

[End of table] 

Selected PHAs Coordinated with Local Service Providers: 

Selected PHAs worked with local service providers to implement HUD's 
self-sufficiency programs. We interviewed officials from a sample of 
five PHAs that had implemented one or more of HUD's self-sufficiency 
programs, as well as officials from workforce and welfare agencies in 
those locations.[Footnote 55] All of the PHAs we interviewed connected 
residents to local agencies that provided a broad range of services, 
including job training, mental health services, child care, 
transportation, food assistance, and homeownership counseling. Some of 
these local agencies received federal funding--for example, workforce 
agencies that administered WIA programs. PHA and local agency 
officials with whom we met stated that coordination efforts sometimes 
were formally established through MOUs, contracts, or regular meetings 
but added that most coordination efforts were informal. This informal 
coordination included referrals to each other's services or the 
sharing of information or updates on new programs and services. 

[Side bar] 

PHAs We Interviewed Coordinate with Local Agencies and Services 
Providers: 

These included:
* Workforce agencies; 
* Community & technical colleges; 
* Financial education providers; 
* Libraries; 
* Head Start programs; 
* Welfare agencies; 
* Job Corps programs; 
* Office for refugees; 
* Homeownership programs; 
* Hospitals; 
* Small business development programs; 
* Child care providers; 
* Adult basic education programs; 
* Food assistance programs; 
* Individual Development Account programs; 
* Energy assistance programs. 

[End of side bar] 

Officials from the five PHAs with whom we met identified few barriers 
to coordinating with local service providers and found current HUD 
guidance related to such coordination sufficient. Officials said that, 
despite resource constraints, they generally were able to obtain the 
services their residents needed from local agencies. PHA officials 
also stated that local agencies, including federally funded agencies, 
were receptive to their coordination efforts. However, while officials 
from two PHAs stated that they had strong relationships with their 
local workforce agencies, officials from the other three PHAs noted 
that these agencies' one-stop centers could be intimidating for 
residents and might not always be able to provide residents with 
appropriate services. Officials from workforce agencies we interviewed 
generally agreed, noting that residents might not have the education 
or work experience needed for some of the training opportunities that 
the centers offered. For example, a workforce agency official told us 
that individuals interested in job training in certain fields, such as 
trucking or nursing, must take tests to demonstrate required levels of 
academic readiness to participate. He said that many assisted housing 
residents were not prepared to pass these tests and might first need 
remedial adult education. HUD officials with responsibility for 
administering the self-sufficiency programs we reviewed and Labor 
officials with responsibilities related to workforce agencies told us 
that the joint HUD-Labor toolkit described above was developed in part 
to improve resident access to services offered through the one-stop 
centers. 

In general, awareness of federal efforts to improve coordination was 
mixed among the PHAs, workforce, and welfare agencies with which we 
met. For example, officials from two of the PHAs were familiar with 
the HUD notice on promoting partnerships, but officials from the other 
three PHAs were not. Additionally, officials from two of the PHAs and 
all five workforce agencies were not aware of a 2009 joint HUD-Labor 
letter on collaboration.[Footnote 56] Still, officials from these 
agencies generally did not express a need for further federal guidance 
or assistance in facilitating relationships between PHAs and local 
agencies. 

Conclusions: 

HUD's housing assistance programs serve millions of low-income 
residents. Statutory and programmatic requirements for HUD and PHAs 
also direct the organizations to undertake activities that would help 
these households increase their economic self-sufficiency. In 
particular, HUD has five programs that, in whole or in part, fund 
activities intended to help families become self-sufficient. However, 
HUD faces two major impediments to effectively operating these 
programs and achieving their goals. First, it does not have reliable 
data on participation in self-sufficiency activities across PHAs. 
Second, even though it has complete data for some PHAs, it generally 
does not use these data to review the progress participants may have 
made in, for example, finding a job or completing more education. 

In relation to participation data for the programs we reviewed, HUD 
has missed opportunities to help ensure that FSS participation data 
are complete. For example, in the two FSS programs significant gaps 
exist in participant entry and completion dates or reasons for 
leaving. These gaps are detrimental for several reasons, including 
their effects on grant funding and congressional reporting. 
Specifically, HUD uses resident participation data as a factor in 
making grant funding decisions and has reported this information in 
Congressional Budget Justifications. Moreover, without such data HUD 
cannot identify PHAs that have low or high completion rates. Federal 
internal control standards state that transactions should be promptly 
recorded to maintain their relevance and value to management and HUD's 
own reporting guidance also directs grant recipients to record program 
start dates, exit dates, reasons for exiting the program prior to 
completion, and completion dates. HUD has recognized the importance of 
having reliable participation data in recent years, emphasizing its 
importance to grantees in funding notices. But HUD could further work 
with agencies to correct participation data during the grant award 
process, or analyze the data outside of this process to help ensure 
that all required data are complete. By doing so, HUD will improve the 
accuracy of these data, improve its ability to assess FSS grantee 
activities and thus make better-informed decisions about funding them, 
and provide Congress a more complete view of program performance. 

Similarly, ROSS SC is designed to provide grant recipients with funds 
to hire staff to help residents progress towards economic independence 
and self-sufficiency. Since 2008, HUD has required grant recipients to 
report participation data. However, HUD has not provided grant 
recipients with program-specific reporting guidance; thus, the 
reported data vary and cannot be easily or reliably used for assessing 
programwide participation. Federal internal control standards state 
that program managers need operational data to determine whether they 
are meeting their agencies' strategic plans and meeting their goals 
for the effective and efficient use of resources. By developing 
program-specific reporting guidance, HUD could help ensure the 
collection of accurate participation data, and establish data sets 
that can be used to assess participation for the ROSS SC program as a 
whole. 

In relation to outcome data for the programs we reviewed, HUD has not 
optimized its use of the information it requires PHAs to collect. We 
acknowledge that determining the outcomes of self-sufficiency 
activities is difficult and requires rigorous analyses. It is also 
difficult to isolate the impact of such activities from other factors 
that may influence participant outcomes. But HUD could do more to put 
itself in a position to look across a program to review participant 
accomplishments. While HUD has collected some indicators for the FSS 
and ROSS SC programs (such as information on hours worked and receipt 
of welfare), it lacks a strategy for using the data it collects, 
whether through PIC or its Excel-based reporting tool. And, as with 
participation data, PHAs have not consistently reported such 
information, a condition exacerbated by the lack of program-specific 
reporting guidance. As stated above, internal control standards 
underline the importance not only of collecting but also using 
information to achieve programmatic goals--helping families increase 
self-sufficiency. Additionally, GPRAMA emphasizes the need for 
information on the effectiveness of federal programs to help improve 
congressional decision making. A strategy for using these data could 
inform overall management review, congressional oversight, and 
planning for these programs. For instance, using such data could help 
HUD identify from which PHAs to draw lessons to help improve HUD 
management of the grant programs as well as PHA management of self-
sufficiency-related activities. 

Recommendations for Executive Action: 

To better inform Congress and improve what is known about residents' 
participation in key grant programs designed to facilitate resident 
self-sufficiency, and their progress towards self-sufficiency, the 
Secretary of the Department of Housing and Urban Development should 
develop and implement: 

* a process to better ensure that data on FSS participant grants are 
complete; such a process should include steps for identifying missing 
data, identifying the reasons for missing data, and taking steps to 
help ensure data are complete; 

* a process to better ensure that PHAs awarded ROSS SC grants annually 
report required participation and outcome data that are comparable 
among grant recipients; this process should include the issuance of 
program-specific reporting guidance; 

* a strategy for regularly analyzing FSS participation and outcome 
data; such a strategy could include identification of PHAs from which 
lessons could be learned and PHAs that may need assistance improving 
completion rates or outcomes; and: 

* a strategy for regularly analyzing ROSS SC participation and outcome 
data; such a strategy could include identification of PHAs from which 
lessons could be learned and PHAs that may need assistance improving 
participation rates or outcomes. 

Agency Comments and Our Evaluation: 

We provided a draft of this report to HUD, Labor, Treasury, and HHS. 
HUD provided written comments, which are reprinted in appendix II. 
Labor and HHS provided technical comments, which we incorporated as 
appropriate, and Treasury did not provide comments. HUD agreed with 
three of our recommendations, and pointed to actions it intends to 
take to implement them. While agreeing with these recommendations, HUD 
noted some concerns about our recommendation that it develop and 
implement a strategy for regularly analyzing FSS participation and 
outcome data, and disagreed with our similar recommendation for the 
ROSS SC program. 

While agreeing with our recommendation that it develop and implement a 
strategy for regularly analyzing FSS participation and outcome data, 
HUD stated that the data captured in PIC for this program are not 
designed for rigorous statistical analysis. However, according to HUD 
guidance, PIC can be used to create reports, analyze programs, monitor 
PHAs, and provide information about residents in HUD-subsidized 
housing. In addition, if participating PHAs annually updated data, as 
required, in the FSS Addendum, PIC would include data on sources of 
assistance received by the family; whether the head of the household 
worked full-time, part-time, or was unemployed; and the highest grade 
of education for the head of the household. By implementing our first 
recommendation--that HUD develop and implement a process to better 
ensure that data on FSS participants are complete--the completeness 
and therefore reliability of outcome data should improve. 
Consequently, usefulness of analysis of such data would also be 
improved. 

HUD disagreed that it should develop a strategy for regularly 
analyzing ROSS SC participation and outcome data. In doing so, the 
agency noted the small size of the program, the difficulty of 
analyzing outcomes, and that the data it collects are administrative 
in nature and not intended to serve as the basis for analysis. 
However, HUD's training materials on the Excel-based tool that it uses 
to collect participation and outcomes data for the ROSS-SC program 
state that the tool is intended to be used to manage, monitor, and 
evaluate program services. These training materials and HUD staff with 
whom we met also indicated that the tool, when populated by 
participating PHAs, contains data on outputs and outcomes. And, as 
noted above, HUD agreed that its reporting guidance to ROSS SC 
grantees should be improved to help make data more meaningful. HUD's 
disagreement with our recommendation to analyze the information 
collected does not accord with its requirement for PHAs to submit 
operational data or with its assessment that data quality ought to be 
improved. Consistent with internal controls for the federal 
government, which apply to programs of all sizes, regular analysis of 
ROSS SC operational data would help HUD determine whether it was 
meeting goals for the effective and efficient use of resources. It can 
be difficult to isolate and definitively assess program outcomes. But 
program data can help identify patterns and anomalies in program 
operations, which can help agencies discover ways to achieve more 
value for the taxpayer's money. Consequently, we continue to recommend 
that HUD develop and implement a strategy for regularly analyzing ROSS 
SC participation and program data. 

In its technical comments, HUD also raised concerns about the accuracy 
of dollar amounts reported and the characterization of outputs related 
to the community and supportive services component of HOPE VI. First, 
HUD questioned the accuracy of the amounts awarded to PHAs through the 
FSS and ROSS SC programs and the amounts HOPE VI agencies spent on 
community and supportive services. Because trends in nominal spending 
may reflect changes in both price and quantity, we chose to present 
inflation-adjusted values by removing the general effects of inflation 
using a price index. Specifically, we adjusted these figures to fiscal 
year 2013 dollars using the fiscal year chain-weighted Gross Domestic 
Product price index. Second, for the HOPE VI program, HUD stated that 
without a costly and time-intensive experimental research design, it 
would not be possible to know whether the outputs residents' 
experienced were directly attributable to the program. We acknowledged 
the difficulties of isolating program outcomes in this report, and 
have not made any recommendations related to HOPE VI. Finally, during 
the course of our review, OMB approved revised reporting requirements 
for the MTW program, which are intended to establish standard metrics 
for activities related to self-sufficiency, among other things. We 
revised the report to recognize these changes. HUD provided additional 
technical comments, which we incorporated as appropriate. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 30 days 
from the report date. At that time, we will send copies to the 
Secretary of Housing and Urban Development and other interested 
committees. In addition, the report will be available at no charge on 
the GAO website at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions concerning this report, please 
contact me at (202) 512-8678 or sciremj@gao.gov. Contact points for 
our Offices of Congressional Relations and Public Affairs may be found 
on the last page of this report. Key contributors to this report are 
listed in appendix III. 

Signed by: 

Mathew J. Scirè: 
Director, Financial Markets and Community Investment: 

[End of section] 

Appendix I: Scope and Methodology: 

Our objectives were to (1) discuss what is known about the costs of 
and residents' participation in Department of Housing and Urban 
Development (HUD) grant programs that encourage work and self-
sufficiency, (2) determine what is known about the effect on residents 
of HUD grant programs to promote self-sufficiency, and (3) describe 
steps HUD has taken to coordinate with other federal agencies and 
increase residents' access to non-HUD programs that encourage work and 
self-sufficiency. 

To discuss what is known about the costs of and residents' 
participation in HUD programs that encourage self-sufficiency we 
reviewed documentation of HUD's programs and determined whether and 
how they defined and addressed resident self-sufficiency. Because HUD 
did not have an agencywide definition of self-sufficiency, and 
included 20 programs in its list of programs that contribute to its 
subgoal of increasing economic security and self-sufficiency, we 
established criteria to narrow the scope of programs to include in our 
review. First, we limited our review to grant programs that were 
intended to encourage work and self-sufficiency among residents of HUD-
assisted rental housing. Second, we included grant programs that did 
not have self-sufficiency as their primary focus but rather as a 
secondary program goal. We sought to include programs that were 
actively awarding grants, or for which grants were still open. Based 
on these criteria, we identified the following programs: 

* Public Housing Family Self-Sufficiency (PH FSS); 

* Housing Choice Voucher Family Self-Sufficiency (HCV FSS); 

* Resident Opportunity and Self-Sufficiency Service Coordinators (ROSS 
SC); 

* Moving to Work (MTW); 

* HOPE VI. 

HUD officials agreed that we had identified the key grant programs 
that encourage work and self-sufficiency. To describe what is known 
about the programs' costs, we met with program staff and obtained 
documentation of the total grant amounts awarded for specific periods, 
as follows: 

* For the Family Self-Sufficiency (FSS) programs, we obtained grant 
award amounts for fiscal years 2006 through 2011 and analyzed data 
from HUD's Public and Indian Housing Information Center (PIC) system 
on escrow account disbursements from fiscal years 2006 through 2011. 
We adjusted all dollar amounts to fiscal year 2013 dollars using the 
chain-weighted Gross Domestic Product price index. 

* For ROSS SC, we obtained grant award amounts for fiscal years 2008 
through 2011, because 2008 was the first year of the current version 
of the program. 

* For the MTW program, we met with program staff and staff from HUD's 
Grants Management Office to determine whether HUD collected 
information on MTW agencies' expenditures for activities related to 
self-sufficiency. 

* For the HOPE VI program, we reviewed and summarized the fiscal year 
2006-2010 Notice of Funding Availability (NOFA) requirements related 
to community and supportive services expenditures and the amount of 
HOPE VI revitalization grant funds that HOPE VI agencies expended on 
community and supportive services in that period. 

To analyze residents' participation in each of the five programs, we 
met with HUD staff to determine how participation data were collected 
and summarized any readily available information. 

* For the two FSS programs, we analyzed participation data from PIC 
for fiscal years 2006 through 2011. Specifically, we analyzed data 
that public housing agencies (PHA) entered in the FSS Addendum, a PIC 
subcomponent. As a part of this analysis, we identified the extent to 
which program start dates, end dates, and annual updates for 
participating households were entered. We interviewed HUD officials 
with responsibility for administering both FSS programs and the PIC 
system about the completeness of the data and HUD's use of the 
information. We evaluated the information available on families' 
participation in the two FSS programs in relation to internal control 
standards for the federal government as well as HUD's own guidance. 
[Footnote 57] Based on interviews with HUD officials and our review of 
the completeness of data fields in the FSS Addendum component of PIC, 
we determined that the available data were not sufficiently reliable 
to provide an accurate count of residents' participation in FSS for 
fiscal years 2006 through 2011. We summarized data that HUD posted on 
its website as a part of the NOFA process for fiscal years 2009 
through 2012. 

* For the ROSS SC program, we obtained and reviewed aggregated data on 
program participation that a HUD contractor created in 2013. We 
evaluated the information available on residents' participation in 
ROSS SC in relation to internal control standards for the federal 
government.[Footnote 58] We determined that the aggregated 
participation data were not sufficiently reliable to provide an 
accurate count of residents' participation in ROSS SC for fiscal years 
2008 through 2011. 

* For the HOPE VI program, we summarized data from the HOPE VI 
Quarterly Reporting System on the total number of nondisabled 
residents between the ages of 18 and 64 who had received services 
through the community and supportive services component of HOPE VI 
from the program's inception through the end of 2012. 

* For the MTW program, we interviewed program administrators to 
determine whether and how they used information from participating 
agencies' Annual MTW Reports or MTW-PIC to determine the number of MTW 
households participating in MTW and activities related to self-
sufficiency. We analyzed the FSS Addendums associated with MTW-PIC in 
an effort to determine the number of MTW families that had 
participated in either of the two FSS programs. We determined, based 
on interviews with HUD officials and our analysis of FSS Addendum data 
for MTW agencies, that these data were not sufficiently reliable prior 
to 2011. We summarized available data on MTW families' participation 
in the two FSS programs in 2011. 

To determine what is known about the effect on residents of HUD's 
grant programs that encourage work and self-sufficiency, we examined 
whether and how HUD collected information that could indicate progress 
toward self-sufficiency, such as information on income and employment, 
for each program. We reviewed available data dictionaries and guidance 
HUD provided to PHAs required to report this information. We 
determined that for the five programs we reviewed, HUD used PIC, MTW-
PIC, Excel-based reporting tools, written reports, and other reporting 
systems to collect this information. We interviewed HUD staff about 
their use of the self-sufficiency indicators they require PHAs to 
report, the reliability of the data collected, and program-specific 
guidance. We evaluated HUD's processes for aggregating self-
sufficiency-related outcomes data in relation to our standards for 
internal control. For the two FSS programs, we analyzed PIC data on 
the 14,690 families that HUD's data indicated started the programs in 
2006. Based on PIC data for fiscal years 2006 through 2012, we 
identified the number of families that completed the two FSS programs, 
exited the programs without completing them, and received an extension 
to continue the program past 2011, and for which HUD's system lacked 
either program exit, completion, or extension information. For the 
subset of families that HUD's data indicated had completed the two FSS 
programs in 5 years or less (1,937 households), we analyzed changes in 
median and mean income, income for those at the tenth and ninetieth 
percentiles, and employment experiences. We adjusted all dollar 
amounts to fiscal year 2013 dollars using the chain-weighted Gross 
Domestic Product price index. This sample represented about 25 percent 
of the families that started in 2006 and for which exit, completion, 
or extension data were available. In addition, this analysis does not 
control for other factors that may have affected participants' 
progress towards self-sufficiency. We excluded any households that 
were receiving rental assistance from an MTW agency because HUD 
officials indicated that their FSS participation data were not as 
reliable, particularly before 2011. 

Through interviews and a literature search, we identified several 
studies of the two FSS programs, HOPE VI, and MTW. We reviewed these 
studies to identify information on the programs' impact on resident 
self-sufficiency. We determined that these reports were 
methodologically sound and reliable for our purposes. We did not 
identify any studies of the ROSS SC program. 

To describe the steps HUD has taken to coordinate with other federal 
agencies and increase residents' access to non-HUD programs, we 
reviewed HUD regulations, policies, and guidance related to 
coordination. We also interviewed officials from HUD, the Department 
of Health and Human Services, and the Department of Labor, and 
reviewed materials related to interagency coordination provided by 
these officials. To better understand how PHAs connect residents with 
non-HUD programs, we interviewed staff from PHAs administering each of 
the self-sufficiency programs included in our review.[Footnote 59] To 
select PHAs, we compiled lists of PHAs that received grant funding (or 
permission to participate in MTW) between 2004 and 2011. We then 
randomly selected a PHA to interview for each program and adjusted our 
initial sample to ensure that the five selected PHAs varied in terms 
of size and region.[Footnote 60] Four of the selected PHAs implement 
more than one of the programs in our review. The selected PHAs were: 

* Boulder County Housing Authority (Boulder, Colorado); 

* Jersey City Housing Authority (Jersey City, New Jersey); 

* Kingsport Housing and Redevelopment Authority (Kingsport, Tennessee); 

* Louisville Metro Housing Authority (Louisville, Kentucky); 

* Washington County Housing and Redevelopment Authority (Woodbury, 
Minnesota). 

For each of these locations, we also interviewed officials from the 
local agencies that administered Temporary Assistance for Needy 
Families and the Workforce Investment Act programs. 

We conducted this performance audit from August 2012 through July 2013 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our 
findings and conclusions based on the audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Housing and Urban 
Development: 

U.S. Department of Housing and Urban Development: 
Assistant Secretary for Public and Indian Housing: 
Washington, DC 20410-5000: 

June 21, 2013: 

Matthew J. Scirè: 
Director: 
Financial Markets and Community Investment: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Mr. Scirè: 

Thank you for the opportunity to comment on the Government 
Accountability Office's (GAO) Draft Report entitled, Assisted Housing: 
HUD Has Not Fully Evaluated its Self-Sufficiency Programs (GAO-13-581). 

The Department of Housing and Urban Development has reviewed the 
report. Comments to the report are enclosed in the attached chart, 
narrative response, and in notes embedded in the PDF version of the 
report. These documents will also be sent via email correspondence. 

If you have any questions, please contact Dominique 810m, Deputy 
Assistant Secretary, Office of Public Housing Investments at (202) 
402-8500. 

Sincerely, 

Signed by: 

Deborah Hernandez: 
General Deputy Assistant for Public and Indian Housing: 

HUD Comments To GAO-13-581 Draft Report: 

The Government Accountability Office (GAO) conducted a review of five 
programs in the office of Public and Indian Housing (PIH) that provide 
either grant funding or flexibility to Public Housing Agencies (PHAs) 
to support clients' self-sufficiency. GAO's review points to
challenges that pertain to HUD's methods for data collection and 
analysis relating to four of the five programs.[Footnote 1] HUD is 
aware of these challenges and has taken concrete steps to mitigate them
within the constraints of very limited staffing and Information 
Technology (IT) resources. These four programs, relative to other 
Public and Indian Housing programs, are quite small. In addition to 
program size, HUD allocates its limited staffing and IT resources 
based on internal risk analysis and other considerations. Nonetheless, 
as outlined below, HUD has taken and will continue to take steps to 
improve its management and oversight of these programs. For instance,
HUD published Notice PIH 2011-65 ("Timely Reporting Requirement of the 
Family Report") (form HUD-50058 and HUD 50058-MTW) into the Public and 
Indian Housing Information Center") emphasizing to PHAs the importance 
of the quality of the data they put into PIC. In the notice: 

* HUD informed PHAs that "The Department relies on PHAs to submit 
accurate, complete and timely data to administer, monitor and report 
on the management of its rental assistance programs. In order to fully 
justify its budget requests to Congress, the Department needs full 
cooperation from all PHAs in meeting their reporting requirements in a 
timely manner." 

* HUD focused on FSS specifically, reminding PHAs what information 
must be reported, how often it must be reported, and that "FSS 
Enrollment, Progress and Exit reports must also be submitted within 60 
days of the effective date of the FSS action." 

The HOPE VI Program: 

With respect to HOPE VI, while GAO did not recommend specific 
improvements in data collection or analysis, the report did state that 
outputs of the Community and Supportive Services (CSS) component could 
not be attributable to the CSS program itself and that the data is 
self-reported. In order to ascertain with certitude whether the CSS 
program has achieved specific outputs or outcomes, costly and time 
intensive experimental design research would have to be conducted. HUD 
does not have the resources to invest in this type of research. HUD 
believes it is reasonable to assume that most of the positive outcomes 
seen in the CSS program would not have been achieved without its 
significant investment. 

The Family Self-Sufficiency Programs: 

The primary data collection tool HUD uses for the Public Housing and 
Housing Choice Voucher (HCY) Family Self-Sufficiency (FSS) programs is 
the Public & Indian Housing (PIR) Information Center (PIC). The Office 
of Public Housing Investments (OPHI) and the Office of Public Housing 
and Voucher Programs (OPHVP), the two offices with responsibility for
overseeing the FSS programs, recognize that the current configuration 
of the PIC system affects the quality of the PIC data. For example, 
PHAs are able to enter incomplete FSS participant data without the 
system generating an error message. As a result, problems such as 
under-reporting of FSS participant data occur. A lack of staff 
resources has limited HUD's ability to analyze or quantify the degree 
to which this is happening. Also, IT resource constraints have limited 
HUD's ability to make the necessary upgrades within PIC. Now that HUD 
is in the process of developing the Next Generation Management System 
(NGMS), further significant modifications to PIC will not be made. It 
is envisioned that NGMS will include a module that will address the 
data collection and qUality issues that have been identified. 

Until such time that improvements are implemented, HUD will continue 
to rely on the information in PIC, despite its limitations. The 
information currently in the PIC system provides benchmark information 
to gauge the direction of the program. For this reason, HUD has used
PIC-generated information in its Congressional Justifications. PIC 
data are still the best available data. 

The Resident Opportunity and Self-Sufficiency Service Coordinator 
Program: 

The primary data collection tool for the ROSS-SC program is the Logic 
Model (LM) which uses an Excel-based platform. The LM was a tool 
selected by the Division of Grants Management and Oversight, the 
office that oversees all of HUD's grant programs, and was approved by 
the Office of Management and Budget (OMB). While the tool itself could 
be useful, the form can be confusing for grantees, and the method used 
to collect and extract the data is cumbersome for both grantees and 
HUD staff. Over the years, HUD has provided training in the form of 
webinars and written instructions to applicants and grantees about how 
to use the LM; however. HUD agrees with GAO's finding that more 
specific guidance is warranted. OPHI, the office overseeing the ROSS 
program, is aware that the data collected by the LM was inconsistent 
across grantees' submissions, and therefore any attempt to analyze 
aggregated data would be of limited use. HUD is aiming to transition 
to a simpler web-based reporting tool. but limited funding and 
competing needs for IT dollars preclude the Department from developing
such a system in the near future. In an effort to mitigate the 
reporting challenges presented by the LM. HUD headquarters holds 
monthly training conference calls with HUD field office staff to 
underscore the importance of monitoring the information contained in 
the LM reports. HUD has and will continue to provide ongoing 
programmatic oversight for the LM. 

The Moving to Work Demonstration Program: 

PHAs participating in the MTW demonstration submit data into standard 
HUD systems, including PIC. In addition, MTW PHAs provide an Annual 
MTW Plan and an Annual MTW Report to HUD. These document submissions 
must contain the information requested in the HUD Form 50900 
("Elements for the Annual MTW Plan and MTW Report") and provide
program and activity specific data for each individual MTW PHA, 
including metrics for measuring the effects of these activities. 
Historically, HUD allowed each MTW PHA to establish its own metrics 
and performance indicators for activities, including those relating to 
self-sufficiency, however the latest revision to HUD form 50900 
establishes standard metrics for self-sufficiency. 

On May 31,2013, HUD received OMB approval for a revised version of the 
HUD-50900. The revision defines key program terms and standardizes 
metrics in an outcome-oriented and quantifiable way. This 
standardization of the data collection process will allow for analysis
across the MTW demonstration as a whole and within activity objectives 
such as self-sufficiency. The standard metrics will also provide HUD 
with the ability to compare MTW PHA self-sufficiency programs with one 
another and with other programs available to non-MTW PHAs. Such 
analysis could not only provide information on PHA performance, but 
also broader insight into the effectiveness of particular self-
sufficiency interventions. 

HUD Responses to GAO's Recommendations: 

1. GAO Recommendation: [HUD should develop and implement] a process to 
better ensure that data on FSS participants grants are complete; such 
a process should include steps for identifying' the reasons for 
missing data, and taking steps to help ensure data are complete. 

HUD Response: HUD has taken many steps to improve the quality of PHAs' 
data on FSS participants (as discussed above). HUD agrees that more 
can be done to ensure the completeness of this data. To this end, HUD 
intends to develop a process to identify missing data, identify the 
reasons for the missing data, and to take steps to help ensure data 
are complete. Specifically, HUD will work with PHAs identified through 
the funding award process as having incomplete data in PIC to ensure 
that they submit all FSS participant records into PIC. Additionally, 
HUD will continue to seek changes to the PIC system and/or NOMS itself 
so that error messages are generated when a PHA fails to completely 
enter data on FSS participants into PIC. However, HUD's ability to
implement these processes is subject to the availability of resources, 
including staffing and IT resources. 

2. GAO Recommendation: [HUD should develop and implement] a process to 
better ensure that PHAs awarded ROSS SC grants annually report 
required participation and outcome data that are comparable among 
grant recipients; this process should include the issuance of program-
specific reporting guidance. 

BUD Response: In order to ensure that grantees submit required reports 
(financial reports using form SF-425 and outcomes using the Logic 
Model (LM)) on an annual basis, HUD has a system for preventing 
payments when grantees do not submit these reports. However, due to 
lack of specific program guidance, information contained in the LM 
reports submitted by grantees, though technically complete, may not be 
of meaningful use when aggregated. HUD agrees with GAO's 
recommendation that program-specific reporting guidance is needed. HUD 
will develop guidance that will include definitions of indicators so 
that all grantees will have a common understanding of the data HUD is 
seeking to capture. In addition, HUD will work to simplify the LM. HUD 
will continue to work towards developing a web-based reporting tool 
for grantees. Until such time, a more streamlined LM with reporting 
guidance, combined with the monthly training conference calls between 
HUD headquarters and HUD field offices, will allow HUD to better 
monitor the program. 

3. GAO Recommendation: [HUD should develop and implement] a strategy 
for regularly analyzing FSS participation and outcome data; such a 
strategy could include identification of PHAs from which lessons could 
be learned and PHAs that may need assistance improving completion 
rates or outcomes. 

BUD Response: The data captured in PIC for the two FSS programs is 
administrative and descriptive in nature. It is not designed to 
support rigorous statistical analysis. Despite these limitations, HUD 
agrees that a better strategy for regularly analyzing FSS 
participation and outcome data would be beneficial to program 
management and oversight. Within the confines of the existing data, it 
would be possible for HUD to identify poor performing as well as high 
performing PHAs, however, the criteria for doing so must be selected 
very carefully so as to take into account the widely varying needs of 
enrolling FSS families. For instance, GAO recommends reviewing data on 
the length of stay in the program, presuming that shorter is better. 
However, length of time in the FSS program is not necessarily a good 
indication of a strong program because families enter the program at 
various levels of readiness to enter the workforce. If we judged 
programs on length of stay, PHAs would be encouraged to "cream" to 
serve only the families most ready for work. Each indicator must be 
reviewed for such bias. Nevertheless, OPHI and OPHPV will work with 
PD&R to implement regular analysis to identify PHAs from which lessons 
could be learned and identify PHAs that may require assistance. 

4. GAO Recommendation: [HUD should develop and implement] a strategy 
for regularly analyzing ROSS SC participation and outcome data; such a 
strategy could include identification of PHAs from which lessons could 
be learned and PHAs that may need assistance improving participation 
rates or outcomes. 

HUD Response: HUD would like to be able to analyze participation and 
outcome data for the ROSS-SC program. However, due to the small size 
of the program, the fact that the data is administrative in nature and 
not intended to be used for rigorous analysis, and that outcomes based 
on services are very difficult to study in and of themselves, HUD is 
not in a position to agree with this "recommendation. However, by 
streamlining the LM we can, as noted in our response to GAO's second 
recommendation, improve the quality of our data so that we can better 
monitor the program, derive lessons learned, and provide targeted 
assistance to PHAs whose reporting indicates that such assistance is 
necessary. 

Footnote: 

[1] Public Housing Family Self-Sufficiency, Housing Office Voucher 
Family Self-Sufficiency, the Resident Opportunity and Supportive 
Services Service Coordinator program, and the Moving to Work 
Demonstration Program. 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Mathew J. Scirè, (202) 512-8678 or sciremj@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Paul Schmidt (Assistant 
Director), Stephen Brown, Emily Chalmers, Geoff King, John McGrail, 
Marc Molino, Lisa Moore, Michael Pahr, Barbara Roesmann, Beverly Ross, 
and Andrew Stavisky made key contributions to this report. 

[End of section] 

Footnotes: 

[1] In fiscal year 2012, the budget authority for these programs was 
$18.3 billion for housing choice vouchers, $9.3 billion for project-
based rental assistance, and $5.8 billion for public housing. The 
amount for public housing includes $1.9 billion for the Public Housing 
Capital Fund and nearly $4 billion for the Public Housing Operating 
Fund, which are meant to supplement rents and help meet the operation, 
maintenance, and capital needs of public housing. 

[2] HUD defines an elderly family as a family whose head (including co-
head), spouse, or sole member is a person at least 62 years of age. A 
disabled family is one whose head (including co-head), spouse, or sole 
member is a person living with disabilities; or two or more persons 
with disabilities living together; or one or more persons with 
disabilities living with one or more live-in aides. 

[3] Rather than operate two separate and independently administered 
FSS programs for voucher and public housing families, the 
Administration proposes in its 2014 budget proposal to consolidate the 
two FSS programs to enable PHAs to more uniformly serve program 
participants. 

[4] According to recent FSS notices of funding availability (NOFA), 
case management should help ensure that the services in the 
participants' contracts of participation are provided on a regular, 
ongoing, and satisfactory basis; that participants are fulfilling 
their responsibilities under the contracts; and that FSS escrow 
accounts are established and properly maintained for eligible families. 

[5] 42 U.S.C. § 1437u. 

[6] For the FSS programs, welfare assistance means income assistance 
from federal or state welfare programs, and includes only cash 
maintenance payments designed to meet a family's ongoing basic needs. 
It does not include nonrecurrent, short-term benefits, or food 
assistance and emergency rental and utilities assistance, among 
others. See 24 CFR § 984.103. 

[7] PHAs can designate whether the service coordinator will serve 
families, the elderly, or disabled residents, or a mix of the three. 
In the case of elderly or disabled residents, services should help 
improve living conditions and enable residents to age in place. 

[8] Completion of the contract occurs when the PHA determines that: 
(1) the family has fulfilled all of its responsibilities under the 
contract; or (2) 30 percent of the family's monthly adjusted income 
equals or is greater than the Fair Market Rent amount for the unit 
size for which the family qualifies. 24 C.F.R. § 984.303(g). Families 
are expected to complete their contracts within 5 years, although a 
PHA may extend a family's contract for up to 2 additional years in 
some circumstances. 24 C.F.R. § 984.303(c), (d). 

[9] For the purposes of this report, we focused on programs that were 
active or still had open grants at the time of our review. HOPE VI 
revitalization grants were not awarded after fiscal year 2010; this 
program was replaced by the Choice Neighborhood Program. HUD 
previously funded other demonstration programs that encouraged 
resident self-sufficiency, such as the Jobs Plus Demonstration 
Program, which was implemented by six PHAs from 1998 through 2003. For 
more information about this program, see Howard S. Bloom, James A. 
Riccio, and Nandita Verma, Promoting Work in Public Housing: The 
Effectiveness of Jobs-Plus, Final Report (New York, N.Y.: March 2005). 

[10] Authorized by the Omnibus Consolidated Rescissions and 
Appropriations Act of 1996 (Pub. L. No.104-134, 110 Stat. 1321, 1321-
281), MTW is a demonstration program. MTW agencies do not receive 
special funding allocations. Rather, they receive funds from the three 
primary funding sources (public housing capital funds, public housing 
operating funds, and Housing Choice Vouchers). 

[11] Authorized by Pub. L. No. 105-276, § 535 (42 U.S.C. § 1437v). 

[12] HOPE VI NOFAs used from 2006 through 2010 noted that PHAs would 
be evaluated based on their Community and Supportive Services Plans. 
Recipients can also establish endowment trusts so that they can 
continue to fund such activities after the grant funds have been 
expended. Examples of activities include adult education, employment 
training, training on parenting and budgeting, homeownership 
counseling, and substance and alcohol abuse treatment and counseling. 
A NOFA announces the availability of grant funding. HUD's NOFAs 
generally include a description of the funding opportunity, 
eligibility requirements, and relationship to HUD's strategic goals 
and policy priorities. 

[13] 24 CFR 984.103. 

[14] See 24 C.F.R. § 984.305. 

[15] HUD, HUD Strategic Plan Fiscal Year 2010-2015 (Washington, D.C.: 
May 2010). 

[16] GAO, Moving to Work Demonstration: Opportunities Exist to Improve 
Information and Monitoring, [hyperlink, 
http://www.gao.gov/products/GAO-12-490] (Washington, D.C.: Apr. 19, 
2012). 

[17] Although first implemented in 1999, the current version of the 
ROSS SC program was implemented in 2008. 

[18] PHAs have the authority to disburse some funds to a family before 
contract completion. If a PHA determined that a family had fulfilled 
certain goals in its FSS contract and needed a portion of the escrowed 
funds for purposes consistent with the contract goals (such as 
completion of higher education), the PHA could disburse a portion of 
the funds to the family. 

[19] GAO, Standards for Internal Control in the Federal Government, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] 
(Washington, D.C.: Nov. 1999). 

[20] We excluded MTW households from our analysis of FSS participation 
because available data were not reliable prior to 2011. 

[21] PH FSS funding notices for fiscal years 2007 through 2011 stated 
that PHAs seeking funding could submit participation data using a 
separate HUD form, and notices for fiscal years 2009 through 2011 
stated that this form would take precedence over PIC data. HCV FSS 
funding notices state that participation data should come from PIC or 
other documents, including a separate HUD form or a MTW report. 

[22] For nearly 600 of the 11,596 families, the data included more 
than one program start date. 

[23] PHAs can extend a families' participation in either FSS program 
for up to 2 additional years. 

[24] Funding announcements did not specify similar procedures for PHAs 
that believed they would be overfunded as the result of an incorrect 
participation count posted by HUD. 

[25] According to HUD officials, PHAs are continually encouraged to 
make corrections to their PIC data. 

[26] GAO, Information Technology: HUD Needs to Strengthen Its Capacity 
to Manage and Modernize Its Environment, [hyperlink, 
http://www.gao.gov/products/GAO-09-675] (Washington, D.C.: July 31, 
2009). 

[27] HUD refers to the reporting tools used to collect this 
information as a Logic Model. Recipients of other HUD grants, 
including the grants under the two FSS programs, also must report data 
to HUD using this tool. Training materials for the Logic Model state 
that it can be used to manage, monitor, and evaluate program services. 
It contains data on needs, outputs, and outcomes. 

[28] GAO, Entrepreneurial Assistance: Opportunities Exist to Improve 
Programs' Collaboration, Data-Tracking, and Performance Management, 
[hyperlink, http://www.gao.gov/products/GAO-13-452T] (Washington, 
D.C.: Mar. 20, 2013). 

[29] For the purposes of tracking residents' participation in 
community and supportive services, HUD defined "working age" as 
residents between the ages of 18 and 64. While HOPE VI agencies also 
provided community and supportive services to the elderly and 
disabled, HUD did not require the agencies to track these residents' 
participation. 

[30] Data are from HUD's HOPE VI Quarterly Reporting System as of the 
end of 2012, but HUD does not verify the accuracy of these self-
reported data. About 95,000 (nonelderly and nondisabled) residents 
lived at the HOPE VI sites before revitalization. 

[31] [hyperlink, http://www.gao.gov/products/GAO-12-490]. 

[32] MTW agencies may seek permission to update this information less 
frequently than regular PHAs--for example, they may verify an elderly 
resident's income every 2 years instead of annually. 

[33] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. 

[34] Pub. L. No. 111-352, 124 Stat. 3866 (2011). GPRAMA amends the 
Government Performance and Results Act of 1993 (GPRA), Pub. L. No. 103-
62, 107 Stat. 285 (1993). See also Office of Management and Budget, 
Preparation and Submission of Strategic Plans, Annual Performance 
Plans, and Annual Program Performance, Circular A-11, Part 6, sec. 
200.4 (Washington, D.C.: August 2012). 

[35] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. 

[36] [hyperlink, http://www.gao.gov/products/GAO-12-490]. 

[37] We recommended in July 1998 that HUD develop consistent national, 
outcome-based measures for community and supportive services at HOPE 
VI sites. HUD has used its HOPE VI reporting system to collect data 
from grantees on the major types of community and supportive services 
they provide and outcomes achieved by some of these services. See GAO, 
HOPE VI: Progress and Problems in Revitalizing Distressed Public 
Housing, [hyperlink, http://www.gao.gov/products/GAO/RCED-98-187] 
(Washington, D.C.: July 20, 1998); and Public Housing: HOPE VI 
Resident Issues and Changes in Neighborhoods Surrounding Grant Sites, 
[hyperlink, http://www.gao.gov/products/GAO-04-109] (Washington, D.C.: 
Nov. 21, 2003). 

[38] These outcomes cannot be directly attributed to participation in 
the FSS program. Other factors for which we did not control may have 
influenced them. 

[39] Dollar amounts are adjusted to fiscal year 2013 dollars using the 
chain-weighted Gross Domestic Product price index. For the HCV FSS 
program, forfeited funds are placed in the Housing Assistance Payment 
account. For the PH FSS program, forfeited funds are credited to the 
PHA's operating fund and counted as other income in the calculation of 
the operating subsidy eligibility for the next budget year. 

[40] We did not identify any studies of ROSS SC by HUD or others. 

[41] Robert C. Ficke, Andrea Piesse, et al., Evaluation of the Family 
Self-Sufficiency Program: Retrospective Analysis, 1996 to 2000 a 
report prepared for the Department of Housing and Urban Development, 
Office of Policy Development and Research, April 2004. 

[42] Lalith de Silva, Imesh Wijewardena, Michelle Wood, and Bulbul 
Kaul, Evaluation of the Family Self-Sufficiency Program: Prospective 
Study, a report prepared for the Department of Housing and Urban 
Development, Office of Policy Development and Research, February 2011. 

[43] Jerry Anthony, "Family Self-Sufficiency Programs: An Evaluation 
of Program Benefits and Factors Affecting Participants' Success," 
Urban Affairs Review, 41, no. 1 (September 2005). 

[44] Mary Joel Holin, Larry Buron, Gretchen Locke, and Alvaro Cortes, 
Interim Assessment of the Hope VI Program, Cross-Site Report, prepared 
by Abt Associates for the Department of Housing and Urban Development, 
Office of Policy Research, September 2003. 

[45] [hyperlink, http://www.gao.gov/products/GAO-04-109]. 

[46] Diane K. Levy and Mark Wookey, Relocation Is Not Enough: 
Employment Barriers among HOPE VI Families, Metropolitan Housing and 
Communities Center of the Urban Institute, Brief No. 6 (June 2007). 

[47] Susan J. Popkin, Diane K. Levy, and Larry Buron, "Has HOPE VI 
Transformed Residents' Lives? New Evidence from the HOPE VI Panel 
Study," Housing Studies, 24, no. 4 (July 2009). 

[48] HUD, Office of Public and Indian Housing and Office of Policy 
Research and Development, Moving to Work: Interim Policy Applications 
and the Future of the Demonstration, a report to Congress (Washington, 
D.C.: 2010). 

[49] Martin D. Abravanel, Robin E. Smith, et al., Housing Agency 
Responses to Federal Deregulation: An Assessment of HUD's "Moving to 
Work" Demonstration (Washington, D.C.: The Urban Institute, 2004). 

[50] HUD defines rent reform as any change from the Code of Federal 
Regulations to the way rent is calculated for an assisted household. 
This includes adopting alternative recertification schedules, flat or 
tiered rent structures, modifications to current exclusion and 
deduction requirements, and alternate income verification procedures. 

[51] HUD, HUD Strategic Plan Fiscal Year 2010-2015 (May 2010). 

[52] HUD, Promoting Partnerships to Utilize Housing as a Platform for 
Improving Quality of Life, HUD Notice PIH-2011-51 (Sept. 20, 2011). 

[53] Resources for Community and Supportive Services, [hyperlink, 
http://portal.hud.gov/hudportal/HUD?src=/program_offices/public_indian_h
using/programs/ph/hope6/css/resources], accessed June 11, 2013. 

[54] Although program requirements for the MTW program do not 
specifically call for PHAs to coordinate with local service providers, 
MTW agencies can use their funding to contract with local agencies to 
provide supportive services. 

[55] For more information on our selection methodology, see appendix I. 

[56] HUD and Labor released a joint letter on May 26, 2009, 
encouraging PHAs and workforce agencies to work collaboratively to 
identify opportunities to train and place public housing residents 
into jobs created by PHAs' capital improvement projects funded by the 
American Recovery and Reinvestment Act of 2009. 

[57] GAO, Standards for Internal Control in the Federal Government, 
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] 
(Washington, D.C.: Nov. 1999). 

[58] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. 

[59] Interviews with these PHAs also covered other topics including 
how the PHAs collect and track participation and outcomes data for 
their self-sufficiency programs. 

[60] We consolidated size categories from HUD's Housing Authority 
Profiles database into the following three categories: small (fewer 
than 250 total units); medium (more than 249 units but fewer than 
1,250 units); and large (more than 1,249 units). We then combined the 
small and medium categories because very few PHAs participating in 
these five programs would meet the small category standards. For 
regions, we used the four Census Bureau regions: Midwest, Northwest, 
South, and West. 

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 
reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e-mail you a list of newly 
posted products, go to [hyperlink, http://www.gao.gov] and select 
“E-mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 
information. 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov]. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; 
E-mail: fraudnet@gao.gov; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Katherine Siggerud, Managing Director, siggerudk@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, DC 20548. 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, DC 20548. 

[End of document]