This is the accessible text file for GAO report number GAO-02-752
entitled 'School Vouchers: Characteristics of Privately Funded 
Programs' which was released on September 10, 2002. 

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States General Accounting Office: 
GAO: 

Report to the Honorable Judd Gregg, U.S. Senate: 

September 2002: 

School Vouchers: 

Characteristics of Privately Funded Programs: 

GAO-02-752: 

Contents: 

Letter: 

Results in Brief: 

Background: 

Programs Shared Characteristics and Challenges, but Varied in Size and 
Voucher Amounts: 

Recent Research on Privately Funded Voucher Programs Has Explored 
Academic Achievement and Parental Perceptions of Their Children’s 
Schools: 

Agency Comments: 

Appendixes: 

Appendix I: Description of the Privately Funded Voucher Studies 
Reviewed: 

Appendix II: Comments from the Department of Education: 

Appendix III: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

Staff Acknowledgments: 

Bibliography: 

Reports on the New York Study: 

Reports on the Dayton Study: 

Reports on the Washington, D.C., Study: 

Reports on the Studies in All Three Cities: 

Tables: 

Table 1: CSF Annual Income Eligibility Scale for School Year 2001-02 
Vouchers: 

Table 2: Details of Privately Funded Voucher Studies: 

Table 3: Ethnicity of Mothers of Voucher Applicants Participating in 
Studies: 

Table 4: New York City Study Summary—1997 through 2000 (3 academic 
years): 

Table 5: Dayton, Ohio, Study Summary—1998 through 2000 (2 academic 
years): 

Table 6: Washington, D.C., Study Summary—1998 through 2001 (3 academic 
years): 

Abbreviations: 

CFA: Children First America: 

CSF: Children’s Scholarship Fund: 

BASIC: Bay Area Scholarships for Inner-city Children: 

[End of section] 

United States General Accounting Office: 
Washington, D.C. 20548: 

September 10, 2002: 

The Honorable Judd Gregg: 
United States Senate: 

Dear Senator Gregg: 

Privately funded voucher programs are a relatively new development in 
the nation’s experiment with school vouchers. These programs, started 
in the early 1990s, provide low-income families with private, 
nongovernmental tuition assistance at private schools for kindergarten 
through grade 12. While private schools have long offered various forms 
of financial assistance, many of these privately funded voucher 
programs are different from traditional scholarship efforts in two key 
respects: they are open to any applicant solely on the basis of family 
income level, and recipients are free to decide which schools their 
children should attend. Such programs now serve about 46,000 of the 
estimated 53 million school age children nationwide, awarding nearly 
$60 million in tuition assistance. 

Privately funded voucher programs are becoming part of an evolving 
approach to achieving greater school choice. The Congress continues to
show a strong interest in school choice issues—debating a number of
choice issues, enacting choice provisions in Title I of the No Child 
Left Behind Act of 2001, and enacting the Public Charter Schools 
program and the Magnet Schools Assistance program. It was within this 
context that you asked us to provide information on both publicly 
funded and privately funded voucher programs. We addressed publicly 
funded voucher programs in a prior report. [Footnote 1] This report on 
privately funded voucher programs focuses on answers to the following 
questions: 

* What are the characteristics of privately funded school voucher
programs, including such factors as amount of tuition assistance,
determination of student eligibility, and long-term challenges? 

* What is known about the academic performance of students
participating in these programs and the degree of parental satisfaction
with the programs? 

To report on the characteristics of the programs and long-term 
challenges, we focused on the 78 privately funded voucher programs that 
were open to all low-income applicants and allowed recipients a wide 
choice in deciding what schools their children would attend. We 
developed our list in consultation with officials from two major 
national organizations with which many programs are affiliated and 
summarized information on these programs collected by the 
organizations. In addition, we searched the literature, reviewed 
reports and other documents, and corroborated information through on-
site and telephone interviews with local programs and national 
organization officials. To determine what is known about student 
academic performance and parent satisfaction associated with privately 
funded school voucher programs, we relied on existing studies. We 
identified 11 studies, but for our review only included those that:
(1) analyzed student academic achievement and/or parental satisfaction,
(2) compared voucher students or their parents with an appropriate 
control or comparison group of students or parents, and (3) gathered 
data on student achievement or parental satisfaction both before and 
after the vouchers were awarded. For our analysis, we included findings 
from those studies that reached the 95 percent confidence level as 
statistically significant effects. (There is a 95 percent certainty 
that these results would not occur by chance alone.) See appendix I for 
detailed information about these studies. Two social scientists 
examined each study to assess the adequacy of the samples and measures 
employed, the reasonableness and rigor of the statistical techniques 
used to analyze them, and the validity of the results and conclusions 
that were drawn from the analyses. We conducted our work between 
October 2001 and April 2002 in accordance with generally accepted 
government auditing standards. 

Results in Brief: 

The 78 privately funded voucher programs we reviewed shared numerous
characteristics and faced common challenges, but the programs varied
widely in the dollar amount of the vouchers they awarded and the number
of students receiving them. While all programs used income-based
eligibility criteria, many used a sliding scale based on family income 
and number of family members to determine award amounts. Programs
typically required a minimum parental contribution toward the tuition, 
and many automatically accepted siblings and awarded vouchers using a
lottery. The average voucher amount in school year 2001-02 ranged widely
among programs from about $600 to about $2,000 per student and program
size ranged from as few as 4 students, to over 3,000 students. Beyond 
the information needed to determine eligibility and conduct financial
oversight, most programs reported collecting little data about 
participating students, their families, or the schools they attended. 
Looking to the future, program officials said voucher programs face two 
major challenges— specifically, sustaining programs for the long term 
and retaining students in the programs. Some programs we contacted have 
begun developing ways to address these challenges. 

Rigorous evaluations of privately funded voucher programs in New York,
Dayton, and Washington, D.C., provide some evidence that African
American students who used vouchers to attend private schools showed
greater improvements in math and reading than students in the comparison
group, and have also found that the parents of voucher users of all 
racial and ethnic groups were consistently more satisfied with their 
children’s education than parents of comparison group students. More 
specifically, the New York study found consistently greater improvement 
in math and reading achievement for African American elementary 
students using privately funded vouchers. Voucher users in Dayton 
showed no significant improvements in reading or math test scores. The 
Washington, D.C., study demonstrated positive effects for African 
American students in the second year of the study, but these 
disappeared in the third and final year of the study. No significant 
differences were found in any of the studies for students in other 
ethnic or racial categories. The three studies also found that families 
that used vouchers were generally more satisfied with their children’s 
schooling on such factors as safety and academic quality than were 
similar families in the comparison groups. In addition, parents of
children using privately funded vouchers reported that their children’s
schools communicated with them more frequently and had a more positive
environment as shown by less disorderly behavior among students. While
the results of these studies suggest positive academic achievement 
effects for African American students and families in New York City, 
the programs examined were relatively small in scale, therefore, the 
findings cannot be generalized beyond the specific programs and 
geographic areas where they were conducted. 

Background: 

Privately funded school voucher programs got their start in 1991 when an
Indianapolis businessman founded a local program that provided tuition
assistance to about 750 low-income students in grades kindergarten
through 8 (K-8). After the initiation of the Indianapolis program, a 
number of other communities also established privately funded voucher 
programs based on the Indianapolis model and used funding from local 
donors. In 1994, a local voucher program that had been founded in San 
Antonio obtained a grant to establish a national clearinghouse 
organization—now called Children First America (CFA) Footnote 2] —to 
assist new and existing local programs with funding and technical 
assistance. By 1997, there were 31 local programs offering over 12,000 
privately funded vouchers to K-12 students in 18 states plus the 
District of Columbia. 

In 1998, a second national organization, the Children’s Scholarship Fund
(CSF), was established. CSF used the word scholarship in its name—rather
than voucher—to distinguish itself from publicly funded voucher
initiatives. CSF helped found several new local voucher programs and
established partnerships with many existing programs. In its first 
year, CSF provided one-to-one matching funds for a total of 40 programs 
to provide vouchers for low-income students for a period of 4 years. 
Beginning in school year 1999-2000, CSF-funded programs provided school 
vouchers to about 40,000 K-8 students. CSF currently provides support 
to 47 affiliated programs. Nine of the programs are administered by the 
CSF national office, including a program that provides vouchers to 
students not residing in an area with a local program. Like CSF, CFA 
continues to be a source of funding support and technical assistance to 
local privately funded voucher programs. The two organizations work 
with each other in a number of efforts but differ somewhat in their 
stated missions. CSF offers tuition assistance to needy families in 
what it describes as a purely charitable effort, while CFA states that 
its mission is to promote parental choice, including both privately and 
publicly funded options. 

In addition to programs offering vouchers for low-income students to 
use, with virtually no restriction as to their choice of private 
schools, there are other privately funded tuition support or 
scholarship programs. However, these programs fall outside the scope of 
our study because they limit student eligibility or school choice in 
different ways. For example, we did not review traditional scholarships 
offered by private schools, merit-based programs, or programs that 
limit choice to schools of a particular religious denomination. 

According to a recent survey done by CSF, 78 privately funded school
voucher programs used family income as their only eligibility criteria 
and permitted families to use their award at nearly any private school. 
Although these programs receive their funding from private individuals 
and groups— in contrast with publicly funded school voucher programs 
[Footnote 3] —they nonetheless may affect public funding in several 
ways. Most programs are not-for-profit organizations and, as such, are 
eligible for the associated federal tax benefits. [Footnote 4] Programs 
may also have an impact on local public school funding because much 
state and some federal funding to school districts is allocated on the 
basis of formulas incorporating the number of students attending the 
schools. Some states have laws in place to provide state tax credits to 
individuals or businesses for their contributions to tuition assistance 
organizations, including privately funded voucher programs. [Footnote 
5] 

Programs Shared Characteristics and Challenges, but Varied in Size and 
Voucher Amounts: 

The privately funded voucher programs we reviewed had a number of
common characteristics, including several long-term challenges, but they
varied widely in the dollar amounts of the vouchers they awarded and
number of students served. Most programs had local financial support.
Programs also used similar methods to determine student eligibility and
many programs used a sliding scale based on family size and income.
Nearly all programs required parents to contribute at least a specified
minimum amount toward the cost of tuition. Many automatically accepted
siblings and some ensured multiyear funding as part of an emphasis on
helping families. Most programs collected only the information they
needed to determine student eligibility and administer the programs, 
such as family income and number of siblings. However, information on 
other student and family characteristics was limited. Looking to the 
future, program officials said voucher programs face two major 
challenges— specifically, sustaining programs for the long term and 
retaining students in the programs. Some programs we contacted have 
begun developing ways to address these challenges. 

Programs Had Many Common Characteristics but Differed Widely in Numbers 
of Students Served and Voucher Amounts: 

Although scattered across the United States, the local privately funded
voucher programs we studied shared many things in common. For
example, most were largely supported by local donations, and the 
majority of the 47 programs affiliated with CSF raised funds locally to 
receive matching funds from that national organization. Often programs 
were supported by one, or a few, “anchor” donors—people or 
organizations that had committed to making a substantial, and often 
multiyear, financial contribution to support vouchers in their 
community. For example, a single local donor committed to provide the 
matching funds for the Children’s Scholarship Fund of Seattle/Tacoma 
for 4 years, and in San Antonio two local donors have provided nearly 
all of the funding for two voucher programs serving that area. 

Programs also had similar voucher recipient selection processes. Most
programs selected recipients by lottery from among all eligible 
applicants, although a few selected them on a first-come-first-serve 
basis. According to officials, some programs awarded vouchers to all 
eligible students in the first year because they had more money for 
vouchers than they had qualified applicants. As more people became 
aware of the vouchers and demand grew, nearly every program adopted a 
lottery system. Many programs have established separate lotteries for 
private and public school students. Several program officials pointed 
out that awarding tuition assistance to students already in private 
school was not accomplishing the mission of extending school choice 
because those families had already exercised choice. Programs we 
contacted typically awarded a larger percent of their vouchers to 
students transferring from public schools and established a specific 
limit to the percent that would be awarded to students already in 
private school. For example, the Washington Scholarship Fund, in 
Washington, D.C., reported that it awards at least 75 percent of its 
vouchers to students coming from public schools. The program maintains 
two pools of applicants—one for public school students and one for 
private school students—and selects students by separate lottery from 
each pool. 

Eligibility criteria were also similar among programs. Eligibility 
depended on both family income and size. Almost all the programs 
targeted the vouchers to families eligible for the federal free and 
reduced price lunch program—those making less than 185 percent of the 
federal poverty guidelines. However, many programs included families 
with incomes up to 270 percent of the poverty guidelines. Many programs 
also used a sliding scale based on family income and size to determine 
both eligibility and how large a voucher the family could receive. 
Families with higher incomes received a smaller percentage of school 
tuition. The CSF national organization has established sliding scale 
income eligibility standards for its affiliated programs. See table 1 
for the CSF 2001-02 income eligibility scale. Programs also typically 
had a maximum voucher amount that could be awarded. Maximum award 
amounts for CSF programs varied from program to program but ranged from 
$1,000 to $2,100 for K-8 vouchers and $1,000 to $2,900 for high school 
vouchers. Since voucher amounts did not typically cover the full amount 
of tuition, the family was responsible for the balance. 

Table 1: CSF Annual Income Eligibility Scale for School Year 2001-02 
Vouchers: 

Household size: 2; 
Household income level for voucher of up to 75% of tuition: $0 - 
$11,250; 
Household income level for voucher of up to 50% of tuition: $11,251 - 
$20,813; 
Household income level for voucher of up to 25% of tuition: $20,814 - 
$30,375. 

Household size: 3; 
Household income level for voucher of up to 75% of tuition: $0 - 
$14,150; 
Household income level for voucher of up to 50% of tuition: $14,151 - 
$26,178; 
Household income level for voucher of up to 25% of tuition: $26,179- 
$38,205. 

Household size: 4; 
Household income level for voucher of up to 75% of tuition: $0 - 
$17,050; 
Household income level for voucher of up to 50% of tuition: $17,051 - 
$31,543; 
Household income level for voucher of up to 25% of tuition: $31,544 - 
$46,035. 

Household size: 5; 
Household income level for voucher of up to 75% of tuition: $0 - 
$19,950; 
Household income level for voucher of up to 50% of tuition: $19,951 - 
$36,908; 
Household income level for voucher of up to 25% of tuition: $36,909 - 
$53,865. 

Household size: For each additional child; 
Household income level for voucher of up to 75% of tuition: Add $2,900 
to household income; 
Household income level for voucher of up to 50% of tuition: Add $5,365 
to household income; 
Household income level for voucher of up to 25% of tuition: Add $7,830 
to household income. 

Source: Children’s Scholarship Fund Program Manual. 

[End of table] 

Despite the size of a voucher for which a family is eligible, most 
programs required families to contribute a minimum amount toward their 
child’s tuition payment—at least about $500. However, many programs do 
not require families with more than one child receiving a voucher to 
pay the minimum amount for each child. For example, in Buffalo, New 
York, the program required each family to pay just $500 toward the 
tuition, regardless of the number of children receiving a scholarship.
For many programs, including all CSF affiliated programs, siblings of
award recipients were automatically offered vouchers. In addition, the
programs typically made a commitment to the families to provide the
vouchers for multiple years, as long as the family remained eligible 
for the program. For example, in Cincinnati, all children in the family 
can receive a voucher, and eligible students will continue to receive a 
voucher for at least 4 years. Officials said accepting siblings and 
committing to a multiyear voucher were part of their approach to 
helping the children and the families. In addition, CSF plans to 
continue its commitment to families by extending financial support for 
vouchers through the year 2013 to include all younger siblings through 
the eighth grade. 

Programs we contacted also conducted similar financial oversight
activities. They typically paid voucher amounts directly to the private
schools in several tuition installments throughout the school year. 
Prior to each payment, programs received confirmation from 
participating private schools that the voucher students were still 
enrolled. Many programs also required confirmation that the students 
were attending class regularly and some required parents to sign a 
confirmation form each time a payment was made. For example, the 
Houston Children’s Educational Opportunity Foundation pays voucher 
installments to participating schools every month during the school 
year. Prior to each payment, the program official sends a commitment 
form to each school. A school official and the parents of each voucher 
student are required to sign it. The CSF national organization directs 
its affiliated programs to have the parents of voucher students visit
their schools to sign a Scholarship Verification Report. The form is 
also signed by the school principal and is returned to the program 
before payment is made. Program officials are instructed to retain the 
forms as an audit trail of the scholarships and proof that they were 
verified. 

Although the programs shared many similar characteristics, they differed
in the average dollar amount of their voucher awards. Award amounts are
based, in part, on school tuition and the tuition at participating 
private schools varied considerably among program locations. For 
example, for K-8 voucher recipients in school year 2001-02, the average 
private school tuition in Buffalo, New York, was about $1,500 and the 
average K-8 voucher award amount for the Buffalo program was $620. The 
maximum Buffalo program award was $1,000, depending on family size and 
income. In contrast, the private schools attended by K-8 voucher 
students in the Atlanta CSF program charged an average tuition of about 
$4,300 and the average K-8 voucher amount for the Atlanta program was 
$1,663—with an award maximum of $2,100. The average percent of tuition 
covered by CSF program vouchers ranged from about 20 percent in 
Pittsburgh to about 65 percent in Baton Rouge. Overall, the average K-8 
voucher award for the 47 CSF affiliated programs was about $1,100, 
according to CSF officials, and the average per-child K-8 tuition among 
all programs that receive funding from the CSF was about $2,550. 
Average high school voucher awards from individual programs also varied 
widely from $1,009 to $2,411. The Department of Education reports that 
the average Catholic or other religious private school tuition for 
grades K-12 in school year 2000 was about $2,800 and for private 
nonsectarian schools was about $8,900. 

The number of vouchers awarded per program also varied considerably.
For example, programs in Phoenix; St. Louis; Erie, Pennsylvania; and
Midland, Texas, awarded 10 or fewer scholarships each, while the 
largest— the CSF local program in New York City—awarded over 3,100. 
Seventeen of the local programs we studied awarded more than 1,000 
vouchers. In addition, the national CSF organization awarded about 
3,600 vouchers, through its national program, to eligible students not 
living in a local area served by a voucher program. 

Information on the students or families who receive privately funded
vouchers is limited, as is information on the private schools the 
students attend. Most programs we contacted reported that they 
collected only the student and family data needed to determine student 
eligibility, and the administrative data needed to manage and oversee 
program activities. The eligibility data collected by programs included 
family income, size, and place of residence, and the administrative and 
oversight data included the amounts of the vouchers awarded and the 
tuition at the schools voucher students attended. Information on the 
private schools students attended was often limited to the school 
affiliation—for example, Catholic, nondenominational Christian, or 
independent. Programs reported that the majority of the schools voucher 
students attend are religiously affiliated. [Footnote 6] Most programs 
we contacted did not collect information about the public schools 
voucher students previously attended, and most did not gather data to 
evaluate program outcomes or results. While some program officials said 
they saw value in such analyses, several said that evaluations were 
costly and that they preferred to use their limited funds for student
vouchers. 

Some Programs Initiated Activities to Meet Future Challenges: 

Through our discussions with program officials, we identified two major
challenges faced by privately funded voucher programs—ensuring program
sustainability and maximizing student retention rates. We found that 
many of the programs had recognized these challenges and identified or
implemented activities and initiatives to address them. 

Sustaining Programs for the Long Term: 

Most programs we contacted were concerned with sustaining their
activities over the long term. Officials generally said they expected 
their programs to continue operating for the foreseeable future or 
indefinitely. Some had determined that certain program modifications 
would be needed to foster sustainability. Specifically, these programs 
recognized that they should broaden their funding base and have more 
community representation on their boards of directors. Several programs 
had been founded and initially funded largely by a few local donors. In 
many cases, the donors had agreed to provide funding for 4 years. One 
program official said he believed there was a danger of what he called 
“donor fatigue” and determined it could be averted by expanding the 
number of donors— particularly small contributors. Some programs also 
saw the benefits of broadening community representation on their boards 
of directors. For example, the Washington Scholarship Fund told us it 
is currently building a broad donor base and is expanding its board of 
directors—which had been limited to large donors—to include members who 
may not be able to provide funding, but are actively involved in the 
community. Additionally, an official at Children First Utah told us 
that to help ensure stability and sustainability, the program has 
established a board of directors with broad community representation, 
including representatives from several large corporations and a public 
relations firm, community activists, and a representative from the 
Hispanic community. In addition, Children First Utah has implemented 
efforts to broaden its funding base, including direct mail 
solicitations, media public relations efforts, and several funding
proposals to foundations. 

The two national private voucher organizations, CFA and CSF, support
local programs in both implementing and sustaining their activities. In
addition to providing matching funds to some programs, they also provide
technical assistance. CFA offers a detailed “how to” manual to new
programs and will work on site with programs as needed at no cost. CFA
officials told us they also work with existing programs to help them 
solve problems and enhance program stability, and they emphasized that 
their focus is on the sustainability of the programs. For example, CFA 
assists programs in broadening the community representation on their 
boards and among their donors. According to CSF officials, that 
organization provides its affiliates with extensive guidance and 
technical support for program administration activities such as 
scholarship disbursement procedures, enrollment verification, and 
financial planning. CSF also holds an annual conference for its 
affiliates, provides a detailed program management manual, and assists 
affiliates in the use of a sophisticated program administration 
database. 

Maximizing Student Retention: 

Maximizing the retention rate among voucher recipients was another
challenge faced by the programs we contacted. In many programs, about
20 percent or more of the voucher recipients leave the program each 
year. However, according to a CSF official, only a small percent of 
voucher recipients leave CSF affiliated programs because they graduate 
from the eighth grade. Program officials said that other reasons 
recipients leave might include moving or dissatisfaction with the 
school. Many said they believe family financial difficulties are a 
common cause of attrition. According to some officials, families are 
sometimes unprepared for all the costs of having children in private 
schools—for example, uniforms, books, and activity fees. Additionally, 
parents of voucher students may be confronted with financial 
emergencies that preclude their paying their required portion of the 
tuition and continuing in the program. Some programs we contacted 
attempted to determine the reasons students drop out and several had 
initiated activities to promote student retention. For example, the CSF 
program in New York City and the Washington Scholarship Fund have 
implemented “stay in school funds” to help parents who are experiencing 
financial emergencies such as unexpected medical expenses. Washington 
Scholarship Fund, which served nearly 900 families in school year 2001-
02, reported that it has provided such funding assistance to about 26 
families so far. The Horizon program in San Antonio, Texas, initiated a 
voucher parent group to increase the student retention rate. According 
to a program official, the group, “Las Comadres,” meets weekly to 
discuss aspects of their children’s education and learn from each other 
about schools and curricula. The meetings also include guest speakers 
on such topics as preparing children for high school and college,
parental rights and duties, and children’s health issues. The Horizon
program director and staff also attend the meetings to facilitate and 
answer questions. 

Recent Research on Privately Funded Voucher Programs Has Explored 
Academic Achievement and Parental Perceptions of Their Children’s 
Schools: 

We identified three studies of privately funded voucher programs—in New
York; Dayton, Ohio; and Washington, D.C.—that were rigorous enough to
meet our criteria for inclusion. These studies all included quantitative
analysis of program effects, used appropriate comparison groups, and
gathered data before vouchers were awarded, as well as after, to assess 
the equivalence of study groups and to track program effects. In 
general, researchers found that African American students in New York 
City who used vouchers to attend private schools exhibited more 
substantial improvements on test scores in math and reading than 
African American students in the comparison group. For students from 
other ethnic groups, there were no sizable or significant differences 
in the test score gains of voucher users and comparison group students, 
or there were too few participants to draw any conclusions. These 
studies have also found that the parents of voucher users in all 
programs were consistently more satisfied with their children’s 
education, regardless of ethnic group. Although these results suggest 
positive achievement effects for some African American students using 
vouchers, they cannot be generalized beyond the specific programs, 
schools, and geographic areas where the studies were conducted or the 
low-income group of families studied. 

Studies Used Same Rigorous Methodology: 

The studies of privately funded voucher programs in New York; Dayton,
Ohio; and Washington, D.C., all used impact evaluations to study the 
effect of vouchers on private school attendance. Impact evaluations 
isolate a program effect from the effects of other factors that could 
influence participants’ outcomes. To isolate the program’s effect, 
impact evaluations divide participants into two groups: those who 
receive program services and a similar group who do not (the control or 
comparison group [Footnote 7]). Some impact evaluations assign 
participants randomly to one group or the other. Random assignment 
increases the likelihood that the two groups are roughly equivalent on 
all characteristics that could affect outcomes. It helps ensure, for 
example, roughly equal numbers of very low-income students in both 
groups, rather than those students being concentrated more heavily in 
one group or the other. The use of random assignment to create 
equivalent groups allows researchers to compare tests scores (or other 
outcomes) for the two groups and attribute any differences to the
program services rather than other factors, such as differences in 
family income. 

The studies of voucher programs in New York, Dayton, and Washington
randomly assigned applicant families to the voucher offer group and the
control (no voucher offer) group. Before each voucher program began,
students whose families hoped to receive a voucher took the Iowa Test of
Basic Skills in math and reading, and their parents completed surveys
about family background characteristics and their children’s educational
experiences. The researchers also screened families to determine if they
were eligible to participate on the basis of their incomes. Test scores 
and information about family background provided researchers with a 
baseline for making later comparisons. Families who met the income 
qualifications and participated in the testing and survey sessions were 
eligible to enter a lottery to decide which families would receive 
voucher offers and which would not. Because the researchers ensured 
that the lotteries were completely random, the two groups—voucher offer 
recipients and nonrecipients—should have roughly equivalent average 
student test scores and family backgrounds at the outset. To collect 
follow-up data from both voucher offer recipients and nonrecipients in 
subsequent years of the studies, the researchers used the same testing 
and survey procedures. [Footnote 8] Only students attending a public 
school before the lottery were included in the follow-up testing and 
analysis. Table 2 provides additional details about the studies, 
including the numbers of applicants and vouchers awarded. 

Table 2: Details of Privately Funded Voucher Studies: 

First school year under study: 
New York: 1997-98; 
Dayton: 1998-99; 
Washington: 1998-99. 

Years of study follow-up: 
New York: 3; 
Dayton: 2; 
Washington: 3[A]. 

Entry grades of students participating studies[B]: 
New York: 1-5; 
Dayton: 1-9; 
Washington: 1-9. 

Number of students tested before the lottery: 
New York: 1,960; 
Dayton: 803; 
Washington: 1,582. 

Number of vouchers awarded: 
New York: 1,500; 
Dayton: 515; 
Washington: 811. 

Maximum voucher amount: 
New York: $1,400; 
Dayton: $1,200; 
Washington: $1,700. 

Number of private schools attended voucher users[C]: 
New York: 216; 
Dayton: 42; 
Washington: 116. 

Note: The numbers reported here include only students whom the 
researchers included in the studies. The programs awarded additional 
vouchers to students not included in the studies because of their
entry grade or they were already attending a private school. 

[A] Although the Washington study lasted for 3 years, complete results 
from the third year follow-up sessions were unavailable at the time 
this was written. Only composite test scores (combined reading and 
math) were available. Individual reading and math scores and the 
results of the parental survey had not yet been released. 

[B] Students entering first grade were not tested at baseline, and 
students did not take follow-up tests beyond eighth grade. 

[C] These numbers represent the number of schools attended by all of 
the programs’ students in the 2000-01 school year, not just by those 
included in the studies. Because not all students in the programs 
participated in the studies, the number of schools attended by students 
participating in the studies is likely to be somewhat smaller. 

Source: William G. Howell and Paul E. Peterson, The Education Gap: 
Vouchers and Urban Schools (Washington, D.C.: Brookings Institution 
Press, 2002). 

[End of table] 

While the use of control groups helps isolate the cause of any outcome,
researchers must still address circumstances that threaten the quality 
of their results. In the New York, Dayton, and Washington studies, the
researchers addressed two such circumstances. First, not all of those
offered a voucher used their voucher to attend a private school, and 
some students who did not receive a voucher offer ended up in a private 
school anyway. This meant that a comparison between the voucher offer 
group and control group, as designed, revealed the effect of offering a 
voucher to eligible students in a city, not the effect of actually 
using it to attend a private school. To answer the question of the 
effects of vouchers on those actually using them, the researchers used 
a data analysis technique that allowed them to compare the outcomes of 
those who actually used a voucher to attend private school to those of 
students who were most likely to have attended private school had they 
been offered a voucher. [Footnote 9] While both the voucher offer and 
voucher use analyses are valid, they answer different questions. The 
researchers reported primarily on the voucher use analysis. Second, 
substantial numbers of those who initially agreed to participate in the 
studies dropped out in subsequent years. When this occurs, those who 
remain in the study may be different from those who left, which can 
threaten the equivalence of the two study groups, making it difficult 
to assess the effect of the voucher offer. The researchers used a
standard statistical procedure [Footnote 10] to minimize the 
possibility that the loss of some participants undermined their 
results. 

In all three studies, the demographic characteristics of study 
participants surveyed at baseline reflected the programs’ targeting of 
vouchers to low-income families, most of whom lived in inner cities. 
Although these families met program eligibility requirements as low-
income, they were able to pay at least a portion of the private school 
tuition. In both Dayton and Washington, the average income of a 
participant family that received a voucher and used it to attend 
private school was approximately $18,000. However, in New York, the 
average family income of participating voucher users was about $10,000, 
possibly reflecting the more stringent income criteria of the New York 
program. In all three cities, ethnic minorities made up a large 
majority of voucher applicants participating in the studies. According 
to the survey questions, study participants were asked to report their 
race as either Black/African American (non-Hispanic), White (non-
Hispanic), Puerto Rican, Dominican, other Hispanic, American Indian or
Alaskan Native, Chinese, other Asian or Pacific Islander, or other. 
Thus, the African Americans and Whites in the study are likely to be 
non-Hispanics only. (See table 3) Among study participants who made use 
of the voucher offered to them, a majority attended Catholic schools in 
all three cities. 

Table 3: Ethnicity of Mothers of Voucher Applicants Participating in 
Studies (in percentages): 

African American, Non-Hispanic: 
New York: 44; 
Dayton: 70; 
Washington[A]: 95. 

White, Non-Hispanic: 
New York: 5; 
Dayton: 29; 
Washington[A]: 1. 

Hispanic: 
New York: 47; 
Dayton: 0; 
Washington[A]: 3. 

All other: 
New York: 4; 
Dayton: 1; 
Washington[A]: 2. 

[A] Total does not equal 100 percent due to rounding. 

Sources: Paul E. Peterson, Jay P. Greene, William G. Howell, and 
William McCready, Initial Findings from an Evaluation of School Choice 
Programs in Washington, D.C., and Dayton, Ohio (Paper presented at the 
annual meeting of the Association of Public Policy and Management. New 
York City, October 1998); Paul E. Peterson, David Myers, Josh Haimson, 
and William G. Howell, Initial Findings from the Evaluation of the New 
York School Choice Scholarships Program (Washington, D.C.: Mathematica 
Policy Research, November 1997). 

[End of table] 

African American Voucher Users in New York City Had Improved Test 
Scores: 

Of the three studies, the New York study showed the greatest positive
effect on voucher users’ academic achievement. In each of the 3 years of
the New York study, African American students who used a voucher to
attend private school performed better on the reading exams than a
comparable control group. African American voucher users also performed
better in math than those who did not receive a voucher and remained in
public schools. [Footnote 11] This improvement in math scores was found 
in years 1 and 3, but not year 2, of the study. [Footnote 12] 
Achievement gains, however, did not extend to Hispanic voucher users. 
The New York study found that Hispanic students who used privately 
funded vouchers to switch from public to private school had math and 
reading scores that were statistically the same as those of the control 
group. The New York study did not have sufficient numbers of white 
students or students from other ethnic groups to estimate program 
effects for them. In addition, when the New York study sample is 
considered as a whole—pooling together African Americans, Hispanics, 
and white students—there is no significant difference in achievement 
gains between voucher users and nonusers. Although the researchers 
conducted additional analyses to explain why these positive effects 
were seen for African American students and not for others, the cause 
of this difference remains unclear. In Dayton, African American voucher 
users showed improvements in reading in the second year of the study, 
an effect that approached, but did not reach, statistical significance. 
[Footnote 13] No other effects on test scores were found for African
American students or for students of any other racial or ethnic 
background. 

In the Washington, D.C., study, the academic achievement of African
American students who used a voucher to switch from public to private
school was not consistently higher over the 3 years of the study than 
that of the control group of African American students who remained in 
public schools. In the first year of the study, African American 
voucher users scored better than control group students in math, but 
worse in reading. In the second year, African American voucher users 
scored significantly better in both math and reading than the students 
who remained in public schools. In the third and final year of the 
study, however, there was no difference between the combined math and 
reading test scores of African American voucher users and those of the 
control group. [Footnote 14] Because a large majority of participants 
in the Washington study were African American, there were not enough 
students of other ethnic groups to make a reliable estimate of program 
effects. 

Several limitations prevent reaching more definitive conclusions from 
this body of research. Substantial numbers of both voucher recipients 
and nonrecipients who were tested at baseline did not return for follow-
up testing in subsequent years (program attrition). [Footnote 15] The 
researchers compensated for this by weighting the results of those who 
did return, but in Dayton and Washington, the problem of attrition from 
the studies was great enough that it was unclear if the procedures used 
could address it sufficiently. In addition to the problem of attrition 
from the study, the smaller initial sample size in Dayton may have 
resulted in estimates that were less precise and not statistically 
significant. In the Washington, D.C., study, another factor limited 
definitive study conclusions. Specifically, only 68 percent of those 
who were offered a voucher and who returned for follow-up testing the 
first year had actually used the voucher to attend a private school. 
This number declined to 47 percent and 29 percent in the second and 
third years, respectively. The analytical procedures used in the 
studies are more effective when a higher percentage of students who were
offered vouchers actually use them to attend a private school. The New
York study had the fewest problems with attrition and voucher use, and 
the estimated effects from the New York study are probably the most 
reliable. Finally, to our knowledge, at the time this report was 
written, only the New York data had been examined by researchers who 
were not part of the original research team. Confidence in the 
conclusions drawn from these studies will be enhanced when other 
researchers reanalyze these data and examine the assumptions underlying 
the original research. 

The findings of the studies show positive achievement gains [Footnote 
16] for low-income African American voucher users in New York City who 
applied for vouchers while in grades 1-4; however, the findings do not 
provide evidence that African American students elsewhere, or any other 
students, would realize achievement gains if they were offered vouchers 
to attend private schools. Differences in how voucher programs are 
designed and implemented, as well as differences in the participating 
students and in the local public and private schools affected by such 
programs, make it impossible to predict the effects of larger scale 
programs or programs similar in scale in different cities. The authors 
of the studies caution against generalizing from these results to a 
larger scale program involving all children in a large urban school 
system. They point out that only a small percentage of low-income 
students in the three cities received vouchers, and that the outcomes 
of a larger program could be quite different if the applicants to the 
study programs differ from eligible public school students in general. 
[Footnote 17] Additionally, the low-income families in the studies may
represent an even more distinct group: low-income as defined by program
eligibility requirements, but able to pay at least the minimum amount 
of the private school tuition required by the program. 

Parents Consistently Reported Satisfaction with Private School 
Instructional Programs, Teachers, and Environment: 

In addition to measuring student academic achievement, the studies of
privately funded voucher programs in New York, Dayton, and Washington,
D.C., used surveys to measure parents’ perceptions of their children’s
schools. The research team found that parents who used a voucher to send
their children to private schools were more likely to be satisfied with 
their children’s education overall and perceived their children’s 
schools to be better on a number of indicators, compared to parents in 
the control groups. These findings held true for all parents of voucher 
users, not only for African Americans. [Footnote 18] In all three 
cities in each year for which data are available, parents of voucher 
users were more likely than parents of control group students to give 
their child’s school an “A” on an A to F scale. [Footnote 19] In all 
three cities, in at least one study year, when asked about specific 
aspects of their children’s schools, the parents of voucher users were 
more likely than the parents of control group students to say they were 
“very satisfied” with school safety, teaching, and school curricula. In
all three years in New York, and in the second year in Dayton, the 
parents of voucher users were more likely to report being very 
satisfied with the academic quality of their child’s school than were 
the parents of students who did not receive a voucher. [Footnote 20] 
Parents of voucher users were also more likely to be very satisfied 
with discipline in their child’s school than were the parents of 
control group students in all 3 years of the New York study and in the 
first year of the Dayton and Washington studies. 

According to parents surveyed, the private schools attended by voucher
users in New York; Dayton; and Washington, D.C., had significantly
different characteristics than the public schools attended by the 
control group students. For example, in all three studies in every year 
for which data were available, [Footnote 21] the parents of voucher 
users reported their children’s schools had fewer students than did 
parents of the control group students. In at least 1 year for each of 
the three cities, parents of voucher users also reported that their 
children’s schools had smaller classes, were more likely to offer 
individual tutoring, and communicated with parents more frequently than 
did parents whose children remained in public schools. However, all 
three studies found that parents of voucher users reported that their 
children’s schools were less likely to have certain facilities. For
example, based on parents’ reports, all three studies in at least 1 
year found that the private schools used by voucher families were less 
likely than public schools attended by the control group students to 
have a nurse’s office. In New York and Washington, parents of voucher 
users were also less likely to report that their children’s schools had 
a cafeteria or offered programs for students with learning problems and 
non-English speakers than were parents of the control group students. 
[Footnote 22] 

The studies of privately funded voucher programs in New York; Dayton;
and Washington, D.C., found that the parents of voucher users reported 
less disruption in their children’s schools compared to the parents of 
control group students. Parents of voucher users were less likely than 
parents of control group students to report that fighting, truancy, 
cheating, or destruction of property were serious problems in their 
children’s schools. These results were true in all three studies in 
each year for which results were available, except for the second year 
of the Washington study, when only fighting showed a statistically 
significant difference. 

As in their analysis of test scores, several issues prevented the 
studies from resulting in more precise estimates of the effects of the 
voucher programs on parental perceptions of their children’s education. 
In Dayton and Washington, many results that were statistically 
significant in the first year of the studies were no longer significant 
in the second year. This may have been due to the low rates of study 
participants that returned for follow-up in both cities, the relatively 
small sample size in Dayton, and the low rate of voucher use and 
private school attendance by those offered vouchers in Washington. 
Despite these limitations, many of the effects were strong enough that 
statistically significant differences were found between the parents of 
voucher users and the parents of control group students. As with the 
analysis of test scores, the New York study had the least problems with
participants not returning for follow up and families declining voucher
offers, and probably produced the most reliable estimates. 

These three studies constitute an important first step toward
understanding the effect of school vouchers on certain low-income (and 
in the case of test score effects, African American) students. However, 
their results cannot be taken as evidence of the effects of vouchers on 
other types of students in any other settings. Furthermore, to our 
knowledge, at the time this report was written, only the New York data 
had been examined by anyone other than the original research team. 
Further analyses of these data by other researchers and additional high 
quality studies in other settings and involving different types of 
students and schools are important next steps in informing this ongoing 
public debate. 

Agency Comments: 

We received comments on a draft of this report from the Department of
Education. These comments are in appendix II. Education generally 
agreed with the report and provided technical comments that we 
incorporated where appropriate. Education also agreed that more 
research is needed on the effects of expanded parental choice. We also 
received technical comments from the researchers at Harvard and
Mathematica whose work we assessed and from officials at the Children’s
Scholarship Fund and Children First America. These comments were also
incorporated where appropriate. 

As agreed with your office, unless you publicly announce the contents of
this report earlier, we plan no further distribution until 16 days from 
the report date. At that time, we will send copies of this report to 
the Secretary of Education, appropriate congressional committees, and 
other interested parties. In addition, the report will be available at 
no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions or wish to discuss this material
further, please call me at (202) 512-7215 or David Bellis at (415) 904-
2272. 

Sincerely yours, 

Singed by: 

Marnie S. Shaul: 
Director: 
Education, Workforce, and Income Security Issues: 

[End of section] 

Appendix I: Description of the Privately Funded Voucher Studies 
Reviewed: 

The studies of privately funded voucher programs in New York; Dayton,
Ohio; and Washington, D.C., were similar and rigorous in both design and
implementation. All three were longitudinal studies, tracking 
participants over a 2 or 3 academic year period. In all three cities, 
researchers awarded vouchers randomly by lottery, thus maximizing the 
chances that those who received a voucher and those who did not receive 
a voucher (control group) were equivalent on those characteristics that 
could affect test score outcomes. At the outset of each study, 
researchers collected baseline data—information about the participants 
before the voucher program began—to help ensure equivalence of study 
groups and to compare against later outcomes. In addition to basic 
demographic information, the baseline data collected by the researchers 
included student achievement and parental satisfaction data. The 
researchers administered the math and reading sections of the Iowa Test 
of Basic Skills to applicant students who would be entering grades two 
and higher in the coming fall, when voucher use began. While the 
students were taking the tests, the accompanying parents and guardians 
filled out a survey that included questions about their satisfaction 
with a variety of aspects of their children’s education. The 
researchers required applicants to participate in the baseline tests and
surveys as a condition of entering the random lottery to win a privately
funded voucher. 

In each subsequent year of the studies, researchers again administered
math and reading tests to study participants and asked their parents to 
fill out surveys. The researchers invited both voucher and control group
members back for this follow-up data collection. To ensure that as many
participants as possible returned for follow-up, the researchers offered
incentives for the control group members and told voucher group members
that their participation was required as a condition of continuing to 
receive a voucher. [Footnote 23] Each year, the researchers analyzed 
the test scores to look for achievement differences between voucher and 
non-voucher students and analyzed the survey results for differences in 
parental satisfaction levels between the two groups. 

For all three studies, the researchers took two steps to ensure the 
validity of their results. First they weighted their data to adjust for 
participants who failed to return for follow-up data collection. The 
researchers used the known characteristics of study participants (e.g., 
family size, mother’s education, race, etc.) to estimate the 
probability that any given individual would return for follow-up 
testing. Using these calculated probabilities, they modified their data 
so that the responses of those who returned for follow-up testing, who 
were similar on these characteristics to the nonreturners, were counted 
more heavily in the analysis. This means that participants who did not 
return were represented by returning participants who were similar to 
them. [Footnote 24] 

The second way the researchers improved the quality of their results was
by using a statistical technique called an instrumental variable 
analysis to compensate for the fact that not all students who were 
offered a voucher actually used it to attend private school, and some 
control group students who were not offered a voucher attended private 
school anyway. Because of this crossover between the voucher offer and 
control groups, a comparison of outcomes between those offered a 
voucher and those not offered a voucher gave only the effect of the 
offer itself, not the effect of actually using a voucher to attend a 
private school. The instrumental variable analysis involved a 
statistical procedure in which the researchers used a student’s status 
as a voucher recipient or nonrecipient to predict whether that student 
attended private school. The researchers then used these results in a 
second statistical model, which gave the effect of actually using a 
voucher to attend a private school on test scores and survey results. 
[Footnote 25] 

Tables 4, 5, and 6 summarize the results and key strengths and 
weaknesses of the New York City; Dayton; and Washington, D.C., studies. 
They also provide our interpretation of the degree to which the studies 
provide evidence of any effect that voucher use had on test scores and 
parental satisfaction. 

Table 4: New York City Study Summary—1997 through 2000 (3 academic 
years): 

Study documents: 
* Daniel Mayer, Paul Peterson, David Myers, Christina Clark Tuttle, and 
William Howell, School Choice in New York City after Three Years: An 
Evaluation of the School Choice Scholarships Program (Washington, D.C.: 
Mathematica Policy Research, February 2002); 
* William Howell and Paul Peterson, The Education Gap: Vouchers and 
Urban Schools, (Washington, D.C.: The Brookings Institution, 2002); 
* William Howell, Patrick Wolf, Paul Peterson and David Campbell, Test-
Score Effects of School Vouchers in Dayton, Ohio, New York City, and
Washington, D.C.: Evidence from Randomized Field Trials (Paper 
presented at the annual meeting of the American Political Science 
Association, Washington, D.C., Sept. 2000). 

Reported results: 
* Study reports positive and statistically significant[A] differences 
on reading, math, and composite test scores between African American 
voucher users and the control (no voucher) group.[B,C] These results 
held for all 3 years of the study, except for math score differences 
which were not significant in the second year. 

* Study reports no significant differences between test scores of non-
African American voucher and control group students. The non-African 
American students in this study were primarily of Hispanic ethnicity. 

* On a range of issues on which parents were surveyed, the voucher 
group had the following outcomes in all 3 years of the study when 
compared to the control (no voucher) group: 
- Higher parental satisfaction (percent of parents giving the school a 
report card grade of “A,” and reporting satisfaction with academic 
quality, quality of teaching, curriculum, and school safety and 
discipline); 
- Attended smaller class and smaller schools; 
- More access to computer labs and tutors; 
- Less disorderly behavior (fighting, cheating, truancy, destruction of 
property); 
- More homework; 
- More communication from school to parents; 
- Less access to a cafeteria, a nurse’s office, and services for non-
English speakers and those with learning problems. 

Key strengths: 
* The study design involved vouchers being offered at random to 
eligible families. This helps ensure the equivalence of the 
characteristics of students using vouchers to attend private school and 
those of the control group. 

* An analytical technique—the instrumental variable approach[D]—was 
used to adjust for factors that threatened the equivalence of the study 
groups (e.g., some students offered vouchers did not use them to attend 
private schools and other students not offered vouchers attended private
school without them). 

* The researchers used an analytical technique—weighting[E]—to adjust 
for observed differences between the students who returned for testing 
and surveys 1, 2, and 3 years after the voucher program began and those 
who did not remain in the study. 

Key limitations: 

* The researchers acknowledge that these results cannot be generalized 
to larger scale programs, programs in other sites, programs serving 
other than low-income families or, in the case of the test score 
improvements, to groups other than African American students who 
applied for vouchers while in grades 1-4. 

* The analysis did not adjust for the possibility of numerous students 
attending the same school. Thus, it did not completely rule out the 
possibility that the positive effects were actually due to attendance 
at specific schools, rather than voucher use itself. 

Evidence of impact: 
* Evidence for positive impact on test scores for African American 
students. 

* No evidence of test score effects for Hispanic students. 

* Strong evidence for parental survey results as reported in “results” 
column, including increased parental satisfaction by voucher parents. 

[A] We typically use the 95 percent confidence level for determining an 
effect to be statistically significant. 

[B] Composite test scores are the average of the math and reading 
scores. 

[C] For ease of discussion, we use the terms voucher users and control 
(no voucher) group. The comparison of these groups involves students 
who used vouchers to attend private school and those who were likely to 
have used vouchers had vouchers been offered to them. 

[D] The instrumental variable approach involves a statistical procedure 
in which the researchers used a student’s status as a voucher recipient 
or non-recipient in a statistical model to predict whether that student 
attended private school. The researchers then use these results in a 
second statistical model, which gives the effect of using a voucher on 
test scores and survey results. This final model can be thought of as 
comparing those who attended private school with a voucher to those who 
did not but who would have attended private school had a voucher been 
offered to them. 

[E] Weighting is a procedure used to compensate for participant 
attrition. Using this procedure, some participants who returned for 
follow-up data collection represent both themselves and others who are
similar to them on characteristics measured at baseline but who did not 
return for follow-up data collection. 

[End of table] 

Table 5: Dayton, Ohio, Study Summary—1998 through 2000 (2 academic 
years): 

Study documents: 
* Martin West, Paul Peterson, and David Campbell, School Choice in 
Dayton, Ohio, After Two Years: An Evaluation of the Parents Advancing 
Choice in Education Scholarship Program (Cambridge, Mass.: Program on 
Education Policy and Governance, Harvard University, Mass.: August 
2001). 
* William Howell and Paul Peterson, School Choice in Dayton, Ohio: An 
Evaluation after One Year (Paper presented at the Conference on 
Vouchers, Charters and Public Education sponsored by the Program on 
Education Policy and Governance, Harvard University, Cambridge, Mass.: 
March 2000). 
* William Howell and Paul Peterson, The Education Gap: Vouchers and 
Urban Schools, (Washington, D.C.: The Brookings Institution, 2002). 
* William Howell, Patrick Wolf, Paul Peterson and David Campbell, Test-
Score Effects of School Vouchers in Dayton, Ohio, New York City, and 
Washington, D.C.: Evidence from Randomized Field Trials (Paper presented
at the annual meeting of the American Political Science Association, 
Washington, D.C., Sept. 2000). 

Reported results: 
* Study reports positive effects that approach, but do not reach, 
statistical significance on reading and composite scores for African 
American voucher users in the second year of the study.[A,B,F] There 
were no other positive effects on reading, math, or composite test 
scores for voucher users of any other racial or ethnic background. The 
study participants were primarily African American and Caucasian. 

* On a range of issues on which parents were surveyed, the voucher group
had the following outcomes in both years of the study when compared to
the control (no voucher) group:[C] 
- Higher parental satisfaction (percent of parents giving the school a 
report card grade of “A,” and reporting satisfaction with quality of 
teaching, and curriculum); 
- Attended smaller schools; 
- Less disorderly behavior (fighting, cheating, truancy, destruction of
property); 
- More access to music programs. 

Key strengths: 
* The study design involved vouchers being offered at random to eligible
families. This helps ensure the equivalence of the characteristics of
students using vouchers to attend private school and those of the
control group. 

* An analytical technique—the instrumental variable approach[D]—was
used to adjust for factors that threatened the equivalence of the study 
groups (e.g., some students offered vouchers did not use them to attend 
private schools and other students not offered vouchers attended 
private school without them). 

* The researchers used an analytical technique—weighting[E]—to adjust 
for observed differences between the students who returned for testing 
and surveys 1 and 2 years after the voucher program began and those
who did not remain in the study. 

Key limitations: 
* The researchers acknowledge that these results cannot be generalized 
to larger scale programs, programs in other sites, or programs serving 
other than low-income families who applied for vouchers while in grades 
1-8. 

* High levels of attrition (students not continuing to participate in 
data collection for the study) and low rates of voucher use (only 60 
percent of those offered a voucher used it for both years) may make it 
difficult to reliably estimate effects. 

* Researchers did not account for the unequal probability of selection 
of children within families (selection was based on family units but
observations were on children). 

* The analysis did not adjust for the possibility of numerous students 
attending the same school. Thus, it did not completely rule out the 
possibility that the positive effects were actually due to attendance 
at specific schools, rather than voucher use itself. 

Evidence of impact: 
* No evidence of test score effects for students of any racial/ethnic
background. 

* Strong evidence for parental survey results as reported in “results” 
column, including increased parental satisfaction by voucher parents. 

[A] We typically use the 95 percent confidence level for determining an 
effect to be statistically significant. 

[B] Composite test scores are the average of the math and reading 
scores. 

[C] For ease of discussion, we use the terms voucher users and control 
(no voucher) group. The comparison of these groups involves students 
who used vouchers to attend private school and those who were likely to 
have used vouchers had vouchers been offered to them. 

[D] The instrumental variable approach involves a statistical procedure 
in which the researchers used a student’s status as a voucher recipient 
or non-recipient in a statistical model to predict whether that student 
attended private school. The researchers then use these results in a 
second statistical model, which gives the effect of using a voucher on 
test scores and survey results. This final model can be thought of as 
comparing those who attended private school with a voucher to those who 
did not but who would have attended private school had a voucher been 
offered to them. 

[E] Weighting is a procedure used to compensate for participant 
attrition. Using this procedure, some participants who returned for 
follow-up data collection represent both themselves and others who are
similar to them on characteristics measured at baseline but who did not 
return for follow-up data collection. 

[F] Reading and composite score effects for African American students 
in Dayton in the second year of the study reached a 90 percent 
confidence level, an effect that the researchers consider statistically
significant. Although we typically use the 95 percent confidence level 
for determining an effect to be statistically significant, we recognize 
that with smaller sample sizes, as in the Dayton study, statistical 
significance is more difficult to achieve. 

[End of table] 

Table 6: Washington, D.C., Study Summary—1998 through 2001 (3 academic 
years): 

Study document: 
* Patrick Wolf, Paul Peterson, and Martin West, Results of a School 
Voucher Experiment: The Case of Washington, D.C., after Two Years 
(Paper presented at the annual meeting of the American Political 
Science Association, San Francisco, Calif., August 2001). 

* Patrick Wolf, William Howell, and Paul Peterson, School Choice in 
Washington, D.C.: An Evaluation after One Year (Paper presented at the 
Conference on Vouchers, Charters, and Public Education sponsored by the 
Program on Education Policy and Governance, Harvard University, 
Cambridge, Mass., March 2000). 

* William Howell and Paul Peterson, The Education Gap: Vouchers and 
Urban Schools, (Washington, D.C.: The Brookings Institution, 2002). 

* William Howell, Patrick Wolf, Paul Peterson and David Campbell, Test-
Score Effects of School Vouchers in Dayton, Ohio, New York City, and
Washington, D.C.: Evidence from Randomized Field Trials (Paper 
presented at the annual meeting of the American Political Science
Association, Washington, D.C., Sept. 2000). 

Reported results: 
* Year 1 of the 3-year study reports statistically significant,[A] 
positive effects on math scores but statistically significant, negative 
effects on reading scores for African American voucher users as opposed 
to the control (no voucher) group.[B] During the second year, there 
were significant, positive effects on reading, math, and composite
test scores[C] for African American voucher users. In the third and 
final year of the study, significant effects on composite scores 
disappear, and no information is currently available regarding the 
third year results for math and reading individually. 

* Study reports no differences between test scores of non-African 
American voucher and control group students. However, only a small 
percentage of the students in the study were not African American. 

* On a range of issues on which parents were surveyed, the voucher 
group had the following outcomes in both years of the study for which 
data are available[F] when compared to the control (no voucher) group: 
- Higher parental satisfaction (percent of parents giving the school a 
report card grade of “A”, and reporting satisfaction with curriculum 
and school safety); 
- Attended smaller class and smaller schools; 
- Less fighting; 
- More communication from school to parents; 
- Less access to computer labs, a cafeteria, a nurse’s office, special 
education services, and services for non-English speakers and those 
with learning problems. 

Key strengths: 
* The study design involved vouchers being offered at random to eligible
families. This helps ensure the equivalence of the characteristics of
students using vouchers to attend private school and those of the 
control group. 

* An analytical technique—the instrumental variable approach[D]—was
used to adjust for factors that threatened the equivalence of the study 
groups (e.g., some students offered vouchers did not use them to attend 
private schools and other students not offered vouchers attended 
private school without them). 

* The researchers used an analytical technique— weighting[E]—to adjust 
for observed differences between the students who returned for testing 
and surveys 1, 2, and 3 years after the voucher program began and those 
who did not remain in the study. 

Key limitations: 
* The researchers acknowledge that these results cannot be generalized 
to larger scale programs, programs in other sites, programs serving 
other than low-income families or, in the case of the test score 
improvements to groups other than African American students who applied 
for vouchers while in grades 1-8. 

* High levels of attrition (students not continuing to participate in 
data collection for the study) and low rates of voucher use (only 29 
percent of those offered a voucher used it for all 3 years) may make it 
difficult to reliably estimate effects. 

* Researchers did not account for the unequal probability of selection 
of children within families (selection was based on family units but 
observations were on children). 

* The analysis did not adjust for the possibility of numerous students 
attending the same school. Thus, it did not completely rule out the 
possibility that the positive effects were actually due to attendance 
at specific schools, rather than voucher use itself. 

Evidence of impact: 

* While there is evidence for positive impact on overall test scores 
for African American students in the second year of the study,
these positive impacts disappear in the 3rd and final year. 

* Strong evidence for parental survey results as reported in “results” 
column, including increased parental satisfaction by voucher parents. 

[A] We typically use the 95 percent confidence level for determining an 
effect to be statistically significant. 

[B] For ease of discussion, we use the terms voucher users and control 
(no voucher) group. The comparison of these groups involves students 
who used vouchers to attend private school and those who were likely to 
have used vouchers had vouchers been offered to them. 

[C] Composite test scores are the average of the math and reading 
scores. 

[D] The instrumental variable approach involves a statistical procedure 
in which the researchers used a student’s status as a voucher recipient 
or non-recipient in a statistical model to predict whether that student 
attended private school. The researchers then use these results in a 
second statistical model, which gives the effect of using a voucher on 
test scores and survey results. This final model can be thought of as 
comparing those who attended private school with a voucher to those who 
did not but who would have attended private school had a voucher been 
offered to them. 

[E] Weighting is a procedure used to compensate for participant 
attrition. Using this procedure, some participants who returned for 
follow-up data collection represent both themselves and others who are
similar to them on characteristics measured at baseline but who did not 
return for follow-up data collection. 

[F] Survey results were not available for the third year of the 
Washington, D.C. study. 

[End of table] 

[End of section] 

Appendix II: Comments from the Department of Education: 

The Secretary Of Education: 
Washington, D.C. 20202: 
"Our mission is to ensure equal access to education and to promote 
educational excellence throughout the Nation." 

August 9, 2002: 

Ms. Mamie S. Shaul: 
Director, Education, Workforce, and Income Security Issues: 
U.S. General Accounting Office: 
Washington, D.C. 20548: 

Dear Ms. Shaul: 

Thank you for the opportunity to comment on the recent GAO report, 
School Vouchers: Characteristics of Privately Funded Programs. As I am 
sure you are aware, expanding parental options and information is one 
of the four principles of President Bush's education agenda, along with 
ensuring accountability for results, strengthening local control and 
flexibility, and basing programs on sound, scientific research. As your 
report indicates, the parental choice movement has grown tremendously 
in just the last decade, thanks in large part to the same private 
scholarship programs you have examined. 

Your report also reveals the varied nature of private scholarship 
programs, which operate in some 38 States nationwide and the District 
of Columbia. Smaller programs offer just a handful of scholarships, 
while larger organizations, like the local Children's Scholarship Fund 
program in New York City, award up to 3,100 scholarships annually. 
Scholarship amounts also vary, ranging from just a few hundred dollars 
to well over $1,500 annually in other programs. Because of the success 
of these innovative programs, Arizona, Florida, and Pennsylvania have 
all passed tax credit laws, encouraging charitable contributions to 
private scholarship organizations, and Utah and other States are 
contemplating similar legislation. While these types of tax credits 
represent one approach to expanding choice for disadvantaged families, 
States are exploring a variety of different methods, including tax 
credits for low-income families to send their own children to schools 
of choice and directly funding school choice experiments, like the well-
known programs in Cleveland. Milwaukee, and Florida. 

Your report indicates that initial findings from private scholarship 
programs in Dayton, Ohio, New York City, and Washington, D.C., suggest 
that expanded parental choice can boost academic achievement for 
African-American students. However, you also caution these findings are 
limited in scope given the small size of the evaluation. In order to 
develop more information on the effects of expanded parental choice, we 
need more research and experimentation. 

That is why President Bush has requested $50 million in his FY 2003 
Department of Education budget for a Choice Demonstration Fund. This 
fund would support research projects that develop, implement, and 
evaluate innovative approaches to providing parents with expanded 
options, including both public and private school choice. President 
Bush has also proposed a federal tax credit for low-income families 
whose children are trapped in low-performing schools to help pay for 
tuition and other expenses at a local private school. It is also why I 
applaud Representative Dick Armey for his efforts to expand school 
choice for disadvantaged families here in the District of Columbia. 

Support for expanded parental options and information is found 
elsewhere in the President's budget. We have proposed $200 million to 
help fund the development of charter schools, $100 million to help 
charter schools address urgent facilities needs, and $25 million for 
voluntary public school choice programs. These initiatives are part of 
our broader efforts to improve public education. As a longtime educator 
and former superintendent, I understand that greater parental choice 
and competition are not threats to public education but pieces of their 
salvation. We should not shy away from competition and innovation but 
embrace it. 

This is an exciting time for the parental choice movement. The recent 
U.S. Supreme Court ruling upholding the Cleveland school choice plan 
validates that such programs are Constitutional and will hopefully 
encourage States to experiment more widely with this promising reform. 
But we would have never gotten to this point had it not been for the 
pioneering work of the private scholarship programs you identify in 
your report. I applaud groups such as the Children's Scholarship Fund, 
Children First America, and the many local organizations that have made 
it their mission to ensure that all children arc educated and no child 
is left behind. t trust these groups, and Federal, State, and local 
governments, will do all they can to expand choices and options to 
thousands more children in the years to come. 

Sincerely, 

Signed by: 

Rod Paige: 

[End of section] 

Appendix III: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

David D. Bellis (415) 904-2272 (bellisd@gao.gov): 
Susan J. Lawless (206) 287-4792 (lawlesss@gao.gov): 

Staff Acknowledgments: 

In addition to the individuals named above, Andrew S. Bauck, Dianne L.
Whitman Miner, Shana B. Wallace, Michele C. Fejfar, Douglas M. Sloane,
Gail S. MacColl, Jonathan H. Barker, Barry J. Seltser, Patrick J. 
Dibattista, and Stanley G. Stenersen made key contributions to this 
report. 

[End of section] 

Bibliography: 

Reports on the New York Study: 

Peterson, Paul E., David Myers, Josh Haimson, and William G. Howell.
Initial Findings from the Evaluation of the New York School Choice
Scholarships Program. Washington, D.C.: Mathematica Policy Research,
November 1997. 

Peterson, Paul E., David Myers, and William G. Howell. An Evaluation of
the New York City School Choice Scholarships Program: The First Year.
Washington, D.C.: Mathematica Policy Research, October 1998. 

Myers, David, Paul Peterson, Daniel Mayer, Julia Chou, and William G.
Howell. School Choice in New York City after Two Years: An Evaluation
of the School Choice Scholarships Program. Washington, D.C.:
Mathematica Policy Research, August 2000. 

Mayer, Daniel P., Paul E. Peterson, David E. Myers, Christina Clark 
Tuttle, and William G. Howell. School Choice in New York City after 
Three Years: An Evaluation of the School Choice Scholarships Program. 
Washington, D.C.: Mathematica Policy Research, February 2002. 

Reports on the Dayton Study: 

Peterson, Paul E., Jay P. Greene, William G. Howell and William 
McCready. Initial Findings from an Evaluation of School Choice Programs 
in Washington, D.C. and Dayton, Ohio. Paper presented at the annual
meeting of the Association of Public Policy and Management, New York,
N.Y.: October 1998. 

Howell, William G. and Paul E. Peterson. School Choice in Dayton, Ohio:
An Evaluation after One Year. Paper presented at the Conference on
Vouchers, Charters and Public Education sponsored by the Program on
Education Policy and Governance, Harvard University, Cambridge, Mass.:
March 2000. 

West, Martin R., Paul E. Peterson and David E. Campbell. School Choice 
in Dayton, Ohio After Two Years: An Evaluation of the Parents Advancing
Choice in Education Scholarship Program. Cambridge, Mass.: Program on
Education Policy and Governance, Harvard University, August 2001. 

Reports on the Washington, D.C., Study: 

Peterson, Paul E., Jay P. Greene, William G. Howell and William 
McCready. Initial Findings from an Evaluation of School Choice Programs 
in Washington, D.C., and Dayton, Ohio. Paper presented at the annual
meeting of the Association of Public Policy and Management, New York,
N.Y.: October 1998. 

Wolf, Patrick J., William G. Howell, and Paul E. Peterson. School 
Choice in Washington, D.C.: An Evaluation after One Year. Paper 
presented at the Conference on Vouchers, Charters and Public Education 
sponsored by the Program on Education Policy and Governance, Harvard 
University, Cambridge, Mass.: March 2000. 

Wolf, Patrick J., Paul E. Peterson and Martin R. West. Results of a 
School Voucher Experiment: The Case of Washington, D.C., after Two 
Years. Paper presented at the annual meeting of the American Political 
Science Association, San Francisco, Calif.: August 2001. 

Reports on the Studies in All Three Cities: 

Howell, William G., Patrick J. Wolf, Paul E. Peterson, and David E.
Campbell. Test-Score Effects of School Vouchers in Dayton, Ohio; New
York City; and Washington, D.C.: Evidence from Randomized Field Trials. 
Paper presented at the annual meeting of the American Political Science 
Association, Washington, D.C.: September 2000. 

Howell, William G. and Paul E Peterson, with Patrick J. Wolf and David 
E. Campbell. The Education Gap: Vouchers and Urban Schools. Washington,
D.C.: The Brookings Institution, 2002. 

Howell, William G., Patrick J. Wolf, David E. Campbell, and Paul E.
Peterson. School Vouchers and Academic Performance: Results from Three 
Randomized Field Trials. Journal of Policy Analysis and Management, 
vol. 21, no. 2 (2002): 191-217. 

[End of section] 

Footnotes: 

[1] U.S. General Accounting Office, School Vouchers: Publicly Funded 
Programs in Cleveland and Milwaukee, GAO-01-914 (Washington, D.C.: Aug. 
31, 2001). 

[2] Children First America was originally founded in 1992 as Children’s 
Education Opportunity Foundation America (CEO America). 

[3] See GAO-01-914, August 31, 2001. 

[4] Under section 501 of the Internal Revenue Service Code, nonprofit 
organizations are exempted from paying federal income taxes. In 
addition, donations made to these organizations are tax deductible. 

[5] Arizona provides tax credits to individuals for contributions to 
organizations that provide scholarships to students to help meet the 
cost of private school attendance. Florida offers tax credits to 
corporations that fund organizations providing scholarships to 
nongovernmental schools. Pennsylvania grants businesses tax credits for 
contributions to organizations that award scholarships allowing 
children to attend the school of their choice. 

[6] Overall, about 78 percent of private schools nationally are 
religiously affiliated, according to the Department of Education. 

[7] When participants are assigned to the group that receives services 
and the comparison groups randomly, as in these studies, the comparison 
group is called a control group. 

[8] According to the researchers, approximately equal percentages of 
voucher recipients and nonrecipients returned for follow-up testing and 
surveys. 

[9] The method employed was an instrumental variable technique. For 
additional information on this and other aspects of the studies’ 
methodology, see William G. Howell and Paul E. Peterson, The Education 
Gap: Vouchers and Urban Schools (Washington, D.C.: Brookings 
Institution Press, 2002), pp. 49-52. 

[10] Weighting is a procedure used to compensate for participant 
attrition. Using this procedure, some participants who returned for 
follow-up data collection represent both themselves and others who are 
similar to them on characteristics measured at baseline but who did not 
return for follow-up data collection. 

[11] Professor Alan Krueger and Pei Zhu of Princeton University have 
analyzed the data pertaining to the New York experiment. They explore 
some of the assumptions underlying the original research, such as the 
exclusion of students with missing baseline scores (who increase the 
original sample by over 40 percent) and the definition of race. Their 
findings raise doubts about the size and significance of earlier 
findings of a positive effect of vouchers on test scores for African 
American students. (Another Look at the New York City School Voucher 
Experiment, Alan Krueger and Pei Zhu, August 16, 2002, paper presented 
at the Conference on Randomized Experimentation in the Social Sciences 
at Yale University). 

[12] According to results from the final year of the New York study, on 
average, African American students who used a voucher to attend private 
school in at least one year of the study ranked 9.7 percentile points 
higher in math and 5.4 percentage points higher in reading than the 
control group. (Average math percentiles were 25.5 for voucher users and
15.8 for the control group. Average reading percentiles were 25.3 for 
voucher users and 19.9 for the control group.) 

[13] This result reached the 90 percent confidence level, an effect 
that the researchers consider statistically significant. We typically 
use the 95 percent confidence level for determining an effect to be 
statistically significant. However, we recognize that with smaller 
sample sizes as in the Dayton study statistical significance at the 95 
percent level is more difficult to achieve. 

[14] The studies did not provide estimates of the separate math and 
reading scores for the third year of the Washington study. 

[15] In New York, the rate of students returning for follow-up testing 
in subsequent years of the study ranged from 82 percent to 66 percent. 
In Dayton, this rate ranged from 56 percent to 49 percent, and in 
Washington, it ranged from 63 percent to 50 percent. According to the
researchers, approximately equal percentages of voucher recipients and 
nonrecipients returned for follow-up testing. The researchers used 
incentives to maximize the number of nonrecipient students returning. 

[16] The research team indicates that the positive effect on African 
American students of using vouchers could diminish the achievement gap 
between them and white students by roughly one-third. However, Alan 
Krueger, of Princeton University, disputes the estimate offered by
the research team and the standard deviation used in deriving it. He 
estimates that, when the appropriate standard deviation is applied, the 
impact of attending private school using a voucher would diminish the 
achievement gap by one-fifth to one-quarter. Furthermore, he argues 
that from a policy perspective, it is more relevant to analyze the 
impact of a voucher offer, rather than looking only at those who chose 
to use it. Krueger estimates that the impact of offering vouchers would 
be to diminish the achievement gap by no more than one-eighth. 
(Education Next, winter 2001, pp. 4-5). 

[17] Howell and Peterson, The Education Gap, 166-167. 

[18] Researchers used analytical techniques to address attrition 
(students no longer participating in the evaluation) and address that 
some students offered vouchers did not use them and others not offered 
vouchers attended private school without them. For additional 
information on these techniques, see Howell and Peterson, The Education 
Gap, pp. 43-47; 49-52. 

[19] In the second year of the Washington study, the study reported 
that the parents of voucher users were more likely to give a grade of 
“A or B”, not “A.” Although the Washington study lasted for 3 years, no 
third year survey results had been released at the time this report was
written. 

[20] In the first year follow-up surveys for the Dayton and Washington 
studies, researchers asked parents about their children’s academic 
program rather than academic quality and found that the parents of 
voucher users were more likely than the control group parents to be 
very satisfied. During second year follow-up in Washington, the parents 
of voucher users were no more likely to report being very satisfied 
with academic quality than those in the control group. 

[21] As indicated in footnote 16, although the Washington study lasted 
for 3 years, no third year survey results had been released at the time 
this report was written. 

[22] In Dayton, parents reported that public schools were no more 
likely to have a library or cafeteria or to offer special education 
programs than private schools, but participants were not asked about 
programs for non-English speakers. 

[23] No one’s voucher was actually terminated because of failure to 
participate in follow-up. 

[24] For additional detail on the weighting procedure used in the 
studies, see Howell and Peterson, The Education Gap, pp. 209-216. 

[25] For further details on the instrumental variable technique and 
other aspects of the studies’ design and data analysis methods, see 
Howell and Peterson, The Education Gap, pp. 43-47; 49-52. 

[End of section] 

GAO’s Mission: 

The General Accounting Office, the investigative arm of Congress, 
exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO’s commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO’s Web site [hyperlink, 
http://www.gao.gov] contains abstracts and full-text files of current 
reports and testimony and an expanding archive of older products. The 
Web site features a search engine to help you locate documents using 
key words and phrases. You can print these documents in their entirety, 
including charts and other graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as “Today’s Reports,” on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
[hyperlink, http://www.gao.gov] and select “Subscribe to daily E-mail 
alert for newly released products” under the GAO Reports heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. General Accounting Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548: 

To order by Phone: 	 

Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov]: 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, NelliganJ@gao.gov: 
(202) 512-4800: 
U.S. General Accounting Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: