Technical Assessment of Zhao and Thurman's 2001 Evaluation of the Effects of COPS Grants on Crime
GAO-03-867R: Published: Jun 12, 2003. Publicly Released: Jul 14, 2003.
Community Oriented Policing Services (COPS) is a federal public safety program whose goals are to add officer positions to the streets of communities nationwide and to promote community policing. Since the program's inception in 1994, local law enforcement agencies have received billions of dollars in grants to hire additional officers, acquire technology and civilian personnel, and implement innovative crime-prevention programs. To receive COPS grants, agencies are expected to implement or enhance community policing strategies illustrating community partnerships, problem solving, and organizational commitment. Given the large expenditures of funds, it is important for policy makers, among others, to have sound information on the effectiveness of the COPS program in reducing crime. Congress asked us to review one evaluation of the effectiveness of the COPS program--by Zhao and Thurman--and to render an assessment of its quality. In this report, we provide information on the extent to which this particular study's conclusions are supported by the data the researchers used and the analyses they conducted. GAO statisticians and methodology specialists reviewed the study using standard and widely accepted statistical and social science research principles.
Our review of the 2001 study on the effects of COPS grants on crime rates indicated that the results of their study should be viewed as inconclusive. We believe that the study's limitations in data and methods are significant and proclude meaningful interpretation of the results. We cannot agree with Zhao et al. that their 2001 study shows that some COPS grants (hiring and innovative) significantly reduced crime because, among other things, important variables were omitted from their analyses, the analytic models were misspecified, and the sample of cities included in the study was limited. Further, we have concerns about the use of outdated census data for control variables. Aside from concerns about data and methods, we question whether the statistically significant crime reductions that Zhao et al. found are significant in a practical sense. While we cannot agree with the Zhao et al.'s conclusions, we also cannot say that COPS grants are ineffective in reducing crime. A program's effects and researchers' ability to design studies that will acurately measure those effects are two different things. Other studies, which we have not reviewed, may have taken a more rigorous approach to assessing the effects of COPS grants on crime. We believe that a more rigorous study would incorporate, among other things, more reliable, valid, and complete measures; a more complete and generalizable sample of cities; and well specified analytic models.