DOJ Is Enhancing Information on Effective Programs, but Could Better Assess the Utility of This Information
GAO-10-125: Published: Dec 17, 2009. Publicly Released: Dec 17, 2009.
State juvenile justice systems face critical problems when it comes to juvenile delinquency issues such as reentry--when offenders return home from incarceration--and substance abuse. GAO was asked to review juvenile reentry and substance abuse program research and efforts by the Department of Justice's (DOJ) Office of Juvenile Justice and Delinquency Prevention (OJJDP) to provide information on effective programs (i.e., whether a program achieves its intended goal) and cost-beneficial programs (i.e., whether the benefits of programs exceeded their costs). This report addresses (1) expert opinion and available research on these types of reentry and substance abuse programs, (2) the extent to which OJJDP assesses its efforts to disseminate information on effective programs, and (3) OJJDP's plans to accomplish its research and evaluation goals. GAO, among other things, reviewed academic literature, and OJJDP's dissemination efforts and research goals. GAO also interviewed OJJDP officials and a nonprobability sample of 26 juvenile justice experts selected based on their experience with juvenile reentry and substance abuse issues.
The majority of the juvenile justice reentry and substance abuse experts GAO interviewed cited evidence that shows cognitive behavioral therapy--programs that help individuals change their beliefs in order to change their behavior--and family therapy--programs that treat juveniles by focusing on improving communication with family members--are effective and cost beneficial when addressing reentry and substance abuse issues. For example, two juvenile reentry experts cited studies showing that 1 year after participating in a cognitive behavioral therapy program, participants were less likely to commit another offense than nonparticipants. Additionally, experts cited a study that reported that a family therapy program provides about $80,000 in savings per participant when accounting for savings from a decline in crime, such as the cost the police would have incurred. Most experts indicated that there was limited evidence on the effectiveness and cost benefits of reentry programs, such as aftercare--programs that assist juvenile offenders in returning to their communities during the reentry process--and substance abuse programs, such as drug courts--specialized courts that provide programs for substance-abusing juveniles and their families. GAO reviewed two OJJDP efforts that provide information on effective programs across the range of juvenile justice issues, the National Training and Technical Assistance Center (NTTAC) and the Model Programs Guide. OJJDP has mechanisms in place to regularly assess the utility of the information provided by NTTAC, but does not have such a mechanism for the guide. OJJDP ensures the utility of NTTAC's information through evaluations in accordance with federal guidelines that highlight the importance of regularly soliciting feedback from users. However, OJJDP could better ensure the utility of the information disseminated by the Model Programs Guide by having a mechanism in place to solicit regular feedback from members of the juvenile justice field--for example, program practitioners--that is specifically related to the guide. OJJDP has articulated research and evaluation goals to support its mission of improving the juvenile justice system and is developing plans to assist in meeting these goals. OJJDP is required under the Juvenile Justice and Delinquency Prevention Act, as amended, to publish an annual program plan that describes planned activities under accounts authorized for research and evaluation activities, among other things. Additionally, the Office of Management and Budget (OMB) recommended that OJJDP develop a comprehensive evaluation plan for juvenile justice programs. While OJJDP has not published an annual program plan since 2002, in December of 2009, it issued a proposed plan for public comment and aims to publish the final program plan once public comments are incorporated. Additionally, although the office has considered developing a comprehensive evaluation plan to address OMB recommendations, it had not previously done so because of a lack of resources. However, OJJDP is committed to developing a comprehensive evaluation plan once the program plan is finalized.
Recommendation for Executive Action
Status: Closed - Implemented
Comments: In fiscal year 2009, we reviewed and reported on the Office of Juvenile Justice and Delinquency Prevention's (OJJDP's) efforts to disseminate information about effective programs and assess the utility of the information it is providing through these efforts. We reported, among other things, that OJJDP has mechanisms in place to ensure that training and technical assistance meet users' needs, but that OJJDP could better ensure the usefulness of the information it disseminates through the Model Programs Guide (MPG) by having a mechanism in place to solicit regular feedback related to the MPG from the juvenile justice field. According to OJJDP officials and OJJDP's website, OJJDP developed text in the MPG which invites comments from MPG users. OJJDP officials provided documentation to show that they respond and, as appropriate, incorporate the feedback that is collected. This feedback mechanism is consistent with our recommendation and, as a result, this recommendation is closed as implemented.
Recommendation: To help ensure that OJJDP's Model Programs Guide is regularly meeting user needs and providing the most helpful information on effective programs, consistent with federal guidelines, the Administrator of OJJDP should develop a cost-effective mechanism for regularly soliciting and incorporating feedback from the juvenile justice field on the usefulness of the information provided in its Model Programs Guide.
Agency Affected: Department of Justice: Office of Juvenile Justice and Delinquency Prevention