Juvenile Justice:

OJJDP Reporting Requirements for Discretionary and Formula Grantees and Concerns About Evaluation Studies

GAO-02-23: Published: Oct 30, 2001. Publicly Released: Nov 29, 2001.

Additional Materials:

Contact:

Laurie E. Ekstrand
(202) 512-2758
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Although national rates of violent juvenile crime and youth victimization have declined during the past five years, critical problems affecting juveniles, such as drug dependency, the spread of gangs, and child abuse and neglect, persist. The Office of Juvenile Justice and Delinquency Prevention (OJJDP) has funded various demonstration, replication, research and evaluation, and training and technical assistance programs to prevent and respond to juvenile delinquency and juvenile victimization. GAO's review of 16 of OJJDP's major programs found that, although virtually all grantees must report on their progress twice a year, the information they reported varied. Grantees receive standard, general guidance for reporting on their projects and providing OJJDP information to monitor grantee's projects and accomplishments. According to OJJDP officials, such guidance needs to be general because of differences among individual projects and local needs and circumstances. GAO identified eight programs in which all grantees reported the number of juveniles they directly served. OJJDP does not require grantees in all its programs to report directly on the number of juveniles served directly because many of its programs are not intended to serve juveniles directly. GAO's in-depth review of OJJDP's 10 impact evaluations undertaken since 1995 raises concerns about whether the evaluations will produce definitive results. In some of these evaluations, variations in how the programs are implemented across sites make it difficult to interpret evaluation results.

Recommendation for Executive Action

  1. Status: Closed - Implemented

    Comments: Regarding the five impact evaluations in the formative stages to which the GAO recommendation was directed, OJJDP either took action to meet the recommendation's intent, or circumstances changed that make the recommendation no longer valid. (1) At the time GAO made the recommendation, the specifics of the Parents Anonymous outcome evaluation design had not yet been determined. Subsequently, the evaluation was designed to have a longitudinal design with three data collection points (pre-intervention, post-intervention, and follow up) and to use an internal comparison group based on implementation of the Parents Anonymous model. The Parents Anonymous process evaluation (completed prior to the outcome evaluation) showed that it would be better to devise an internal comparison group than to find a valid external comparison group. (2) In response to GAO's recommendation, OJJDP convened a panel of independent peer reviewers to conduct an interim peer review of the Safe Start evaluation design and methodology. The reviewers unanimously agreed that the evaluation was significantly flawed and would not yield the types of outcomes that could effectively demonstrate impact. Accordingly, they discontinued the impact evaluation portion of the evaluation. (3) OJJDP discontinued the Rural Gangs evaluation based on concerns regarding the quality of the evaluation design and budget limitations. (4) OJJDP also discontinued the Violence Prevention (PATHE) evaluation. (5) GAO reported that the design of the Safe Schools/Healthy Students evaluation was basically sound, and OJDDP informed us that no changes were anticipated to that evaluation design.

    Recommendation: GAO's review of the recent Office of Juvenile Justice and Delinquency Prevention (OJJDP) program evaluations has shown that, of the five that are in or near their final stages, some problems with valid comparison groups and data collection could compromise the usefulness of some of their results. Five other program evaluations are in a formative stage where comparison group issues and data collection strategies are not yet finalized. The Attorney General should direct the Administrator of OJJDP to assess the five impact evaluations in the formative stages to address potential comparison group and data collection problems and, on the basis of that assessment, initiate any needed interventions to help ensure that the evaluations produce definitive results.

    Agency Affected: Department of Justice

 

Explore the full database of GAO's Open Recommendations »

Oct 26, 2020

Sep 30, 2020

Sep 10, 2020

Sep 8, 2020

Aug 13, 2020

Jul 29, 2020

Jul 8, 2020

Jun 22, 2020

May 26, 2020

May 12, 2020

Looking for more? Browse all our products here