Juvenile Justice:
OJJDP Reporting Requirements for Discretionary and Formula Grantees and Concerns About Evaluation Studies
GAO-02-23: Published: Oct 30, 2001. Publicly Released: Nov 29, 2001.
Additional Materials:
- Full Report:
Contact:
(202) 512-2758
contact@gao.gov
Office of Public Affairs
(202) 512-4800
youngc1@gao.gov
Although national rates of violent juvenile crime and youth victimization have declined during the past five years, critical problems affecting juveniles, such as drug dependency, the spread of gangs, and child abuse and neglect, persist. The Office of Juvenile Justice and Delinquency Prevention (OJJDP) has funded various demonstration, replication, research and evaluation, and training and technical assistance programs to prevent and respond to juvenile delinquency and juvenile victimization. GAO's review of 16 of OJJDP's major programs found that, although virtually all grantees must report on their progress twice a year, the information they reported varied. Grantees receive standard, general guidance for reporting on their projects and providing OJJDP information to monitor grantee's projects and accomplishments. According to OJJDP officials, such guidance needs to be general because of differences among individual projects and local needs and circumstances. GAO identified eight programs in which all grantees reported the number of juveniles they directly served. OJJDP does not require grantees in all its programs to report directly on the number of juveniles served directly because many of its programs are not intended to serve juveniles directly. GAO's in-depth review of OJJDP's 10 impact evaluations undertaken since 1995 raises concerns about whether the evaluations will produce definitive results. In some of these evaluations, variations in how the programs are implemented across sites make it difficult to interpret evaluation results.
Recommendation for Executive Action
Status: Closed - Implemented
Comments: Regarding the five impact evaluations in the formative stages to which the GAO recommendation was directed, OJJDP either took action to meet the recommendation's intent, or circumstances changed that make the recommendation no longer valid. (1) At the time GAO made the recommendation, the specifics of the Parents Anonymous outcome evaluation design had not yet been determined. Subsequently, the evaluation was designed to have a longitudinal design with three data collection points (pre-intervention, post-intervention, and follow up) and to use an internal comparison group based on implementation of the Parents Anonymous model. The Parents Anonymous process evaluation (completed prior to the outcome evaluation) showed that it would be better to devise an internal comparison group than to find a valid external comparison group. (2) In response to GAO's recommendation, OJJDP convened a panel of independent peer reviewers to conduct an interim peer review of the Safe Start evaluation design and methodology. The reviewers unanimously agreed that the evaluation was significantly flawed and would not yield the types of outcomes that could effectively demonstrate impact. Accordingly, they discontinued the impact evaluation portion of the evaluation. (3) OJJDP discontinued the Rural Gangs evaluation based on concerns regarding the quality of the evaluation design and budget limitations. (4) OJJDP also discontinued the Violence Prevention (PATHE) evaluation. (5) GAO reported that the design of the Safe Schools/Healthy Students evaluation was basically sound, and OJDDP informed us that no changes were anticipated to that evaluation design.
Recommendation: GAO's review of the recent Office of Juvenile Justice and Delinquency Prevention (OJJDP) program evaluations has shown that, of the five that are in or near their final stages, some problems with valid comparison groups and data collection could compromise the usefulness of some of their results. Five other program evaluations are in a formative stage where comparison group issues and data collection strategies are not yet finalized. The Attorney General should direct the Administrator of OJJDP to assess the five impact evaluations in the formative stages to address potential comparison group and data collection problems and, on the basis of that assessment, initiate any needed interventions to help ensure that the evaluations produce definitive results.
Agency Affected: Department of Justice
Explore the full database of GAO's Open Recommendations
»
Oct 26, 2020
-
Intellectual Property:
CBP Has Taken Steps to Combat Counterfeit Goods in Small Packages but Could Streamline EnforcementGAO-20-692: Published: Sep 24, 2020. Publicly Released: Oct 26, 2020.
Sep 30, 2020
-
Federal Criminal Restitution:
Department of Justice Has Ongoing Efforts to Improve Its Oversight of the Collection of Restitution and Tracking the Use of Forfeited AssetsGAO-20-676R: Published: Sep 30, 2020. Publicly Released: Sep 30, 2020.
Sep 10, 2020
-
Federal Tactical Teams:
Characteristics, Training, Deployments, and InventoryGAO-20-710: Published: Sep 10, 2020. Publicly Released: Sep 10, 2020.
Sep 8, 2020
-
VA Police:
Actions Needed to Improve Data Completeness and Accuracy on Use of Force Incidents at Medical CentersGAO-20-599: Published: Sep 8, 2020. Publicly Released: Sep 8, 2020.
Aug 13, 2020
-
Anti-Money Laundering:
FinCEN Should Enhance Procedures for Implementing and Evaluating Geographic Targeting OrdersGAO-20-546: Published: Jul 14, 2020. Publicly Released: Aug 13, 2020.
Jul 29, 2020
-
Federal Prison Industries:
Actions Needed to Evaluate Program EffectivenessGAO-20-505: Published: Jul 29, 2020. Publicly Released: Jul 29, 2020.
Jul 8, 2020
-
Gun Control:
DOJ Can Further Improve Guidance on Federal Firearm Background Check RecordsGAO-20-528: Published: Jul 8, 2020. Publicly Released: Jul 8, 2020.
Jun 22, 2020
-
Federal Prisons:
Additional Analysis Needed to Determine Whether to Issue Pepper Spray to Minimum Security PrisonsGAO-20-342: Published: Jun 22, 2020. Publicly Released: Jun 22, 2020.
May 26, 2020
-
Bureau of Prisons:
Improved Planning Would Help BOP Evaluate and Manage Its Portfolio of Drug Education and Treatment ProgramsGAO-20-423: Published: May 26, 2020. Publicly Released: May 26, 2020.
May 12, 2020
-
Forensic Technology:
Algorithms Used in Federal Law EnforcementGAO-20-479SP: Published: May 12, 2020. Publicly Released: May 12, 2020.
Looking for more? Browse all our products here