Juvenile Justice:

A Time Frame for Enhancing Grant Monitoring Documentation and Verification of Data Quality Would Help Improve Accountability and Resource Allocation Decisions

GAO-09-850R: Published: Sep 22, 2009. Publicly Released: Sep 22, 2009.

Additional Materials:

Contact:

Eileen R. Larence
(202) 512-6510
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

From fiscal years 2006 through 2008, the Office of Juvenile Justice and Delinquency Prevention (OJJDP) within the Department of Justice (DOJ) awarded $1.2 billion in funds through approximately 2,000 grants in support of its mission to help states and communities prevent and reduce juvenile delinquency and victimization and improve their juvenile justice systems. OJJDP awards grants to states, territories, localities, and organizations to address a variety of issues, such as reducing juvenile substance abuse, combating Internet crimes against children, preventing youth gang involvement, and providing youth mentoring services. The scope and administration of OJJDP grants also vary, ranging from private organization recipients that implement programs directly in a single community to states that administer grants by awarding the funds they receive to subgrantees to implement programs locally and statewide. Assessing the performance of these programs through grant monitoring is a key management tool to hold grantees accountable for implementing programs as agreed to in their awards, to verify they are making progress toward the objectives of their programs, and to ensure that grant funds are used in support of OJJDP's mission. DOJ's Office of Justice Programs (OJP) establishes grant monitoring policies for its components, including OJJDP. In 2008, the DOJ Office of the Inspector General identified grant management, including maintaining proper oversight of grantees to ensure grant funds are used as intended, as a critical issue and among the department's top management challenges. In the past, we have identified concerns specific to OJJDP's grant monitoring activities. In October 2001, we reported that OJJDP was not consistently documenting its grant monitoring activities, such as required phone contacts between grant managers and grantees, and as a result could not determine the level of monitoring being performed by grant managers. We recommended that OJJDP take steps to determine why it was not consistently documenting its grant monitoring activities and develop and enforce clear expectations regarding monitoring requirements. Since that time, partially in response to our recommendation, OJJDP has taken steps to address this recommendation. For example, OJJDP conducted an assessment of additional policies and procedures that were needed for grant monitoring, and developed a manual that outlined steps for completing specific monitoring activities, such as review of grantee documentation. To help Congress ensure effective use of funds for juvenile justice grant programs, you asked us to assess OJJDP's efforts to monitor the implementation of its grant programs. This report addresses the following questions: (1) What processes does OJJDP have in place to monitor the performance of its juvenile justice grants, and to what extent does it record results of its monitoring efforts to ensure transparency and accountability? (2) How, if at all, does OJJDP use performance measurement data to make programming and funding decisions, and to what extent does it verify the quality of these data?

In accordance with OJP requirements, OJJDP has processes in place to monitor the performance of its juvenile justice grants including desk reviews, site visits, and postsite visit follow-up; however, the office does not have a process to record all follow-up steps taken to resolve grantee issues identified during site visits. Grant managers are to conduct grantee monitoring through three phases: (1) desk review, used to review grantee documentation to understand a grantee's level of accomplishment relative to its stated goals; (2) site visit, used to verify that grantees are implementing programs consistent with proposed plans; and (3) postsite visit, used to resolve issues identified during the visit. We found, during our review of OJJDP monitoring documentation, that desk review and site visit activities are to be recorded in OJP's automated repository, the Grant Management System, in accordance with OJP requirements. In addition, OJJDP officials said that OJJDP requires grant managers to record postsite visit actions taken through OJP's Correction Action Plan process, which OJJDP reserves for egregious circumstances, such as a failure to meet basic programming requirements.10 However, OJJDP does not require that issues resolved informally, such as by e-mail, during the postsite visit phase be recorded in the system. While OJJDP has developed performance measures for its grant programs and collects performance measurement data from its grantees, the office is making limited use of these data because it is not verifying these data to ensure their quality, which is inconsistent with leading management practices in performance measurement. As we have reported in the past, data verification--assessment of data completeness, accuracy, consistency, timeliness, and related quality control practices--helps to ensure that users can have confidence in the reported performance information. According to OJJDP officials, they have not taken action to verify performance data because since 2002 they have focused on the collection of such data rather than on its utilization. Specifically, since 2002, OJJDP has developed performance measures for each of its grant programs and implemented requirements for all grantees to report on measures at least once a year. Although these officials said that OJJDP has processes in place to assess whether the data are appropriate for the performance measure, they stated that OJJDP does not have data verification processes in place and is dependent on grantees to report complete, accurate, consistent, and timely data. These officials also stated that because OJJDP does not know the quality of the data submitted, they do not use performance data to make resource allocation decisions.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To help ensure the resolution of grantee issues, as well as ensure accountability for grantees fulfilling the conditions of their grants, the Administrator of OJJDP should establish a time frame for fully implementing the requirement established by OJP for grant managers to record in the Grant Management System all actions taken to resolve grantee issues consistent with OJP requirements, standards for internal control, and standard practices for program management.

    Agency Affected: Department of Justice: Office of Juvenile Justice and Delinquency Prevention

    Status: Closed - Implemented

    Comments: In fiscal year 2009, we reviewed and reported on the efforts of the Office of Juvenile Justice and Delinquency Prevention (OJJDP) to monitor the implementation of its grant programs. We reported, among other things, that OJJDP has processes in place to monitor the performance of its grants and to record the results of two of its three monitoring phases in the Grant Management System (GMS). However, OJJDP had not yet required grant managers to record steps taken to resolve all issues identified during site visits in accordance with Office of Justice Programs requirements. According to OJJDP officials, as of September 2009, all grant managers were asked to record all contact in the GMS beginning January 2010 including informal follow-up done via phone or e-mail that is substantive in nature. Such revision to the grant monitoring protocol is consistent with our recommendation.

    Recommendation: To help ensure the quality of performance measurement data submitted by grantees and improve these data to support agency and congressional decision making, the Administrator of OJJDP should (1) finalize a data verification approach that includes how it will execute the approach to assess data completeness, accuracy, consistency, and timeliness; and establish time frames for implementing the approach consistent with leading management practices for performance measurement and standard practices for program management; and (2) note in each document containing performance measurement data that the limitations of the data are currently unknown as the data have not been verified to help ensure that the information in the communication products OJJDP disseminates is transparent, clear, and understood.

    Agency Affected: Department of Justice: Office of Juvenile Justice and Delinquency Prevention

    Status: Closed - Implemented

    Comments: In fiscal year 2009, we reviewed and reported on the Office of Juvenile Justice and Delinquency Prevention's (OJJDP) efforts to monitor the implementation of its grant programs. We reported, among other things, that OJJDP has taken action to collect performance measurement data from grantees but has not yet performed verification efforts to ensure the quality of their data. We recommend that the Administrator of OJJDP finalize a data verification approach that includes how it will execute the approach to assess data completeness, accuracy, consistency, and timeliness; and establish time frames for implementing the approach consistent with leading management practices for performance measurement and standard practices for program management. OJJDP reported taking action to develop and implement a data verification plan by July 2011. In response to our recommendation, OJJDP finalized a Data Verification Plan and an online Performance Measures Verification/ Validation tool for OJJDP program managers to use to review the accuracy and quality of program data collected by grantees. OJJDP began initial implementation of the plan using a random sample of grantees from OJJDP's Title II B Formula Grants programs in the summer of 2012, which will be completed in December 2012. Following completion, the plan will be implemented across OJJDP. The plan allows OJJDP program managers to determine data completeness, accuracy, consistency and timeliness by assessing grantee data again seven criteria intended to verify the quality of the data. As a result of this verification and validation plan and tool, OJJDP should be better positioned to use grantee performance measurement data to inform resource allocation decisions.

    Jul 9, 2014

    May 14, 2014

    Apr 30, 2014

    Mar 26, 2014

    Jan 13, 2014

    Dec 9, 2013

    Dec 6, 2013

    Nov 20, 2013

    Oct 29, 2013

    Sep 25, 2013

    Looking for more? Browse all our products here