Program Evaluation: OMB's PART Reviews Increased Agencies' Attention to Improving Evidence of Program Results
GAO-06-67
Published: Oct 28, 2005. Publicly Released: Oct 28, 2005.
Skip to Highlights
Highlights
The Office of Management and Budget (OMB) designed the Program Assessment Rating Tool (PART) as a diagnostic tool to draw on program performance and evaluation information for forming conclusions about program benefits and recommending adjustments to improve results. To assess progress in improving the evidence base for PART assessments, GAO was requested to examine (1) agencies' progress in responding to OMB's recommendations to evaluate programs, (2) factors facilitating or impeding agencies' progress, and (3) whether agencies' evaluations appear to be designed to yield the information on program results that OMB expects.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Management and Budget | OMB should encourage agencies to discuss their plans for program evaluations--especially those in response to an OMB recommendation--with OMB and with congressional and other program stakeholders to ensure that their findings will be timely, relevant, and credible and that they will be used to inform policy and management decisions. |
Closed – Implemented
OMB reports that agencies are now required to brief their appropriators and authorizers on past, completed, and planned PART assessments, including planned program evaluations. This represents partial implementation of our recommendation because it does not clearly distinguish OMB PART assessments from program evaluation studies, nor recognize the need to discuss evaluation plans and criteria with congressional stakeholders to ensure that their findings will be timely, relevant, credible, and used to inform decisions.
|
| Office of Management and Budget | OMB should engage in dialogue with agencies and congressional stakeholders on a risk-based allocation of scarce evaluation resources among programs, based on size, importance, or uncertain effectiveness, and on the timing of such evaluations. |
Closed – Not Implemented
OMB reports that it has taken no formal actions in this area but that discussion of agency evaluation resources largely occur as part of agency budget discussions. OMB reports continuing discussions with agencies and outside stakeholders on the need to invest in rigorous evaluations, has offered agencies (in 2009 and 2010) additional funds for rigorous evaluations, but does not plan to address how agencies should allocate evaluation resources among its programs.
|
| Office of Management and Budget | OMB should continue to improve its PART guidance and training of examiners on evaluation to acknowledge a wide range of appropriate methods. |
Closed – Implemented
OMB sponsored workshops for OMB and agency staff that clarified that experimental research designs involving randomization were not the only acceptable method for evaluating federal program effectiveness and might not be feasible in some cases. OMB also sponsored a workshop where agency evaluators discussed the applicability of alternative evaluation designs.
|
Full Report
GAO Contacts
Office of Public Affairs
Topics
Agency evaluationEvaluation criteriaEvaluation methodsFederal agenciesPerformance appraisalPerformance measuresProgram evaluationProgram managementRegulatory agenciesStrategic planning