Public Financial Management:

Improvements Needed in USAID's and Treasury's Monitoring and Evaluation Efforts

GAO-12-920: Published: Sep 13, 2012. Publicly Released: Sep 13, 2012.

Additional Materials:

Contact:

Thomas Melito
(202) 512-9601
melitot@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

To develop programs to strengthen developing countries’ Public Financial Management (PFM) systems, the U.S. Agency for International Development (USAID) and the U.S. Department of the Treasury (Treasury) rely on assessments of the host country government’s systems. In 2011, USAID implemented new processes that place a greater emphasis on PFM in its development efforts as the agency aims to increase its use of country systems to deliver assistance. The agency traditionally included PFM capacity-building efforts only as components of broader programs, as it identified relevant weaknesses during the country assessment or program design process. USAID’s new strategy and program development processes include a mandatory assessment of a country’s institutional capacity, including its financial systems, and a requirement to consider the use of country systems to deliver assistance. Most USAID country offices are required to develop a strategy using the new guidance by the end of fiscal year 2013. Treasury’s process for developing programs begins with an initial assessment of the host country’s capabilities. Treasury staff then draft objectives for the program. For example, a Treasury program in Honduras set four objectives, including improving operational efficiency and enhancing accountability by strengthening the organization of the ministry of finance. Once in country, the advisor develops an annual workplan, outlining more specific goals aimed at meeting the overall objectives.

USAID and Treasury use several processes to monitor and evaluate their PFM assistance, but weaknesses exist. USAID uses its regular procedures, which may include performance management plans, periodic progress reporting, site visits, and evaluations, to monitor and evaluate its PFM-related programs. Prior reports by USAID’s Inspector General and GAO have found weaknesses in USAID’s implementation of its monitoring procedures in other programs, including programs from the USAID offices that provide PFM assistance. In addition, USAID is currently unable to monitor overall progress toward its target to obligate 30 percent of its program funds through local systems by 2015. USAID, and GAO in prior reports, have identified a number of weaknesses in evaluation practice. To address weaknesses the agency had identified, USAID adopted a new evaluation policy in January 2011 that states that all large projects are required to have an external evaluation, 3 percent of program budgets should be devoted to external evaluation, and evaluations must use methods that generate the highest quality evidence. Treasury’s processes for monitoring and evaluating its programs include monthly reports, annual quantitative performance measures, voluntary customer feedback surveys, and on-site management reviews, but Treasury does not fully evaluate the performance of its completed technical assistance programs. In addition, Treasury’s quantitative performance measures have been a useful project-level indicator of performance but have not been a useful indicator of overall performance due in part to inherent challenges associated with summarizing program performance and errors introduced when aggregating the performance data. Furthermore, a senior Treasury official reported that Treasury had not yet fully implemented a requirement to conduct independent postproject evaluations of its technical assistance programs.

Why GAO Did This Study

Effective management of public resources can play an important role in a country’s development. In recent years, developing countries committed to strengthen their PFM systems and donors committed to use those systems as much as possible. The United States provides assistance to strengthen PFM systems primarily through USAID and Treasury. USAID conducts capacity building activities to strengthen PFM systems as part of its development programs and has also set a target to obligate 30 percent of its annual assistance through local systems by 2015. Treasury provides technical assistance through advisors who work in country, typically with the finance ministry.

GAO was asked to examine the processes U.S. agencies use to (1) develop programs to strengthen PFM systems and (2) monitor and evaluate those programs. GAO reviewed agency guidance and program documents, interviewed U.S. agency officials, and selected case studies to serve as illustrative examples of PFM-related programs.

What GAO Recommends

USAID should improve its capacity to measure its use of local systems and ensure adequate monitoring of its PFM programs. Treasury should implement additional controls to improve the process for computing program-wide annual performance measures and fully implement its requirement to evaluate the impact of its completed assistance. USAID and Treasury both concurred with GAO’s recommendations.

For more information, contact Thomas Melito at (202) 512-9601 or melitot@gao.gov.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To monitor progress toward USAID's target to obligate 30 percent of its annual assistance through local systems by 2015, the Administrator of USAID should direct the appropriate offices to develop a process to reliably identify and track the agency's use of local systems in all countries receiving assistance.

    Agency Affected: United States Agency for International Development

    Status: Open

    Comments: USAID concurred. In its November 2012 letter, USAID said it planned to provide a report to all USAID missions to validate their progress toward the 30 percent target and provide for continuing improvement of the report. We are following up to verify USAID has done this.

    Recommendation: To help ensure that USAID conducts effective evaluations of PFM-related programs under its new policy, the Administrator of USAID should direct the appropriate offices to ensure that they are establishing adequate monitoring practices for PFM-related programs. Such practices may include selecting proper indicators, collecting reliable baseline data, and ensuring the reliability of reported results.

    Agency Affected: United States Agency for International Development

    Status: Open

    Comments: In its November 2012 letter, USAID said it planned to issue revised guidance, which would include requirements and good practices for selecting appropriate indicators, collecting baselines, setting targets, and ensuring data quality for reported results. USAID said it would supplement its ongoing training to cover performance monitoring. We are following up to verify USAID took these actions.

    Recommendation: To improve the effectiveness of OTA's technical assistance, the Secretary of the Treasury should direct the Office of Technical Assistance (OTA) to implement additional controls to improve the process for computing OTA-wide annual performance measures.

    Agency Affected: Department of the Treasury

    Status: Closed - Implemented

    Comments: In response to the draft of the report provided to OTA, OTA commented that they had already implemented this recommendation by: correcting transcription errors; taking steps to improve data quality controls to prevent transcription errors; improving formatting and usability of the data collection instrument; and increasing staff resources to these efforts. Since the release of the report, OTA reported that in response to the GAO recommendation, OTA has corrected the transcription and arithmetic errors identified in the annual performance measurement exercise, as well as taking steps to strengthen internal controls. OTA's data collection instrument and spreadsheet used to aggregate data now include fields that automatically calculate averages. According to an OTA official, additional internal control measures include an Associate Director's review of the data submitted through the data collection instrument and separate spot checks of the aggregated data set.

    Recommendation: To improve the effectiveness of OTA's technical assistance, the Secretary of the Treasury should direct the Office of Technical Assistance (OTA) to fully implement OTA's existing requirement for end-of-project evaluations and, consistent with its existing guidance, have an independent party conduct the evaluations to improve monitoring and evaluation.

    Agency Affected: Department of the Treasury

    Status: Closed - Implemented

    Comments: OTA fully concurred with this recommendation and in response to this report, began full implementation of this requirement. Since December 2012, OTA has completed 17 end-of-project reports, bringing the program completely up to date for each project concluded since the requirement was instated in December 2009. In addition, an independent party--someone other than the project implementer--conducts the evaluation. According to OTA officials, the evaluations are undertaken typically three to six months following the end of OTA's assistance activities to better understand the program's longer-term impact.

    Aug 28, 2014

    Jul 24, 2014

    Jul 21, 2014

    Jul 9, 2014

    Jul 8, 2014

    Jun 26, 2014

    Looking for more? Browse all our products here