GAO-17-316, Published: Mar 3, 2017. Publicly Released: Mar 3, 2017.
The U.S. government plans to spend about $35 billion on foreign assistance programs in 2017, and program evaluations can help assess and improve the results of this spending.
However, our review of the six agencies providing the most on foreign aid (shown below) found that about a quarter of their program evaluations in 2015 lacked adequate information on results to inform future programs.
We recommended that each agency develop a plan to improve the quality of its evaluations. We also recommended that some agencies improve their procedures to disseminate their evaluation reports—which may help future programs benefit from lessons learned.
Estimated Percentages of Agency Evaluations Generally or Partially Meeting Applicable Quality Criteria or Not Meeting One or More Criteria
Summary Table of Foreign Assistance Evaluation Quality by Agency
An estimated 73 percent of evaluations completed in fiscal year 2015 by the six U.S. agencies GAO reviewed generally or partially addressed all of the quality criteria GAO identified for evaluation design, implementation, and conclusions (see fig.). Agencies met some elements of the criteria more often than others. For example, approximately 90 percent of all evaluations addressed questions that are generally aligned with program goals and were thus able to provide useful information about program results. About 40 percent of evaluations did not use generally appropriate sampling, data collection, or analysis methods. Although implementing evaluations overseas poses significant methodological challenges, GAO identified opportunities for each agency to improve evaluation quality and thereby strengthen its ability to manage aid funds more effectively based on results.
Estimated Percentage of Foreign Assistance Evaluations Meeting Evaluation Quality Criteria
Note: The confidence intervals for our estimates of the quality of agency evaluations according to these categories did not exceed ±8 percent.
Evaluation costs ranged widely and were sometimes difficult to determine, but the majority of evaluations GAO examined cost less than $200,000. Millennium Challenge Corporation (MCC) evaluations had a median cost of about $269,000, while median costs for the U.S. Agency for International Development (USAID), the U.S. Department of Agriculture (USDA), and the Department of State (State) ranged from about $88,000 to about $178,000. GAO was unable to identify the specific costs for the Department of Defense (DOD) and Department of Health and Human Services (HHS) evaluations. High-quality evaluations tend to be more costly, but some well-designed lower-cost evaluations also met all quality criteria. Other factors related to evaluation costs include the evaluation's choice of methodology, its duration, and its location.
Agencies generally posted and distributed evaluations for the use of internal and external stakeholders. However, shortfalls in some agency efforts may limit the evaluations' usefulness.
The U.S. government plans to spend approximately $35 billion on foreign assistance in 2017. Evaluation is an essential tool for U.S. agencies to assess and improve the results of their programs. Government-wide guidance emphasizes the importance of evaluation, and the Foreign Aid Transparency and Accountability Act of 2016 requires the President to establish guidelines for conducting evaluations. However, evaluations can be challenging to conduct. GAO has previously reported on challenges in the design, implementation, and dissemination of the evaluations of individual foreign assistance programs.
GAO was asked to review foreign aid evaluations across multiple agencies. This report examines the (1) quality, (2) cost, and (3) dissemination of foreign aid program evaluations. GAO assessed a representative sample of 173 fiscal year 2015 evaluations for programs at the six agencies providing the largest amounts of U.S. foreign aid —USAID, State, MCC, HHS's Centers for Disease Control and Prevention under the President's Emergency Plan for AIDS Relief, USDA's Foreign Agricultural Service, and DOD's Global Train and Equip program—against leading evaluation quality criteria; analyzed cost and contract documents; and reviewed agency websites and dissemination procedures.
GAO recommends that each of the six agencies develop a plan to improve the quality of its evaluations and that HHS, MCC, State, and USDA improve their procedures and planning for disseminating evaluation reports.
The agencies concurred with our recommendations.
For more information, contact Jessica Farb at (202) 512-6991 or email@example.com.