Foreign Assistance:

Federal Monitoring and Evaluation Guidelines Incorporate Most but Not All Leading Practices

GAO-19-466: Published: Jul 31, 2019. Publicly Released: Jul 31, 2019.

Additional Materials:

Contact:

David B. Gootnick
(202) 512-3149
gootnickd@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Are federal agencies doing a good job keeping track of the effectiveness of the foreign assistance programs they administer?

We looked at how agencies monitor and evaluate these programs. Specifically, we looked at the guidelines the Office of Management and Budget established for federal agencies with foreign assistance programs. We reviewed whether the guidelines incorporated leading practices for monitoring and evaluation, and whether agencies adopted the guidelines.

We found the OMB guidelines included 23 of 28 leading practices, and most agencies adopted most of them. We made 7 recommendations to increase the use of OMB's guidelines.

[Image updated to show examples of country flags.]

Photo of flags

Photo of flags

Additional Materials:

Contact:

David B. Gootnick
(202) 512-3149
gootnickd@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Office of Management and Budget's (OMB) foreign assistance Guidelines incorporate most of GAO's leading practices for monitoring and evaluation (M&E), but gaps exist (see figure).

Summary of Office of Management and Budget's Guidelines Addressing GAO's Leading Practices for Monitoring and Evaluation

Summary of Office of Management and Budget's Guidelines Addressing GAO's Leading Practices for Monitoring and Evaluation

  • Monitoring: The Guidelines define monitoring as the continuous tracking of program or project data to determine whether desired results are as expected during implementation. The Guidelines do not require GAO's leading practices on risk assessments, staff qualifications, and program close-out procedures.

  • Evaluation: The Guidelines define evaluation as the systematic collection and analysis of program or project outcomes for making judgments and informing decisions. They do not require GAO's leading practices on developing staff skills and following up on recommendations.

OMB officials indicated the Guidelines are focused on elements required in the Foreign Aid Transparency and Accountability Act of 2016 (FATAA), but noted that agencies can add additional requirements to their own M&E policies. FATAA requires the President to set forth guidelines “according to best practices of monitoring and evaluation.” OMB staff acknowledged that GAO's leading practices are important, but stated that there is no singular established standard for best monitoring practices. Nevertheless, all of GAO's leading practices can help agencies address impediments, effectively manage foreign assistance, and meet their goals.

When assessing agencies' M&E policies against OMB Guidelines, GAO found that agencies incorporated most of the requirements. However, for monitoring, one of the six agencies GAO reviewed—DOD—did not include the requirements to establish agencies' roles and responsibilities and ensure verifiable data for monitoring activities. For evaluation, agencies required most Guideline requirements, but not all. For example, DOD, HHS, and USDA did not require conducting impact evaluations for pilot programs or projects. Without a clear requirement to do such evaluations, agencies risk duplicating or scaling up programs without fully understanding the factors that could lead to their success or failure. Agencies GAO reviewed have plans or mechanisms in place to oversee the implementation of their M&E policies. For example, State developed a guidance document to operationalize and oversee its M&E policy to ensure the implementation of the Guidelines.

Why GAO Did This Study

The Trump Administration requested $28.5 billion in foreign assistance in fiscal year 2019, to be administered by at least 22 federal agencies. Almost 95 percent of this assistance is administered by six agencies—the Departments of Agriculture (USDA), Defense (DOD), State (State), Health and Human Services (HHS), the Millennium Challenge Corporation (MCC), and the U.S. Agency for International Development (USAID). FATAA required the President to set forth guidelines for M&E of U.S. foreign assistance. In January 2018, OMB issued the required guidelines for federal agencies. FATAA also contained a provision for GAO to analyze the guidelines established by OMB; and assess the implementation of the guidelines by the agencies.

In this report, GAO examined the extent to which (1) OMB's M&E Guidelines incorporate GAO leading practices, and (2) agencies incorporate the OMB Guidelines in their M&E policies and plans. GAO assessed the OMB Guidelines against GAO's 28 leading practices identified in GAO-16-861R . GAO also assessed the six agencies' foreign assistance M&E policies against the Guidelines and interviewed OMB and relevant agency officials in Washington, DC.

What GAO Recommends

GAO is making recommendations to OMB, DOD, State, and USDA. OMB did not agree with the recommendation to update the Guidelines, but GAO maintains that doing so can help to emphasize the importance of the M&E practices we identified. DOD, State, and USDA agreed with GAO's recommendations.

For more information, contact David B. Gootnick at (202) 512-3149 or gootnickd@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Priority recommendation

    Comments: OMB disagreed with the recommendation and suggested it would be more effective to remind agencies that, in addition to the guidelines, they should follow all other relevant OMB guidance affecting monitoring and evaluation. OMB asserted that this guidance contains provisions relevant to our leading practices not included in the Foreign Assistance Monitoring and Evaluation Guidelines. However, we believe it is important for OMB to incorporate this other guidance into the Foreign Assistance Monitoring and Evaluation Guidelines, if only by reference, to emphasize the importance of these practices in the context of monitoring and evaluation of foreign assistance. As of April 28, 2020, the Office of Management and Budget has not implemented this recommendation. GAO will continue to monitor this issue.

    Recommendation: The Director of the Office of Management of Budget should update the Guidelines to include GAO's leading practices of developing monitoring plans that are based on risks, ensuring that monitoring staff have appropriate qualifications, establishing procedures to close-out programs, developing staff skills regarding evaluations, and establishing mechanisms for following up on evaluation recommendations. (Recommendation 1)

    Agency Affected: Executive Office of the President: Office of Management and Budget

  2. Status: Open

    Comments: DOD partially concurred with this recommendation and noted that DOD has not utilized the authority provided by 10 U.S.C. 385, which allows DOD to transfer funds to other departments and agencies of the U.S. government for the purposes of implementing or supporting foreign assistance programs. However, DOD is in the process of updating assessment, monitoring, and evaluation guidelines and plans to include a section on how the Department will agree on roles and responsibilities for agencies involved in transfers.

    Recommendation: The Secretary of Defense should update the Department's monitoring and evaluation policies to define roles and responsibilities among agencies that participate in interagency funding transfers. (Recommendation 2)

    Agency Affected: Department of Defense

  3. Status: Open

    Comments: DOD concurred with this recommendation and noted that DOD is in the process of updating assessment, monitoring, and evaluation guidelines and plans to include a section on on verifiable, reliable, and timely data. In addition, DOD reported that the ODASD for Security Cooperation is conducting a monitoring pilot in coordination with Geographic Combatant Commands to inform updated monitoring policies, guidance, processes, and templates based on verifiable, reliable, and timely data.

    Recommendation: The Secretary of Defense should update the Department's monitoring and evaluation policies to ensure verifiable, reliable, and timely data are available to monitoring personnel. (Recommendation 3)

    Agency Affected: Department of Defense

  4. Status: Open

    Comments: DOD partially concurred with this recommendation and noted that the recommendation does not define "medium program in the agency." However, DOD is in the process of updating assessment, monitoring, and evaluation guidelines and plans to include guidance on evaluating all programs, at least once in their lifetimes, whose dollar value equals or exceeds the median for the relevant account. DOD further noted that its guidance will be based on the median dollar value for each relevant account and apply to all levels of evaluation activities.

    Recommendation: The Secretary of Defense should update the Department's monitoring and evaluation policies to ensure that it evaluates all programs, at least once in their lifetimes, whose dollar value equals or exceeds that of the median program in the agency. (Recommendation 4)

    Agency Affected: Department of Defense

  5. Status: Open

    Comments: DOD partially concurred with this recommendation and noted that impact evaluations require a significant resource investment beyond what would be appropriate for more programs. However, DOD is in the process of updating its evaluation guidelines to allow for, but not require, impact evaluations in appropriate cases at all levels of evaluation activities.

    Recommendation: The Secretary of Defense should update the Department's monitoring and evaluation policies to require the agency to conduct impact evaluations on all pilot programs before replicating or expanding, or conduct performance evaluations for those programs and provide a justification for not conducting an impact evaluation. (Recommendation 5)

    Agency Affected: Department of Defense

  6. Status: Open

    Comments: State agreed with the intent of the recommendation (see appendix V for written comments). State explained that impact evaluations are often not feasible in the context of assistance provided under PEPFAR and described its alternative approach to evaluating new initiatives. State indicated it would update appropriate PEPFAR policies to clarify when agencies should conduct impact and/or performance evaluations. These clarifications will reflect how State evaluates PEPFAR programs in practice in accordance with OMB guidance and legislation, according to State.

    Recommendation: The Department of State's U.S. Global AIDS Coordinator, in collaboration with HHS and other implementing agencies, should update the PEPFAR monitoring and evaluation policies to require these agencies to conduct impact evaluations on all pilot programs before replicating or expanding, or conduct performance evaluations for those programs and provide a justification for not conducting an impact evaluation. (Recommendation 6)

    Agency Affected: Department of State

  7. Status: Open

    Comments: USDA agreed with the recommendation and stated that, in practice, FAD encourages implementers to conduct impact evaluations of their projects and requests that the project's evaluation plan include a written justification if an impact evaluation will not be conducted.. USDA further stated that it plans to update the Food Assistance Division's evaluation plan approval standard operating procedures (SOPs) to specify that a justification must be provided in the evaluation plan for not conducting an impact evaluation of a project.

    Recommendation: The Secretary of Agriculture, in collaboration with the Foreign Agriculture Service, should update their monitoring and evaluation policies to require USDA to conduct impact evaluations on all pilot programs before replicating or expanding, or conduct performance evaluations for those programs and provide a justification for not conducting an impact evaluation. (Recommendation 7)

    Agency Affected: Department of Agriculture

 

Explore the full database of GAO's Open Recommendations »

May 12, 2020

May 6, 2020

May 4, 2020

Apr 30, 2020

Mar 18, 2020

Mar 11, 2020

Mar 4, 2020

  • international icon: Art Explosion

    U.S. Assistance to Central America:

    Status of Funding
    GAO-20-163R: Published: Mar 4, 2020. Publicly Released: Mar 4, 2020.

Feb 27, 2020

Looking for more? Browse all our products here