This is the accessible text file for GAO report number GAO-09-980 
entitled 'International Food Assistance: USAID Is Taking Actions to 
Improve Monitoring and Evaluation of Nonemergency Food Aid, but 
Weaknesses in Planning Could Impede Efforts' which was released on 
September 28, 2009. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Committees: 

United States Government Accountability Office: 
GAO: 

September 2009: 

International Food Assistance: 

USAID Is Taking Actions to Improve Monitoring and Evaluation of 
Nonemergency Food Aid, but Weaknesses in Planning Could Impede Efforts: 

GAO-09-980: 

GAO Highlights: 

Highlights of GAO-09-980, a report to congressional committees. 

Why GAO Did This Study: 

In passing the Food for Peace Act in 2008, Congress authorized up to 
$22 million annually for fiscal years 2009 to 2012 to the U.S. Agency 
for International Development (USAID) to improve, monitor, and evaluate 
the effectiveness and efficiency of nonemergency food aid programs. 
Congress also required USAID to report on its oversight of these 
programs and the Comptroller General to review and report to Congress 
on USAID’s report. Through analysis of agency documents; interviews 
with agency officials, experts, and partners; and visits to Bangladesh 
and Haiti, this mandated report reviews (1) USAID’s plans and actions 
to improve its monitoring and evaluation of nonemergency food aid 
programs and (2) the extent to which USAID has integrated its 
monitoring and evaluation of nonemergency food aid with program 
management. 

What GAO Found: 

USAID’s actions to improve its monitoring and evaluation of these 
programs could be hindered by weak planning. Monitoring is essential to 
ensuring that USAID’s nonemergency food aid programs in developing 
countries are implemented as intended, and evaluation helps to assure 
that these programs achieve their goal of reducing global food 
insecurity. First, with funding from the Food for Peace Act, USAID’s 
Office of Food for Peace (FFP) plans to increase the number of field 
staff responsible for the monitoring of nonemergency food aid programs, 
has provided funding for the Famine Early Warning Systems Network, and 
has initiated an upgrade of its information technology system. However, 
FFP’s plans for the information technology upgrade lack a concept of 
operations document, which describes system characteristics for a 
proposed system from a user’s point of view and includes high-level 
descriptions of information systems, their interrelationships, and 
information flows. Second, with funding from other sources, USAID plans 
to carry out additional actions in an effort to improve its oversight 
of food aid, including the expansion of a computerized system for 
monitoring the implementation and management of nonemergency food aid 
programs. However, USAID has not determined a stable source of funding 
for these initiatives beyond the first year of operations due to legal 
restrictions that preclude the agency from using the newly authorized 
funding for grants and cooperative agreements. 

USAID’s monitoring and evaluation of its nonemergency food aid programs 
are consistent to varying degrees with some of the principles 
established by the American Evaluation Association’s Task Force on 
Evaluation Policy to integrate evaluation into program management. GAO 
found that (1) FFP’s actions were generally consistent with the 
principles for policies and procedures and for independence. For 
example, FFP has issued policies and procedures for monitoring and 
evaluating food aid programs and generally uses external evaluators to 
assess its multiyear assistance programs. (2) FFP’s actions were 
partially consistent with the principles for scope and coverage, 
dissemination of results, professional competence, and resources. For 
example, FFP relies on a range of staff to perform its monitoring and 
evaluation, but does not have reliable data on the numbers of field 
staff who have competencies in monitoring and evaluation, or their 
specific skills. (3) FFP’s actions were not consistent with the 
principles for monitoring and evaluation plans. While FFP plans some of 
its monitoring and evaluation activities—such as final evaluations for 
multiyear assistance programs—it lacks an integrated plan to ensure 
that monitoring and evaluation results will be used to improve program 
management. 

What GAO Recommends: 

GAO recommends that the USAID Administrator (1) develop a concept of 
operations document to help reduce the risks associated with upgrading 
FFP’s information technology system and (2) develop an integrated 
monitoring and evaluation plan to better oversee nonemergency food aid 
programs. USAID agreed with the first recommendation, but disagreed 
with the second recommendation, asserting that GAO did not fully 
consider its planning processes. GAO believes the recommendation 
remains valid in view of current findings. 

View [hyperlink, http://www.gao.gov/products/GAO-09-980] or key 
components. For more information, contact Thomas Melito at (202) 512-
9601 or melitot@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

Weaknesses in Planning Could Hinder USAID's Actions to Improve 
Monitoring and Evaluation of Nonemergency Food Aid Programs: 

USAID's Monitoring and Evaluation of Nonemergency Food Aid Are Not 
Fully Integrated with Program Management: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: USAID Report to Congress: 

Appendix III: Examples of Activities Supported with Nonemergency Food 
Aid: 

Appendix IV: Comments from USAID: 

Appendix V: GAO Contact and Staff Acknowledgments: 

Related GAO Products: 

Table: 

Table 1: Types of Monitoring and Evaluation Activities Performed by FFP 
Staff, Implementing Partners, and External Evaluators: 

Figures: 

Figure 1: Countries That Received U.S. Nonemergency Food Aid in Fiscal 
Year 2008: 

Figure 2: A Framework for Program Management of Nonemergency Food Aid 
That Integrates Monitoring and Evaluation: 

Figure 3: USAID Allocations of the Food for Peace Act Funding for 
Monitoring and Evaluation in Fiscal Year 2009 and Implementation 
Timelines: 

Figure 4: FFP Priority Countries and Locations of New Positions for the 
Monitoring of Nonemergency Food Aid Programs: 

Figure 5: Examples of Activities Supported with Nonemergency Food Aid: 

Abbreviations: 

ADS: Automated Directives System: 

AEA: American Evaluation Association: 

AED: Academy for Educational Development: 

BEST: Bellmon Estimation for Title II: 

DA: Development Assistance: 

DAP: Development Assistance Program: 

DEC: Development Experience Clearinghouse: 

EFP: emergency food product: 

FANTA-2: Food and Nutrition Technical Assistance II Project: 

FAO: Food and Agriculture Organization: 

FEWS NET: Famine Early Warning Systems Network: 

FFP: Office of Food for Peace: 

FFPMS: Food for Peace Management System: 

IEEE: Institute of Electrical and Electronics Engineers: 

MYAP: Multiyear Assistance Program: 

NGO: nongovernmental organization: 

OFDA: Office of Foreign Disaster Assistance: 

PDA: personal data assistant: 

PM2A: Preventing Malnutrition in Children Under 2 Approach: 

SYAP: Single-Year Assistance Program: 

UN: United Nations: 

USAID: U.S. Agency for International Development: 

WFP: World Food Program: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

September 28, 2009: 

The Honorable Blanche Lambert Lincoln: Chair:
The Honorable Saxby Chambliss:
Ranking Minority Member:
Committee on Agriculture, Nutrition, and Forestry: 
United States Senate: 

The Honorable Collin C. Peterson:
Chair:
The Honorable Frank D. Lucas:
Ranking Member:
Committee on Agriculture:
House of Representatives: 

The Honorable Howard L. Berman:
Chair:
The Honorable Ileana Ros-Lehtinen:
Ranking Member:
Committee on Foreign Affairs:
House of Representatives: 

The number of undernourished people worldwide has been growing and now 
exceeds 1 billion, according to the estimates of the United Nations 
(UN) Food and Agriculture Organization (FAO). This calls into question 
the feasibility of attaining the 1996 World Food Summit target of 
reducing the number of hungry people by half to no more than 420 
million by 2015. The United States is the world's largest donor of food 
aid, and the U.S. Agency for International Development (USAID) Office 
for Food for Peace (FFP) administers the largest nonemergency food aid 
program funded under the Food for Peace Act.[Footnote 1] FFP has 
adopted as its strategic objective the reduction of food insecurity in 
vulnerable populations. Specifically, FFP's Strategic Plan for 2006- 
2010 identifies two intermediate results: (1) enhancing FFP's global 
leadership in reducing global food insecurity and (2) increasing FFP's 
program impact in the field.[Footnote 2] 

Monitoring[Footnote 3] and evaluation[Footnote 4] of nonemergency food 
aid--one of the largest sources of U.S. funding focused on food 
security--are critical oversight and program management tools that 
would help to ensure that FFP meets its strategic objective. The size 
and scope of FFP's activities are substantial--in fiscal year 2008, FFP 
supported 20 grantees that implemented 92 activities in 28 countries-- 
but a number of recent reviews have found that monitoring and 
evaluation of these activities have been insufficient. In 2007, we 
found that the USAID Inspector General had previously reported that 
monitoring had not been regular and systematic and that, in some cases, 
intended recipients had not received food aid or the number of 
recipients could not be verified. In April 2007, we reported that USAID 
had only 23 Title II-funded staff assigned to missions and regional 
offices in just 10 countries to monitor programs that cost about $1.7 
billion in 55 countries in the previous fiscal year. Our audit work 
also indicated that monitoring had been insufficient due to various 
factors including limited staff, competing priorities, and legal 
restrictions on the use of food aid resources. To ensure the effective 
use of food aid, we recommended that USAID improve its monitoring of 
food aid programs to strengthen proper management and implementation of 
these programs.[Footnote 5] USAID agreed, noting that monitoring of all 
its grant programs is required and that its implementing partners are 
legally required to monitor and report their findings to USAID. Like 
monitoring, evaluation is crucial to ensuring that best practices and 
lessons learned are considered in the management and implementation of 
existing programs and in designing new ones. However, for many years, 
USAID did not place a great deal of importance or invest many resources 
in evaluation. In a June 2009 speech, the Secretary of State, whose 
department coordinates U.S. development assistance, recognized that "…
too often, program evaluation is treated as an afterthought, and 
sometimes is overlooked entirely." To rectify this situation, the 
Secretary announced that the Department of State (State) had for the 
first time developed a policy for integrating evaluation into its work 
worldwide.[Footnote 6] 

In passing the Food for Peace Act in 2008, Congress authorized up to 
$22 million annually in funding for fiscal years 2009 through 2012 to 
USAID for program oversight to improve, monitor, and evaluate the 
effectiveness and efficiency of nonemergency food aid programs--and, 
consistent with our 2007 recommendation, required the USAID 
Administrator to submit an implementation report on the agency's 
efforts in these areas.[Footnote 7] Congress also required the 
Comptroller General of the United States to review USAID's report and 
provide recommendations for improvement. To address this congressional 
mandate, we reviewed (1) USAID's plans and actions to improve its 
monitoring and evaluation of nonemergency food aid programs provided 
under the Food for Peace Act and (2) the extent to which USAID has 
integrated its monitoring and evaluation of nonemergency food aid with 
program management, particularly with regard to established principles. 
In this review, we focused on oversight practices of FFP, not of USAID 
as a whole. 

To address these objectives, we examined USAID's December 2008 report 
to Congress, which outlines FFP's plans to improve its monitoring and 
evaluation of nonemergency food aid programs, and actions FFP has taken 
to date. We interviewed USAID officials, including FFP field staff 
responsible for monitoring the implementation of nonemergency food aid 
programs, and implementing partners such as nongovernmental 
organizations (NGO) and the World Food Program (WFP). We considered 
established principles for monitoring and evaluation--especially the 
"Roadmap" principles of the American Evaluation Association 
(AEA),[Footnote 8] which have been developed to integrate monitoring 
and evaluation with program management; GAO standards and prior GAO 
work; USAID guidance; and guidelines that the Institute of Electrical 
and Electronics Engineers (IEEE) has established for information 
technology.[Footnote 9] We conducted fieldwork in Bangladesh and 
Haiti,[Footnote 10] where we met with U.S. mission officials, host 
government officials, bilateral and multilateral donor representatives, 
and NGO representatives. In Bangladesh and Haiti, we also visited 
various WFP and USAID project sites supported by nonemergency food aid 
and food aid logistics facilities. 

We conducted this performance audit from March 2009 to September 2009, 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. (Appendix I provides a 
detailed discussion of our objectives, scope, and methodology.) 

Results in Brief: 

USAID's actions to improve its monitoring and evaluation of 
nonemergency food aid programs could be hindered by weak planning. 
First, with funding from the Food for Peace Act, USAID's FFP (1) plans 
to use $5 million to increase the number of field staff responsible for 
the monitoring of food aid programs, but it does not have a systematic 
process in place to determine appropriate staffing levels for effective 
monitoring and evaluation; (2) has provided $7.6 million in funding for 
the Famine Early Warning Systems Network (FEWS NET) operational budget 
in fiscal year 2009; and (3) has issued a contract for $637,000 to 
begin upgrading its information technology system, which is expected to 
fully capture food commodity management data and have the capability to 
interface with USAID's core financial system. However, FFP's plans for 
the information technology upgrade lack a concept of operations 
document, which describes characteristics for a proposed system from a 
user's point of view and includes high-level descriptions of 
information systems, their interrelationships, and information flows. 
Second, with funding from other sources, FFP (1) plans to expand the 
use of Layers--a computerized system for monitoring the implementation 
and management of nonemergency food aid programs--from 3 pilot 
countries to 20 countries by the end of 2012; (2) has commissioned 
several cross-cutting evaluations of nonemergency food aid projects 
through USAID's cooperative agreement with the Food and Nutrition 
Technical Assistance (FANTA-2) Project of the Academy for Educational 
Development (AED) and plans to improve the quality of assessments and 
targeting approaches for FFP programs; and (3) has launched a 3-year 
pilot project intended to improve the market analysis required before 
food aid programs are approved in recipient countries. In May 2009, 
however, USAID's General Counsel determined that the agency does not 
have the legal authority to use Food for Peace Act funding for the 
cooperative agreement required to implement Layers and the cross- 
cutting evaluations, which are essential to improving monitoring and 
evaluation of USAID's nonemergency food aid programs. USAID has not 
made provisions for alternative sources of funding for these 
initiatives beyond the cost of the start-up and first year of 
operations. 

USAID's monitoring and evaluation of its nonemergency food aid programs 
are consistent to varying degrees with some of the "Roadmap" principles 
established by AEA to integrate evaluation into program management. We 
found that USAID's actions were (1) generally consistent with the AEA 
principles for policies and procedures and for independence; (2) 
partially consistent with the principles for scope and coverage, 
dissemination of results, professional competence, and resources; and 
(3) not consistent with the principles for monitoring and evaluation 
plans, as discussed below: 

* Policies and procedures. In accordance with AEA's recommendation for 
established and published policies for monitoring and evaluation, 
USAID's guidance--which applies to FFP--details the agency's monitoring 
and evaluation requirements. In addition, FFP has issued several 
information bulletins for staff and implementing partners that explain 
policies and requirements regarding monitoring and evaluation, 
reporting, and performance indicators. 

* Independence. To help ensure independence as outlined in AEA's 
"Roadmap" principles, FFP requires that each multiyear assistance 
program's final evaluation be performed by external evaluators. In 
addition, it generally uses external entities to perform cross-cutting 
evaluations with funding from the Food for Peace Act and other sources. 

* Scope and coverage. In accordance with AEA's principle that 
monitoring should be conducted for most of an agency's programs and 
throughout the programs' life cycle, FFP requires the monitoring and 
evaluation of all of its multiyear assistance programs and has 
published policies and procedures to guide their oversight. However, in 
recent years, FFP has conducted only a few periodic, cross-cutting 
evaluations that examined the relative effectiveness of a range of its 
programs, projects, and activities. 

* Dissemination of results. AEA recommends that agencies disseminate 
the results of their monitoring and evaluation activities on easily 
accessible Web sites. However, FFP does not routinely and consistently 
disseminate evaluation results. Although an FFP requirement states that 
project evaluations should be posted in a central electronic 
clearinghouse, FFP has not ensured that implementing partners routinely 
comply with it. 

* Professional competence. AEA recommends that agencies should have 
competent and qualified staff to perform monitoring and evaluation. FFP 
relies on a range of staff to perform its monitoring and evaluation, 
including field staff, implementing partners' monitoring and evaluation 
staff, and FANTA-2 staff. However, it does not have reliable data on 
the numbers of field staff who have monitoring and evaluation 
competencies, or their specific skills. 

* Resources. AEA recommends that agencies should have sufficient 
resources for monitoring and evaluation. Although the 2008 Food for 
Peace Act grants USAID up to $22 million in new funding annually in 
fiscal years 2009 through 2012 to improve, monitor, and evaluate the 
effectiveness and efficiency of nonemergency food aid programs, it is 
difficult to determine whether FFP's level of funding for proper 
oversight of nonemergency food aid is sufficient because FFP does not 
know exactly how much it spends on monitoring and evaluation and has 
not done an evaluation of its resource needs. 

* Monitoring and evaluation plans. AEA recommends that agencies should 
have multiyear plans, updated annually, that take into account 
evaluation results for program budgeting, as well as ongoing program 
development and management. While FFP plans some of its monitoring and 
evaluation activities, such as final evaluations for multiyear 
assistance programs, it lacks an integrated plan for all of its 
oversight activities that takes into account the need for evaluation 
results as well as risk-based policy-making and management needs. In 
addition, FFP neither systematically prioritizes its monitoring and 
evaluation needs nor links resources to risk-based needs. 

To reduce the risks associated with upgrading FFP's information 
technology system, we are recommending that the Administrator of USAID 
develop a concept of operations document. Such a document should adhere 
to industry best practices and include key elements such as major 
system components, interfaces to external systems, and performance 
characteristics. It would also describe system characteristics for a 
proposed system from the user's point of view and include high-level 
descriptions of information systems, their interrelationships, and 
information flows. 

In view of USAID's new authority to direct up to $88 million over a 4- 
year period to improve, monitor, and evaluate the effectiveness and 
efficiency of nonemergency food aid programs, we also recommend that 
USAID develop an integrated monitoring and evaluation plan for its 
nonemergency food aid programs. Such a plan should, among other things, 
(1) link monitoring and evaluation to key USAID and FFP goals; (2) 
establish a systematic process for determining appropriate budget 
levels and staff resources for monitoring and evaluation based on an 
analysis of factors such as the nature, scope, and size of the 
programs; (3) examine all available options for funding monitoring and 
evaluation at headquarters and in recipient countries; and (4) 
establish time frames for implementing and evaluating the plan. 

USAID concurred with our first recommendation, but disagreed with the 
second recommendation, asserting that we did not adequately take into 
account documents that detail its planning processes. We considered all 
the documents that USAID cited and found that they did not constitute a 
plan, but only provided guidance for planning. Furthermore, the 
weaknesses we identified in FFP's monitoring and evaluation efforts, 
including in the areas of resources and professional competence, 
demonstrate the need for a comprehensive monitoring and evaluation 
plan. USAID provided written comments on a draft of this report. We 
believe this recommendation remains valid. 

We have reprinted USAID's comments in appendix IV, along with our 
responses. Additionally, USAID provided technical comments on a draft 
of this report, which we have addressed as appropriate. 

Background: 

USAID Plans to Administer Larger Nonemergency Food Aid Programs in 
Fewer Priority Countries: 

FFP is shifting to larger programs in fewer countries in order to 
leverage nonemergency food aid resources more effectively. FFP's 
Strategic Plan for 2006-2010 concentrates on a single objective of 
reducing food insecurity in vulnerable populations with an aim to fund 
programs that can be modified when circumstances change. In fiscal year 
2008, under the Food for Peace Act, the United States provided $354 
million in nonemergency food aid and FFP administered nonemergency food 
aid programs in 28 countries. Beginning in fiscal year 2010, FFP will 
limit its nonemergency food aid programs to 20 priority countries. 
[Footnote 11] Of these, 16 are in sub-Saharan Africa (Burkina Faso, 
Burundi, Chad, the Democratic Republic of Congo, Ethiopia, Liberia, 
Madagascar, Malawi, Mali, Mauritania, Mozambique, Niger, Sierra Leone, 
Sudan, Uganda, and Zambia), 2 are in Central and South Asia 
(Afghanistan and Bangladesh), and 2 are in Latin America and the 
Caribbean (Guatemala and Haiti). Figure 1 lists the countries that 
received nonemergency food aid in fiscal year 2008 and the most recent 
available statistics on the number of undernourished people and the 
prevalence of undernourishment in these countries, respectively. 

Figure 1: Countries That Received U.S. Nonemergency Food Aid in Fiscal 
Year 2008 (U.S. dollars in millions): 

[Refer to PDF for image: horizontal bar graph] 

Country; Mali; 
Nonemergency food aid: $2.0 million; 
Number of undernourished people, 2003-2005: 1.2 million; 
Percentage of population that was undernourished in 2003-2005: 11%. 

Country; Guinea; 
Nonemergency food aid: $3.0 million; 
Number of undernourished people, 2003-2005: 1.5 million; 
Percentage of population that was undernourished in 2003-2005: 17%. 

Country; Senegal; 
Nonemergency food aid: $4.1 million; 
Number of undernourished people, 2003-2005: 3.0 million; 
Percentage of population that was undernourished in 2003-2005: 26%. 

Country; Chad; 
Nonemergency food aid: $4.5 million; 
Number of undernourished people, 2003-2005: 3.8 million; 
Percentage of population that was undernourished in 2003-2005: 39%. 

Country; Burundi; 
Nonemergency food aid: $4.9 million; 
Number of undernourished people, 2003-2005: 4.8 million; 
Percentage of population that was undernourished in 2003-2005: 63%. 

Country; Mauritania; 
Nonemergency food aid: $5.0 million; 
Number of undernourished people, 2003-2005: 0.2 million; 
Percentage of population that was undernourished in 2003-2005: 8%. 

Country; Ghana; 
Nonemergency food aid: $6.9 million; 
Number of undernourished people, 2003-2005: 1.9 million; 
Percentage of population that was undernourished in 2003-2005: 9%. 

Country; Sierra Leone; 
Nonemergency food aid: $6.9 million; 
Number of undernourished people, 2003-2005: 2.5 million; 
Percentage of population that was undernourished in 2003-2005: 47%. 

Country; Liberia; 
Nonemergency food aid: $7.7 million; 
Number of undernourished people, 2003-2005: 1.3 million; 
Percentage of population that was undernourished in 2003-2005: 40%. 

Country; Bolivia; 
Nonemergency food aid: $9.0 million; 
Number of undernourished people, 2003-2005: 2.0 million; 
Percentage of population that was undernourished in 2003-2005: 22%. 

Country; Zambia; 
Nonemergency food aid: $9.1 million; 
Number of undernourished people, 2003-2005: 5.1 million; 
Percentage of population that was undernourished in 2003-2005: 45%. 

Country; Nicaragua; 
Nonemergency food aid: $9.3 million; 
Number of undernourished people, 2003-2005: 1.2 million; 
Percentage of population that was undernourished in 2003-2005: 22%. 

Country; Kenya; 
Nonemergency food aid: $9.4 million; 
Number of undernourished people, 2003-2005: 11.0 million; 
Percentage of population that was undernourished in 2003-2005: 32%. 

Country; Democratic Republic of Congo; 
Nonemergency food aid: $9.8 million; 
Number of undernourished people, 2003-2005: 43.0 million; 
Percentage of population that was undernourished in 2003-2005: 76%. 

Country; Afghanistan; 
Nonemergency food aid: $9.9 million; 
Number of undernourished people, 2003-2005: not available; 
Percentage of population that was undernourished in 2003-2005: not 
available. 

Country; Burkina Faso; 
Nonemergency food aid: $10.0 million; 
Number of undernourished people, 2003-2005: 1.3 million; 
Percentage of population that was undernourished in 2003-2005: 10%. 

Country; Honduras; 
Nonemergency food aid: $10.2 million; 
Number of undernourished people, 2003-2005: 0.8 million; 
Percentage of population that was undernourished in 2003-2005: 12%. 

Country; Rwanda; 
Nonemergency food aid: $11.4 million; 
Number of undernourished people, 2003-2005: 3.6 million; 
Percentage of population that was undernourished in 2003-2005: 40%. 

Country; Madagascar; 
Nonemergency food aid: $11.8 million; 
Number of undernourished people, 2003-2005: 6.6 million; 
Percentage of population that was undernourished in 2003-2005: 37%. 

Country; Niger; 
Nonemergency food aid: $12.8 million; 
Number of undernourished people, 2003-2005: 3.7 million; 
Percentage of population that was undernourished in 2003-2005: 29%. 

Country; India; 
Nonemergency food aid: $13.5 million; 
Number of undernourished people, 2003-2005: 230.5 million; 
Percentage of population that was undernourished in 2003-2005: 21%. 

Country; Guatemala; 
Nonemergency food aid: $13.9 million; 
Number of undernourished people, 2003-2005: 2.0 million; 
Percentage of population that was undernourished in 2003-2005: 16%. 

Country; Malawi; 
Nonemergency food aid: $17.9 million; 
Number of undernourished people, 2003-2005: 3.8 million; 
Percentage of population that was undernourished in 2003-2005: 29%. 

Country; Mozambique; 
Nonemergency food aid: $19.9 million; 
Number of undernourished people, 2003-2005: 7.5 million; 
Percentage of population that was undernourished in 2003-2005: 38%. 

Country; Ethiopia; 
Nonemergency food aid: $23.2 million; 
Number of undernourished people, 2003-2005: 35.2 million; 
Percentage of population that was undernourished in 2003-2005: 46%. 

Country; Uganda; 
Nonemergency food aid: $26.4 million; 
Number of undernourished people, 2003-2005: 4.1 million; 
Percentage of population that was undernourished in 2003-2005: 15%. 

Country; Haiti; 
Nonemergency food aid: $34.2v
Number of undernourished people, 2003-2005: 5.3 million; 
Percentage of population that was undernourished in 2003-2005: 58%. 

Country; Bangladesh; 
Nonemergency food aid: $47.8 million; 
Number of undernourished people, 2003-2005: 40.1 million; 
Percentage of population that was undernourished in 2003-2005: 27%. 

Sources: GAO analysis of USAID and Food and Agriculture Organization of 
the United Nations data. 

[End of figure] 

Food for Peace Nonemergency Food Aid Programs Are Implemented through 
Grants for Single-Year and Multiyear Assistance Programs: 

FFP provides food aid funding under the Food for Peace Act using two 
primary funding mechanisms--Single-Year Assistance Programs (SYAP) and 
Multiyear Assistance Programs (MYAP). SYAPs address emergency needs and 
are used for direct distribution of food aid in emergency 
contexts.[Footnote 12] MYAPs, long-term programs with duration of 3 to 
5 years, are used to implement development activities that target 
chronically food-insecure populations.[Footnote 13] In carrying out 
nonemergency programs through its implementing partners, FFP uses a 
combination of Food for Peace commodities for direct distribution, 
local currency generated through monetization,[Footnote 14] and cash 
resources to implement activities that target chronic food insecurity. 
(For illustrative examples of the types of activities supported by 
nonemergency food aid, such as project activities we observed in 
Bangladesh and Haiti, see appendix III.) 

Monitoring and Evaluation Are Essential to Effective Program 
Management: 

Monitoring and evaluation perform two separate but related functions. 
Monitoring is the collection of data to determine whether programs are 
being implemented as intended and the tracking of progress using 
preselected indicators throughout the life of the program. Evaluation 
consists of periodic or ad hoc studies to assess how and whether 
programs achieved their expected goals. It can also consider what 
outcomes can be attributed to the program, as well as program cost- 
effectiveness. Evaluations can rely on a range of quantitative and 
qualitative measures as well as preselected indicators, comprehensive 
research designs, and the use of appropriate statistical analysis of 
the data. Figure 2 outlines a framework based on our review of Food for 
Peace nonemergency programs, as well as AEA and WFP guidance that 
illustrates how monitoring and evaluation could be integrated with 
program management[Footnote 15] of nonemergency food aid. 

Figure 2: A Framework for Program Management of Nonemergency Food Aid 
That Integrates Monitoring and Evaluation: 

[Refer to PDF for image: illustration] 

Program Management: 

Assessment and analysis: 
* Assess the context, extent, and magnitude of needs arising from 
current or impending food insecurity. 
* Prioritize assistance for affected population groups and geographical 
areas. 

Planning: 
* Identify characteristics and needs of target population. 
* Develop objectives.
* Select food distribution and/or development activity. 

Monitoring: 
* Determine if necessary inputs (equipment, supplies, and personnel) 
are in place. 
– Ensure that food is properly stored. 
– Check to see if health clinics are sufficiently stocked for intended 
beneficiaries. 
* Determine if programs are achieving their expected outputs by 
tracking performance indicators. 
* Ensure that target number of beneficiaries are receiving food and 
attending clinics. 

Implementation: 
* Early childhood development (accompanied by photograph); 
* Mother and child health and nutrition (accompanied by photograph); 
* Income generating activities (accompanied by two photographs). 
 
Evaluation: 
Focus on the extent to which program objectives were achieved and what 
outcomes can be directly attributed to programs. 
* Direct distribution programs may consider how the ration is affecting 
the health and nutritional status of beneficiaries. 
* Food-for-work programs may focus on how the activity is addressing 
temporary food insecurity and developing a sustainable community asset. 

Sources: GAO analysis, adapted from WFP information; GAO and USAID 
(photos). 

[End of figure] 

FFP Relies on a Range of Staff to Perform Various Types of Monitoring 
and Evaluation Activities: 

FFP relies on a range of staff to perform its monitoring and 
evaluation--including FFP officers in the field, implementing partners, 
contractors, and FANTA-2 staff.[Footnote 16] According to FFP, field 
staff visit project sites to ensure that programs are being implemented 
as expected. They also approve and oversee the implementing partners' 
selection of performance indicators and data collection. In addition to 
regularly collecting performance data, the implementing partners 
perform baseline studies and midterm evaluations for their multiyear 
programs and hire external evaluators to conduct the final evaluations 
for the multiyear programs. In addition, FFP relies on appropriately 
credentialed consultants and researchers for some cross-cutting 
evaluations, which, for example, consider the agency's exit strategies 
for nonemergency food aid programs in Bolivia, Honduras, and Kenya. 

Conducting evaluations can present many challenges, such as obtaining 
reliable data and employing qualified evaluators. These challenges can 
be compounded in developing countries, where local conditions may be 
difficult and local capacity is often limited. For example, a recent 
review of evaluations conducted by an international NGO cited numerous 
challenges in-country including climate, poor economic status, civil 
unrest, migration of beneficiaries, cultural biases, poor 
recordkeeping, and turnover in government and NGO staff.[Footnote 17] 

Table 1 summarizes the various types of monitoring and evaluation 
activities that FFP performs. The table also indicates which staff have 
primary responsibility for the activities, briefly describes the 
activities, and provides examples of them. 

Table 1: Types of Monitoring and Evaluation Activities Performed by FFP 
Staff, Implementing Partners, and External Evaluators: 

Types of monitoring and evaluation activities: Monitoring of project 
sites and activities; 
Staff with primary responsibility: USAID mission staff; 
Description of activities: Field staff visit project sites to check on 
inputs and program implementation; 
Examples: Field staff checks on, for example: 
* storage conditions of food aid warehouses, and; 
* whether clinics for mothers and children offer the required services. 

Types of monitoring and evaluation activities: Tracking of performance 
measures; 
Staff with primary responsibility: Implementing partners with oversight 
from FFP and the USAID mission; 
Description of activities: Implementing partners select performance 
measures used by FFP, State/F,[A] and the mission. USAID headquarters 
staff review and approve the indicators. Implementing partners collect 
data on the measures; 
Examples: USAID and implementing partners track, for example: 
* FFP indicators (e.g., the number of farmers who received extension 
services), and; 
* State/F indicators (e.g., the number of beneficiaries of agricultural 
assistance). 

Types of monitoring and evaluation activities: Evaluating program 
outcomes; 
Staff with primary responsibility: External evaluators; Description of 
activities: Implementing partners hire external evaluators to conduct 
the projects' final evaluations, and determine whether the goals were 
achieved; 
Examples: One final evaluation reported a reduction in acute 
malnutrition in children aged 6-59 months during the life of the 
project. The evaluations generally also provided reasons that goals 
were or were not met, and lessons learned. 

Types of monitoring and evaluation activities: Evaluating program 
impacts; 
Staff with primary responsibility: FANTA-2, its partner organizations, 
and contractors to FFP; 
Description of activities: FFP has commissioned several cross-cutting 
studies designed to evaluate what impacts can be attributed to its 
programs and to analyze the relative effectiveness of alternative 
delivery mechanisms; 
Examples: One evaluation examined the impact and cost-effectiveness of 
two approaches to prevent malnutrition in infants, using randomly 
selected comparison groups to judge the impacts and effectiveness of 
each approach. 

Types of monitoring and evaluation activities: Related studies; 
Staff with primary responsibility: FANTA-2, its partner organizations, 
and contractors to FFP; 
Description of activities: Studies that are intended to help understand 
and improve aspects of program management; 
Examples: FFP plans another full review of all its multiyear projects. 
The last such review recommended a series of improvements in, for 
example, program design and results reporting. 

Sources: GAO analysis of AED/FANTA-2 and USAID documents. 

[A] In June 2006, the Office of the Director of Foreign Assistance 
(State/F) was created at State to carry out the Director of Foreign 
Assistance's responsibilities and focus the use of foreign assistance 
on achieving the Secretary's transformational diplomacy goal: "to help 
build and sustain democratic, well-governed states that will respond to 
the needs of their people, reduce widespread poverty, and conduct 
themselves responsibly in the international system." State/F was given 
responsibility for developing, among other things (1) a coherent, 
coordinated U.S. government foreign assistance strategy; (2) multiyear 
country-specific assistance strategies and annual country specific 
operational plans; (3) consolidated policy, planning, budget, and 
implementation mechanisms and staff functions required to provide 
leadership to USAID and State foreign assistance; and (4) guidance for 
foreign assistance delivered through other U.S. government agencies. 

[End of table] 

Weaknesses in Planning Could Hinder USAID's Actions to Improve 
Monitoring and Evaluation of Nonemergency Food Aid Programs: 

USAID has begun taking a series of actions to improve its monitoring 
and evaluation of nonemergency food aid programs, as mandated by the 
Food for Peace Act passed in July 2008. These efforts include (1) 
increasing the number of field staff responsible for the monitoring of 
nonemergency food aid programs; (2) expanding FEWS NET; (3) upgrading 
FFP's information technology system; (4) expanding the use of Layers, a 
computerized monitoring system for quality assurance; (5) improving the 
quality of evaluations, assessments, and targeting approaches; and (6) 
launching a pilot project to improve market analysis in countries 
receiving U.S. nonemergency food aid. Weaknesses in planning could 
hinder USAID's ability to effectively implement these actions, but it 
is too early to assess impact because the agency's efforts are still in 
progress. As of August 2009, USAID estimated that, in fiscal year 2009, 
it will have spent about $13 million of the $22 million authorized for 
monitoring and evaluation of nonemergency food aid under the Food for 
Peace Act. Figure 3 summarizes the allocation of the funds and the 
implementation timelines. 

Figure 3: USAID Allocations of the Food for Peace Act Funding for 
Monitoring and Evaluation in Fiscal Year 2009 and Implementation 
Timelines: 

[Refer to PDF for image: pie-chart and timeline] 

Allocation of Food for Peace Act funding: 

New FFP field staff positions, $5,000,000[A]: 23%; 
FEWS NET, $7,600,000: 35%; 
Information technology upgrade, $637,000: 3%; 
Monetization study, $5,000: .02%; 
Unused funding, $8,758,000: 40%. 

Implementation timelines: 

New FFP field staff positions: Late FY 2008 through Mid-FY 2010; 
FEWS NET: Early FY2009 through Late FY2009; 
Information technology upgrade: Late FY2009 through Late FY2010; 
Monetization study: Late FY2009 through Early FY2010. 

Source: GAO analysis of USAID data. 

[A] As of August 2009, USAID had filled 9 of the 21 new positions and 
will therefore not spend the full $5 million it planned to spend within 
the fiscal year. 

[End of figure] 

USAID Has Allocated New Funding to Increase Staffing Levels and Upgrade 
Information Systems but Has Not Developed Key Processes and Plans in 
These Areas: 

Staffing Levels: 

USAID has announced plans to increase the number of field staff 
responsible for the monitoring of food aid programs, which would 
provide minimal FFP presence in all priority countries. The agency 
plans to use approximately $5 million of its new Food for Peace Act 
funding to add 21 full-time field staff whose responsibilities include 
the monitoring of nonemergency food aid programs. According to USAID, 1 
of the new positions is located in USAID's Southern Africa Regional 
Office in Pretoria (South Africa) while the other 20 new positions are 
or will be located in designated priority countries for nonemergency 
food aid programs (including 19 in sub-Saharan Africa and 1 in 
Guatemala). As of August 2009, all 21 of the new positions had been 
established, of which 9 had been filled.[Footnote 18] Figure 4 shows 
the location of the new positions, including those filled, in FFP's 20 
priority countries for nonemergency food aid. 

Figure 4: FFP Priority Countries and Locations of New Positions for the 
Monitoring of Nonemergency Food Aid Programs: 

[Refer to PDF for image: 3 maps] 

The map utilizes shading to depict the following: 

20 priority countries for nonemergency food aid programs: 
Afghanistan; 
Bangladesh; 
Burkina Faso; 
Burundi;
Chad; 
Democratic Republic of Congo; 
Ethiopia; 
Guatemala; 
Haiti; 
Liberia; 
Madagascar; 
Malawi; 
Mali; 
Mauritania; 
Mozambique; 
Niger; 
Sierra Leone; 
Sudan; 
Uganda; 
Zambia; 

New Food for Peace-funded positions, Filled: 
Burundi; 
Madagascar (3); 
Mail; 
Malawi; 
Niger; 
Southern Africa Regional Office; 
Zambia. 

New Food for Peace-funded positions, To be filled: 
Burkina Faso (2); 
Democratic Republic of Congo; 
Guatemala; 
Liberia; 
Madagascar; 
Niger; 
Sierra Leone (2); 
Sudan. 

Sources: GAO analysis of USAID data; Map Resources (map). 

[End of figure] 

We found that USAID does not have a systematic process in place to 
determine appropriate staffing levels for effective monitoring and 
evaluation. According to USAID officials, creating new positions in 
field missions where FFP has lacked staff was the primary consideration 
for their allocation. Hence, 17 of the new positions are in countries 
where USAID runs nonemergency food aid programs but has lacked FFP 
program management staff to date, namely: Burkina Faso, Burundi, 
Liberia, Madagascar, Malawi, Mali, Mauritania, Niger, Sierra Leone, and 
Zambia.[Footnote 19] The remaining 4 new positions augment existing FFP 
program management staff in the Democratic Republic of Congo, 
Guatemala, Sudan, and the Southern Africa Regional Office in Pretoria. 

Information Technology System: 

USAID plans to replace its current Food for Peace Information System 
with a new Food for Peace Management System (FFPMS) that it expects 
will address several of FFP's current information technology issues, 
including the collection, tracking, and interfacing of food aid program 
data. According to USAID's December 2008 report to Congress, the new 
FFPMS is expected to involve collaboration and testing with users in 
each business area in which the system is used. These areas include 
budgeting, proposal, document management, commodity purchase, commodity 
booking and shipping, tracking, monitoring and evaluation, reporting, 
closeout, and interface with internal and external information systems. 
In addition, USAID officials explained that, unlike FFPIS, FFPMS is 
intended to be an all-inclusive system that will fully capture the 
movement of U.S. food aid commodities, used for both emergency and 
nonemergency purposes, from the initial transfer authorization to the 
commodity's final distribution site. FFPMS is also expected to (1) 
capture implementing partners' data on Food for Peace resources, 
including revenues from monetization transactions as well as costs 
incurred; (2) allow FFP to aggregate information for its implementing 
partners' annual results reports; and (3) have the capability to 
interface with USAID internal systems--such as USAID's core financial 
system, known as Phoenix, which captures all USAID transactions--as 
well as with external agency information systems with which FFP 
conducts business to allow the collection and sharing of information 
among and between agencies. With systematic and electronic collection 
of information, USAID anticipates being able to monitor the relative 
efficiency of its nonemergency food aid programs and adequately monitor 
the degree to which monetization revenues cover costs. 

USAID is in the initial stages of this planned information systems 
upgrade. In June 2009, USAID awarded a contract for $637,000 to develop 
a management strategy and framework for implementing FFPMS by the end 
of fiscal year 2010. The contractor's responsibilities, as defined in 
the contract, are to revalidate assumptions made within the 2005 
requirements and architecture diagram;[Footnote 20] revalidate FFP's 
technical environment; and update the requirements, documents, and 
architectural design. The contractor is also expected to make a 
recommendation to USAID by December 2009 for a commercial off-the-shelf 
software package that would best fit FFP's needs. 

A concept of operations document is a critical element in developing 
information system requirements and testing activities. However, USAID 
has not developed such a document for this initiative. This document is 
normally one of the first documents produced during a disciplined 
development effort because it describes system characteristics for a 
proposed system from a user's point of view.[Footnote 21] The key 
elements that should be included in the document are major system 
components, interfaces to external systems, and performance 
characteristics, such as speed and volume. A concept of operations 
document defines how an organization's day-to-day activities will be 
carried out to meet mission needs and includes high-level descriptions 
of information systems, their interrelationships, and information 
flows. It also describes the operations that must be performed, who 
must perform them, and where and how the operations will be carried 
out. 

Famine Early Warning Systems Network: 

To provide timely and rigorous early warning and vulnerability 
information on emerging food security issues, USAID is making a 
contribution to the FEWS NET fiscal year 2009 operational budget. FEWS 
NET monitors and analyzes relevant data, using multiple technologies 
such as satellites and field observations, to identify threats to food 
security and their impacts on livelihoods and markets. Chemonics, the 
private firm responsible for implementing FEWS NET along with several 
U.S. government agencies, currently maintains 20 national and 3 
regional field offices in sub-Saharan Africa, Central America, Haiti, 
and Afghanistan. Anticipating an increase in food insecurity in 
countries where there is no FEWS NET presence, Chemonics plans to 
implement three remote monitoring pilot projects--in Burundi, Pakistan, 
and Yemen--that are expected to be completed by the end of fiscal year 
2010.[Footnote 22] 

In fiscal year 2009, USAID is providing $7.6 million of the newly 
authorized Food for Peace Act funding to support FEWS NET. This funding 
is not designated to support the remote monitoring pilot projects but 
is a contribution to FEWS NET's operational budget of $20 million, with 
the remaining $12.4 million coming from the Development Assistance 
(DA), the Office of Foreign Disaster Assistance (OFDA), and other 
sources. 

Legal Restrictions on Usage of Food for Peace Act Funding Could Hamper 
USAID's Plans to Expand FFP's New Monitoring System and Improve 
Evaluations: 

Plans to Expand FFP's Computerized Monitoring System for Quality 
Assurance: 

Legal restrictions on the allowable uses of Food for Peace Act funding 
could hamper USAID's plans to expand FFP's new monitoring system for 
quality assurance. USAID plans to expand the use of Layers--a 
computerized system using personal data assistant (PDA) devices for 
monitoring the implementation and management of nonemergency food aid 
programs--from 3 pilot countries to 20 countries by the end of 2012, 
through its multiyear cooperative agreement with AED.[Footnote 23] The 
purpose of Layers is to help FFP staff responsible for the monitoring 
of food aid programs to collect data to assess the quality of 
implementing partner operations in the field. Layers can be used to 
monitor a wide range of programs and activities, including the 
conditions of storage in commodity warehouses and the quality of 
services delivered by FFP's implementing partners, such as food-for- 
work activities, provision of community health services, and maternal 
and child health and nutrition. Prior to 2009, Layers was piloted in 
Haiti, Ethiopia, and Madagascar. USAID estimates the one-time cost for 
the start-up to introduce Layers to a new country to be approximately 
$144,000. It began rolling out the system in Guatemala, Mali, and 
Uganda in 2009 and plans to do so in all the remaining priority 
countries by the end of 2012 as their multiyear assistance programs are 
approved. However, in May 2009, USAID's General Counsel determined that 
the Food for Peace Act precludes the agency from using the newly 
authorized funding for nonemergency food aid monitoring and evaluation 
for grants and cooperative agreements.[Footnote 24] Section 3012(f)(5) 
of the Food for Peace Act authorizes USAID to enter into personal 
services contracts to carry out monitoring and evaluation but does not 
refer to grants or cooperative agreements.[Footnote 25] As a result, 
FFP cannot move forward with its original plans to use Food for Peace 
Act funding to expand the implementation of Layers and will instead 
need to use other sources of funding that could be made available for 
this purpose.[Footnote 26] According to FFP officials, the current plan 
is to use other funding streams, such as the DA account, to fund the 
costs of the start-up of Layers and the first year of the system's 
operation. However, these officials noted that due to competing 
priorities for DA, it cannot be relied upon as a stable source of 
funding. 

Plans to Improve the Quality of Evaluations: 

USAID plans to improve the quality of evaluations through a number of 
ongoing and future studies. Using $5,000 of the newly authorized 
funding for monitoring and evaluation, the agency has signed a contract 
with Michigan State University for a study of monetization that is 
expected to be available for release in December 2009. It has also 
commissioned several cross-cutting evaluations and related studies that 
are expected to improve the quality of assessments, targeting 
approaches, design, and evaluation of nonemergency food aid programs. 
For example, through a cooperative agreement with AED, USAID is funding 
the FANTA-2 project to initiate the following multiyear studies in 
2009: 

* a study of exit strategies in Bolivia, Honduras, and Kenya, which 
will produce recommendations for effective ways to sustain program 
benefits after U.S. nonemergency food aid programs have been phased 
out; 

* a study to identify key components of success in Preventing 
Malnutrition in Children Under 2 Approach (PM2A) programs in Guatemala 
and Burundi; 

* a study to improve the early warning and response capacity in FFP's 
multiyear assistance programs in Haiti, Niger, and Uganda; 

* an update of the 2002 Food Aid and Food Security Assessment, which 
will evaluate progress made under FFP's Strategic Plan for 2006-2010; 
and: 

* a study testing the efficacy and safety of emergency food products 
(EFP) that is co-funded by USAID's Bureau for Global Health and FFP. 

In addition to these cross-cutting evaluations and studies, FANTA-2 
prepared four food security country frameworks--for Bangladesh, Burkina 
Faso, Liberia, and Sierra Leone--in 2009.[Footnote 27] Food security 
country frameworks are assessments that provide country-specific 
guidance to partners developing new MYAP proposals and describe how 
Food for Peace resources should be programmed and integrated with other 
U.S. government funds by defining objectives, approaches, regions, and 
institutional partnerships for effective use of U.S. government 
resources dedicated to reducing food insecurity. FFP estimates the cost 
of FANTA-2's work on the cross-cutting evaluations, related studies, 
and food security country frameworks in fiscal year 2009 to be $4.4 
million. 

To improve the quality of assessments and targeting approaches, FFP is 
also initiating a new cooperative agreement to identify best practices 
that will increase the efficiency and effectiveness of Food for Peace 
food aid program management and capacity building. The aim is to 
promote the broader adoption of best practices and collaboration among 
FFP's implementing partners. As of August 2009, FFP had issued a 
Request for Applications seeking to identify potential partners' 
interests, priorities, and capacities and estimates the cost of the 
award at $3.5 million. 

However, USAID has determined that the Food for Peace Act precludes the 
agency from using the newly authorized funding for monitoring and 
evaluation of nonemergency food aid for grants and cooperative 
agreements. As a result, the completion of the multiyear cross-cutting 
evaluations, as well as future food security country frameworks and 
"best practices" assessments is not assured. FFP's current plan is to 
fund these studies from the DA account. 

USAID Is Using Other Funding to Improve Assessments Conducted Prior to 
Approving U.S. Food Aid Programs in Recipient Countries but Faces 
Implementation Challenges: 

In August 2008, USAID launched a 3-year Bellmon Estimation for Title II 
(BEST) pilot project intended to improve the market analysis required 
before food aid programs are approved in recipient countries--commonly 
referred to as the Bellmon analysis.[Footnote 28] The project is funded 
through resources other than the newly authorized Food for Peace Act 
funding for monitoring and evaluation of nonemergency food aid. It aims 
to improve how nonemergency food aid data are collected and analyzed, 
to determine the appropriate level of food aid, as well as the 
appropriate commodities that can be distributed and monetized in each 
country. 

The pilot project reduces the potential for conflict of interest 
because the organization that does the analysis is no longer the same 
organization whose interest is directly affected by the outcome of the 
analysis. In the past, USAID's implementing partners for nonemergency 
food aid distribution or monetization conducted and submitted the 
Bellmon analysis to USAID. However, the outcome of the analysis 
affected whether they could conduct monetization and, if so, how much 
food aid they could monetize--which, in turn, directly affected their 
operating budgets. Under the pilot project, USAID will have one 
organization, Fintrac, carry out the initial market analysis for all 20 
of USAID's priority countries. According to Fintrac, having one 
organization do the analysis for all countries could help to improve 
consistency in analytical methodology, reporting structure, and format. 

BEST is still in its early stage of implementation. It encountered some 
difficulties in its first year of implementation and has yet to address 
some additional methodological challenges. One key challenge is 
ensuring the quality and consistency of the analysis, which can be 
impaired by a lack of reliable and up-to-date data. For example, 
Fintrac was able to use data from a recent livelihoods survey done by 
the World Food Program and FEWS NET in the Malawi assessment but had to 
conduct additional research and trips to Guatemala because it had no 
comparable updated data for that country. In addition, Fintrac 
officials told us that they have not developed quantifiable measures 
and thresholds agreed upon by USAID and its implementing partners to 
help guide USAID's programming decisions on levels of monetization. 
Without quantifiable measures and thresholds, implementing partners 
could challenge USAID's determination of monetization levels in a 
country. Finally, the law requires that the distribution of commodities 
will not result in a substantial disincentive or interference with 
domestic production or marketing in a recipient country--but the 
information Fintrac uses for its analysis, such as the historic levels 
of monetization relative to a country's imports or production, does not 
directly answer the question of whether U.S. food aid will cause such 
disincentives. USAID and Fintrac officials acknowledged that additional 
research is needed in this area. As of July 2009, Fintrac had completed 
analyses for 7 priority countries: Burundi, Ethiopia, Guatemala, 
Madagascar, Malawi, Sudan, and Uganda, with plans to complete analyses 
for the remaining 13 priority countries through 2011. 

USAID's Monitoring and Evaluation of Nonemergency Food Aid Are Not 
Fully Integrated with Program Management: 

USAID's monitoring and evaluation of its nonemergency food aid programs 
are consistent to varying degrees with some of the principles 
established by AEA to integrate evaluation into program management. We 
found that USAID's actions were (1) generally consistent with the AEA 
principles for policies and procedures and for independence; (2) 
partially consistent with the principles for scope and coverage, 
dissemination of results, professional competence, and resources; and 
(3) not consistent with the principles for monitoring and evaluation 
plans. 

USAID's Actions Were Generally Consistent with the AEA Principles for 
Policies and Procedures and for Independence: 

FFP Has Issued Policies and Procedures for Monitoring and Evaluation: 

[Side bar: 
Policies and Procedures: 
Principles: 
* Agency should publish policies and procedures and adopt quality 
standards to guide the conduct of evaluations. 
* Policies and procedures should identify the kinds of evaluations to 
be performed and the criteria and administrative steps for developing 
evaluation plans and setting priorities; selecting the appropriate 
evaluation approaches and methods to be used; consulting subject matter 
experts; ensuring the quality of the evaluation products; publishing 
evaluation reports; and ensuring the independence of the evaluation 
function. 

Source: AEA, An Evaluation Roadmap for a More Effective Government. 
End of sidebar] 

FFP has established policies and procedures for monitoring and 
evaluation that are contained primarily in several information 
bulletins for staff and implementing partners.[Footnote 29] Guidance 
can also be found in the FFP Process and Procedures Manual, which 
outlines field and headquarters staff's roles and responsibilities for 
monitoring and evaluation. In addition, FFP has issued a series of 
technical papers by FANTA-2 that discuss the methodological standards 
and techniques that can be used for many aspects of monitoring and 
evaluation, such as statistical sampling for surveys of beneficiaries 
and preparing survey questions for key indicators. For example, one 
technical paper discusses in detail a key FFP indicator, household 
dietary diversity, and explains how to collect data through a 
questionnaire, tabulate the responses, and set appropriate targets to 
track progress over time. FFP's policies and procedures, in conjunction 
with USAID's official guidance, cover the types of evaluations that can 
be performed, methodologies that can be employed, ways to plan 
evaluations, the means to disseminate studies, and the independence of 
the staff performing the evaluations. 

USAID's new central evaluation office collaborated with State's Office 
of the Director of Foreign Assistance (State/F) to develop evaluation 
guidelines and standards that were placed on the United States Foreign 
Assistance Web site in March 2009. These standards and guidance 
elaborate on the official USAID and FFP guidance, and they specifically 
address some elements that are not directly addressed in the official 
guidance, most notably the standards for quality. Collectively, the FFP-
specific, USAID-wide, and Foreign Assistance policies and procedures 
are generally consistent with the AEA principles. 

FFP Requires External Evaluations to Help Ensure Independence: 

[Sidebar: 
Independence: 
Principle: Evaluation managers should be independent with respect to 
the design, conduct, and results of their evaluation studies. 
Source: AEA, An Evaluation Roadmap for a More Effective Government. 
[End of sidebar] 

To help ensure independence as outlined in the AEA principle, FFP 
requires that each multiyear assistance program's final evaluation be 
performed by external evaluators. It also relies on external entities 
to perform cross-cutting evaluations using the Food for Peace Act and 
other funding sources.[Footnote 30] 

Independence in the context of final evaluations for FFP multiyear 
assistance programs means that the key evaluation experts, the team 
leader, and sectoral leads, should be external consultants hired by the 
project. Our review of the 16 final evaluations that we found on 
USAID's Development Experience Clearinghouse (DEC) Web site determined 
that external consultants were generally listed as the prime 
authors.[Footnote 31] In some instances, the external evaluators were 
affiliated with a particular consulting firm and, in other instances, 
several external evaluators formed a team to perform the final 
evaluation. 

In addition, the cross-cutting evaluations and other FFP-funded studies 
are to be conducted by FANTA-2 and external consultants. FANTA-2 is 
part of AED, a nonprofit organization that cites integrity as one of 
its core values.[Footnote 32] Our review of selected task orders, 
signed and made available by FANTA-2, found them in line with the AEA 
principle for independence. For example, not only did its task order 
for a subrecipient of a new 3-year cross-cutting evaluation describe 
the project's phases, activities, and methodologies, but it also 
clearly specified that the subrecipient's staff would develop 
preliminary conclusions based on the collected data, draft interim 
reports, and submit a final report to USAID.[Footnote 33] 

USAID's Actions Were Partially Consistent with the AEA Principles for 
Scope and Coverage, Dissemination of Results, Professional Competence, 
and Resources: 

USAID Requires Few Cross-Cutting Evaluations: 

[Sidebar: 
Scope and Coverage: 
Principles: 
* Evaluation should span the life cycle of programs and policies, 
making it integral to planning, developing, and managing government 
programs at all stages of their development. 
* In general, federal programs and policies should be subject to 
evaluation, which is a responsible and necessary part of good 
management. 
Source: AEA, An Evaluation Roadmap for a More Effective Government. 
End of sidebar] 

In line with the AEA principles, FFP requires that all of its MYAPs be 
monitored and evaluated at all stages of their life cycle--its guidance 
specifies the activities that are required from proposal writing 
onward. The implementing partners must: 

* perform a baseline survey at the beginning of each MYAP; 

* submit annual reports, including annual results reports that present 
performance indicators and a narrative regarding progress made; 

* submit annual questionnaires that respond to questions about FFP's 
food security indicators; and: 

* submit midterm and final project evaluations. 

In addition, USAID project officers must review the evaluation plans 
and final reports. Finally, the final evaluation must include 
questionnaires that are comparable to the baseline surveys, along with 
information about sample design, universe, methods of analysis, and 
sampling frame. 

During our fieldwork in Bangladesh and Haiti, we saw several examples 
of FFP's implementing partners following these directives. For example, 
we visited a project in Bangladesh that had collected baseline data on 
a wide range of indicators that measured aspects of land ownership, 
height and weight-for-age measures, maternal and child health 
practices, access to water and sanitation, household food security, and 
disaster preparedness. In addition, we visited a project in Haiti that 
had produced a quarterly report including targets for more than 50 
measures, along with data on whether or not those targets had been 
achieved, and reasons why some targets had not been met. 

FFP's required monitoring and evaluation of the multiyear programs 
focus on whether the programs are achieving their expected outcomes-- 
but they do not assess program impacts, or the effectiveness and 
efficiency, of particular approaches. To achieve this, FFP would need 
to conduct periodic, cross-cutting evaluations that compare what 
actually happened with what would have happened in the absence of the 
program or activity.[Footnote 34] While FFP has commissioned some of 
these studies, according to USAID officials, it has no requirements to 
do so[Footnote 35] and has conducted only a relatively small number 
over the last 5 years.[Footnote 36] In contrast, FFP required every one 
of more than 100 multiyear projects concluded over the last 5 years to 
be evaluated for outcomes. 

Implementing Partner Evaluation Results Are Not Routinely and Widely 
Disseminated: 

[Sidebar: 
Dissemination of Results: 
Principles: 
* The results of all evaluations related to public accountability 
should be made available publicly and in a timely manner. 
* Evaluations should be easily accessible through the Internet with 
user friendly search and retrieval technologies. 
* Evaluations of promising and effective program practices should be 
systematically and broadly disseminated to potential users. 
Source: AEA, An Evaluation Roadmap for a More Effective Government. 
End of sidebar] 

Although USAID policy requires final evaluations of its nonemergency 
food aid programs to be posted on a special Web site known as DEC, the 
implementing partners responsible for these evaluations often do not 
adhere to this requirement. The DEC Web site is accessible to the 
general public and contains both simple and more advanced search 
engines. For example, a user interested in Food for Peace nonemergency 
programs could perform a search for "agriculture," "maternal and child 
health," or "food security" final evaluations conducted in recent 
years. A user could also search for final evaluations sponsored by FFP. 
However, FFP officials told us that the MYAP implementing partners 
often do not submit their final evaluations to DEC, even though this is 
a program requirement. According to these officials, there is no 
penalty if the implementing partners do not submit their reports to 
DEC. We were unable to find in DEC a final evaluation of an integrated 
food security program that FFP staff in Bangladesh provided to us in 
hard copy. In addition, more than 100 FFP-sponsored final evaluations 
of multiyear programs should have been conducted from 2004 to 2009, 
according to FFP staff, but our search in DEC found only 16. 

Professional Competence Varies among the Different Groups of Staff Who 
Monitor and Evaluate FFP Programs: 

[Sidebar: 
Professional Competence: 
Principles: 
* Evaluations should be performed by professionals with evaluation 
training and experience appropriate to the evaluation activity in which 
they are engaged. 
* Federal agencies should be encouraged to recognize the 
multidisciplinary nature of evaluation and assure that the diversity of 
disciplines is appropriately represented in both internal and 
independent consulting teams. 
Source: AEA, An Evaluation Roadmap for a More Effective Government. 
End of sidebar] 

Although the entities that USAID has selected to perform cross-cutting 
evaluations have staff with appropriate professional training and 
experience, questions have been raised about the technical skills of 
some staff that perform monitoring and evaluation for the implementing 
partners. As FFP does not have reliable data on the number of its field 
staff that perform monitoring and evaluation functions, and the 
specific skills and competencies of those staff, it is not possible to 
determine their overall competence. In addition, as FFP's ability to 
hire monitoring and evaluation specialists in Washington, D.C., is 
limited, and as USAID has only recently reestablished its central 
evaluation unit, there has been an absence of skilled evaluators in 
headquarters that could assist FFP's officers in the field and 
implementing partners' staff. 

FFP's Partners Have Training and Experience Consistent with the AEA 
Principles: 

The partner organizations that FFP relies on to provide technical 
assistance and support and to conduct cross-cutting evaluations have 
training and expertise that are consistent with the AEA principles for 
professional competence. The AED FANTA-2 project, with which USAID has 
a cooperative agreement for various types of technical assistance, 
employs staff experienced in monitoring and evaluation, with 
backgrounds in economics, sociology, nutrition, anthropology, public 
health, and epidemiology. In addition, FANTA-2 has partnered with 
universities and research institutes, such as Tufts University and the 
International Food Policy Research Institute, for particular 
evaluations and other research projects. 

Concerns Have Been Raised about Certain Aspects of Implementing 
Partners' Monitoring and Evaluation Skills: 

FFP's implementing partners' staff perform important monitoring and 
evaluation functions, such as gathering data on performance indicators 
and tracking these indicators. Yet FFP officials in Washington, D.C., 
and in the field, as well as some external evaluators, have expressed 
concerns about the competency of these staff to perform certain aspects 
of monitoring and evaluation, as follows: 

* Staff can lack key evaluation skills. Implementing partners' staff 
can lack key evaluation skills, according to an AED employee who 
provided training and support to FFP's implementing partners and 
several external evaluators. According to the AED employee, 
implementing partners' staff could execute research designs adequately 
but often were not capable of designing the research themselves, or of 
taking responsibility for key aspects of data collection, such as 
developing surveys or selecting statistical samples. FFP officials 
stated that they recognize the need to build capacity among the 
implementing partners, and are providing monitoring and evaluation 
training in the field for these FFP and implementing partner staff. 

* Difficulties in hiring and retaining qualified staff. Adequacy of 
staffing levels and staff turnover have been cited as a concern in some 
countries. For example, the implementing partners in Haiti told us that 
it was difficult to attract and retain technically qualified staff. 
They stated that Haiti has only a limited supply of people with the 
appropriate skills and training for monitoring and evaluation. To 
address the shortage, one implementing partner recently hired a new 
monitoring and evaluation program officer from an African country who 
had appropriate methodological skills but did not speak either French 
or Creole, Haiti's official languages, at the time of his arrival in- 
country. 

FFP Lacks Data on the Monitoring and Evaluation Competencies of Its 
Field Staff: 

FFP does not have the data necessary to assure that it has enough staff 
with the right skills. According to FFP officials, USAID does not 
collect an inventory of its staff's skills and abilities,[Footnote 37] 
which would help the agency to determine whether it has staff with 
sufficient training and skills in monitoring and evaluation. 
Furthermore, neither USAID nor FFP have data that can indicate which 
staff spend the majority of their time on monitoring and evaluation, or 
how much staff time is spent on various monitoring and evaluation 
activities, such as checking on warehouses, advising implementing 
partners on performance indicators, or participating in evaluation 
designs. While FFP attempted to gather some data for us on its field 
staff who perform monitoring and evaluation, these data were not 
reliable. 

In Haiti, for example, FFP officials initially told us that nine staff 
acted as field monitors. However, we found that this number included 
staff members who did not spend much of their time involved in routine 
monitoring and evaluation, such as the FFP officer, his deputy, and two 
secretaries. Of the FFP staff in country, only four appeared to spend 
substantial parts of their time on specific monitoring activities, such 
as performing routine checks of warehouses or health clinics and giving 
implementing partners advice on indicators. 

One reason that it is difficult for FFP to determine which staff spend 
the majority of their time on monitoring is that definitions of this 
function vary. For example, while a recent FFP bulletin defined 
monitoring as the regular tracking of program implementation, FFP 
guidance to field staff defined monitoring more broadly, to include 
activities such as reviewing contracts and ensuring that financial 
audits are performed.[Footnote 38] If the broader definition of 
monitoring is used, more staff members can be classified as performing 
this function, but then it is not possible to determine which of those 
staff are spending the majority of their time on monitoring program 
inputs and program implementation. 

FFP Lacks Staff in Headquarters Who Are Skilled in Monitoring and 
Evaluation: 

AEA has recommended that to support evaluations agencies either 
establish centers of evaluation or appoint evaluation coordinators for 
individual programs that conduct their own evaluation. According to FFP 
officials, FFP has been unable to hire monitoring and evaluation staff 
at headquarters to assist and support its officers in the field. FFP 
officials attributed this to restrictions in the Food for Peace Act, 
which precludes them from hiring any staff in Washington, D.C., to 
monitor and evaluate nonemergency food aid programs.[Footnote 39] 
Moreover, as USAID has only recently reestablished its central 
evaluation unit,[Footnote 40] FFP has not been able to seek advice and 
guidance from professional, experienced evaluators at headquarters, or 
to ask them to perform evaluations of FFP's programs. 

Greater Resources Are Now Available for Monitoring and Evaluation, but 
Their Sufficiency Is Difficult to Determine Absent a Needs Assessment: 

[Sidebar: 
Resources: 
Principle: 
Sufficient resources should be made available for evaluation, including 
stable annually recurring sources of funds and special one-time funds 
for evaluation projects of interest to executive branch and 
congressional policymakers. 
Source: AEA, An Evaluation Roadmap for a More Effective Government. 
End of sidebar] 

The Food for Peace Act provides FFP with a stable source of new funding 
of up to $22 million annually for each of the fiscal years 2009 through 
2012 to improve, monitor, and evaluate the effectiveness and efficiency 
of nonemergency food aid programs. In addition, FFP has access to other 
funding streams, such as DA funds, which have recently increased. As a 
result, FFP officials told us that they currently have more resources 
to devote to monitoring and evaluation than in the past. 

Although availability of the newly authorized funding for monitoring 
and evaluation as well as funding from other sources is generally 
consistent with the AEA principle on resources, it is not possible to 
determine if funding is sufficient because FFP has not systematically 
determined its research needs, prioritized those needs, or estimated 
the costs associated with them. In the course of our audit work, we 
identified several areas where the current resource levels might not be 
sufficient. For example, experts we interviewed provided examples of 
potential evaluations or studies that could help improve FFP's 
programs. However, according to USAID officials, FFP has not undertaken 
the formal exercise of compiling a list of these studies, or 
prioritizing its needs for them. In addition, FANTA-2 staff who review 
many of the implementing partners' multiyear plans for evaluation 
raised concerns that FFP does not sufficiently ensure that these 
evaluations are adequately funded. However, FFP has not reviewed or 
estimated the amounts its implementing partners are spending on 
monitoring and evaluating their multiyear projects because, according 
to FFP, that would be burdensome as it would require a review of each 
project's contract.[Footnote 41] Without such an assessment, it is not 
possible to determine whether sufficient resources are being spent on 
the required final evaluations. 

USAID's Actions Were Not Consistent with the AEA Principles for 
Monitoring and Evaluation Plans: 

[Side bar: 
Evaluation Plans: 
Principles: 
* Agency should require its major program components to prepare annual 
and multiyear evaluation plans of the studies and other evaluation 
activities that it will undertake. The plans should be updated 
annually. 
* The planning should take into account the needs of evaluation results 
for informing program budgeting, reauthorization, agency strategic 
plans, ongoing program development, and management and responses to 
critical issues. 
Source: AEA, An Evaluation Roadmap for a More Effective Government. 
End of sidebar] 

According to FFP officials, FFP plans some of its monitoring and 
evaluation activities,[Footnote 42] but lacks an overall plan that 
integrates its monitoring and evaluation with program management. FFP 
officials reported that they make decisions about monitoring and 
evaluation based on discussion and a thorough consideration of options 
but have not undertaken the rigorous planning process that is 
recommended by AEA. Therefore, FFP does not have an integrated plan for 
all of its oversight activities to facilitate strategic planning, 
budgeting, and program management, and to indicate how it will use 
monitoring and evaluation results to improve program management. 
Moreover, FFP does not have a mechanism to link resources to needs 
based on risk, according to USAID officials. As a result, when FFP 
allocates resources for monitoring and evaluation, it does not take 
into account factors such as the country context, vulnerability to 
fraud, waste, and abuse, program efficiency, and critical issues. As we 
noted earlier, we cannot determine whether FFP has sufficient resources 
because this type of planning has not been conducted, which means that 
FFP's monitoring and evaluation needs have not been prioritized and 
compared against available resources. 

A senior FFP official told us that he could see the potential value of 
such a plan but wondered whether FFP--which has a staff of about 60 
people and which he characterized as being small in relation to other 
USAID units--could support such an effort. However, FFP appears to have 
the resources for such a planning effort, as the Food for Peace Act has 
provided it with up to $88 million to improve, monitor, and evaluate 
the effectiveness and efficiency of nonemergency food aid programs over 
a 4-year period, and has access to other resources that can be used for 
monitoring and evaluation, such as DA funds.[Footnote 43] 

Conclusions: 

Nonemergency food aid programs are an essential component of the U.S. 
strategy to reduce global food insecurity, and Congress has authorized 
new funding for USAID to improve its oversight of these programs. In 
response, USAID's FFP has initiated actions over the past year to 
strengthen its monitoring and evaluation of food assistance. However, 
weak planning could impede these efforts, including the upgrade of 
FFP's information technology system. In addition, USAID has yet to 
determine how it intends to assure stable resources, within existing 
authorities, to fully implement all the actions planned by FFP to 
improve monitoring and evaluation of nonemergency food aid. As a 
result, planned actions that are essential to effective program 
management, such as the expansion of a computerized monitoring system 
for food aid programs and cross-cutting evaluations, could be 
jeopardized. Finally, FFP lacks an integrated and comprehensive 
monitoring and evaluation plan that links resources to needs based on 
risk. In the absence of such a plan, it is difficult to determine 
whether USAID is allocating resources for monitoring and evaluation to 
its priority needs, such as program efficiency, critical issues, and 
programs vulnerable to fraud, waste, and abuse. 

Recommendations for Executive Action: 

To facilitate the implementation of ongoing efforts to improve 
monitoring and evaluation and ensure effective program management of 
nonemergency food aid, we recommend that the Administrator of USAID 
take the following two actions: 

* Develop a concept of operations document to help reduce the risks 
associated with upgrading FFP's information technology system. Such a 
document should adhere to industry best practices and include key 
elements such as major system components, interfaces to external 
systems, and performance characteristics. It would also describe system 
characteristics for a proposed system from a user's point of view and 
include high-level descriptions of information systems, their 
interrelationships, and information flows. 

* Develop an integrated monitoring and evaluation plan to ensure that 
the results of its oversight activities are used for effective 
management of nonemergency food aid programs. Such a plan should, among 
other things, (1) link monitoring and evaluation to key USAID and FFP 
goals; (2) establish a systematic process for determining appropriate 
budget levels and staff resources for monitoring and evaluation based 
on an analysis of the nature, scope, and size of the programs, and the 
unique conditions presented, in the recipient countries; (3) examine 
all available options for funding monitoring and evaluation at 
headquarters and in recipient countries; and (4) establish time frames 
for implementing and evaluating the plan. 

Agency Comments and Our Evaluation: 

USAID provided written comments on a draft of this report. We have 
reprinted these comments in appendix IV, along with our responses. 
Additionally, USAID provided technical comments on a draft of this 
report, which we have addressed as appropriate. 

USAID concurred with our first recommendation to develop a concept of 
operations document to help reduce the risks associated with upgrading 
FFP's information technology system. USAID disagreed with the second 
recommendation, asserting that we did not adequately take into account 
documents that detail its planning processes. We considered all the 
documents that USAID cited and found that they did not constitute 
plans, but only provided guidance for planning. Furthermore, the 
weaknesses we identified in FFP's monitoring and evaluation efforts, 
including in the areas of resources and professional competence, 
demonstrate the need for a comprehensive monitoring and evaluation 
plan. 

USAID acknowledged that it does not have an integrated monitoring and 
evaluation plan, but said that it does not see the benefits of 
developing it since the elements of such a plan exist in some form 
already. USAID noted that we did not mention FFP's Strategic Plan for 
2006-2010 and a related document, which link monitoring and evaluation 
to agency goals and lay out performance indicators. We considered the 
strategic plan irrelevant to our analysis because it was prepared in 
2005, prior to the passage of the 2008 Food for Peace Act, and covers 
only some aspects of monitoring. USAID also stated that its planned 
studies form part of an agency agenda to inform program management. We 
reported on FFP's planned studies but noted that, because funding for 
these studies is not assured due to legal restrictions, USAID may need 
to find alternative funding sources. Moreover, USAID officials reported 
that they did not use almost $9 million of last year's available funds 
authorized by the 2008 Food for Peace Act for oversight of nonemergency 
food aid. We consider these to be compelling reasons for USAID to 
develop an integrated monitoring and evaluation plan. 

In terms of staffing, FFP officials said that their first priority is 
to place staff in all the countries where they have nonemergency food 
aid programs. FFP acknowledged that it does not have data on field 
staff that perform monitoring and evaluation functions, but said that 
such data would be costly to collect, and that we had not made the case 
for collecting them. However, we found that FFP currently does not know 
what percentage of time its staff spend on monitoring and evaluation 
and what specific monitoring and evaluation functions they perform. We 
believe that because FFP had a substantial new funding authority for 
oversight activities, it should invest in a survey of staff skills and 
an assessment of monitoring and evaluation needs, which would provide 
FFP with essential data for future staffing decisions. We believe that 
this basis for our recommendation for an integrated monitoring and 
evaluation plan remains valid. 

We are sending copies of this report to interested Members of Congress 
and the Administrator of USAID. The report is also available at no 
charge on the GAO Web site at [hyperlink, http://www.gao.gov]. 

If you or your staffs have any questions about this report, please 
contact me at (202) 512-9601 or melitot@gao.gov. Contact points for our 
Office of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made major contributions to 
this report are listed in appendix V. 

Signed by: 

Thomas Melito: 
Director, International Affairs and Trade: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

In passing the Food for Peace Act in 2008, Congress required the 
Comptroller General of the United States to review the U.S. Agency for 
International Development's (USAID) Report to Congress on the agency's 
efforts to strengthen the monitoring and evaluation of its nonemergency 
food aid programs and provide recommendations for improvement. To 
address this congressional mandate, we reviewed (1) USAID's plans and 
actions to improve its monitoring and evaluation of nonemergency food 
aid programs provided under the Food for Peace Act and (2) the extent 
to which USAID has integrated its monitoring and evaluation of 
nonemergency food aid with program management, particularly with regard 
to established principles. In this review, we focused on oversight 
practices of USAID's Office of Food for Peace (FFP), not of the agency 
as a whole. 

To address USAID's plans to improve the monitoring and evaluation of 
nonemergency food aid programs using the funds provided under the Food 
for Peace Act, we reviewed the list of activities that the agency 
reported to Congress in December 2008. This included plans to make 
improvements in: 

* the number of field staff monitoring food aid programs, 

* computerized systems for monitoring food aid programs, 

* the market analysis required before food aid programs are approved in 
recipient countries, 

* cross-cutting evaluations of nonemergency food aid projects, 

* FFP's information technology system, and: 

* the Famine Early Warning Systems Network (or FEWS NET). 

We interviewed knowledgeable USAID officials and partners responsible 
for implementing these plans and reviewed relevant documents, including 
contracts and plans. We also examined FFP data on the allocations of 
the Food for Peace Act funding for oversight of nonemergency food aid 
in fiscal year 2009. Based on interviews, and our analysis of the 
available documents, we determined that the data were sufficiently 
reliable to indicate the relative amounts allocated to each of the 
oversight activities, and to demonstrate that almost $9 million were 
not used in fiscal year 2009. 

In addition, we conducted fieldwork in Bangladesh and Haiti. We 
selected these two countries because they have the largest Food for 
Peace nonemergency food aid programs, but they differ in their 
approaches to monitoring and evaluation. Haiti has more staff working 
on these programs than does Bangladesh and uses the Layers computerized 
monitoring system, which Bangladesh does not. Haiti's staff spend 
relatively more time monitoring programs directly, while FFP in 
Bangladesh relies more on contract officers and implementing partners 
to perform its monitoring and evaluation. In addition, Haiti is in the 
Caribbean while Bangladesh is in South Asia, which provided a regional 
contrast. While we could not generalize from these two countries to all 
the countries that have Food for Peace programs, we gained useful 
insights into monitoring and evaluation from our fieldwork. 

To consider the extent to which USAID's monitoring and evaluation of 
nonemergency food aid are aligned with program management, we selected 
the American Evaluation Association's (AEA) principles for integrating 
evaluation into program management. We selected these principles as 
criteria because they are consistent with the goal set by the Secretary 
of State, who has ultimate responsibility for USAID, to ensure evidence-
based decisions in development programs. 

In this review, we combined the AEA standards for scope and for 
coverage because we found overlap between them. In addition, we did not 
assess one of the AEA standards on methods because that would have 
involved a review of studies that was beyond the scope of this report, 
and because that principle was less specific than those for the other 
elements, essentially noting that a mix of methods should be used, but 
that the actual methods used should depend upon the topic to be 
evaluated. 

The AEA principles differ in length and detail, with some including 
several parts and clauses, while others are more succinct. For the more 
detailed principles, we identified the sentences that summarized them 
at the highest level and used those as our operating criteria. For the 
sake of transparency, we published the exact principles we used in our 
report next to our discussion of them. 

To gather evidence to apply against the principles, we interviewed 
cognizant FFP officials in Washington, D.C., and in Bangladesh and 
Haiti. We also interviewed the Chief of USAID's Evaluation Office, 
senior staff at the Food and Nutrition Technical Assistance (FANTA-2) 
project of the Academy for Educational Development (AED), with which 
USAID has ongoing cooperative agreement, as well as experts 
knowledgeable about USAID's evaluation and monitoring. We reviewed 
USAID and FFP policy guidance and bulletins that discuss and describe 
evaluation. We reviewed the evaluations that we found on USAID's 
Development Experience Clearinghouse (DEC) Web site and that were given 
to us by USAID and its implementing partners. We also examined related 
monitoring and evaluation documents, such as performance plans and 
annual project reports, and documentation of the training USAID 
provides on monitoring and evaluation. 

We reviewed this evidence and compared it against each of the AEA 
principles to make a general, high-level determination on the broad 
degree to which FFP was consistent or inconsistent with them. In the 
course of this review, we recognized that FFP has its own guidance and 
practices, but that it follows the overall guidance and practices set 
by USAID for the agency as a while, and considered each source as 
appropriate. 

In addition to the AEA principles, we also considered GAO standards and 
prior GAO work, particularly on workforce planning; USAID guidance; and 
guidelines that the Institute of Electrical and Electronics Engineers 
(IEEE) has established for information technology. 

As USAID performs a wide range of activities to monitor and evaluate 
its Food for Peace program, we defined evaluation broadly for the 
purposes of this report. This definition includes evaluating inputs and 
implementation,[Footnote 44] outcomes, and impacts. We also considered 
other research studies that FFP had conducted to help manage its 
programs, such as prospective studies on the impacts of providing 
particular food commodities to particular markets, and analyses of 
individual countries' food strategies. In addition, we included the 
selection and tracking of performance indicators in our review because 
FFP and USAID define monitoring to include the tracking of these 
indicators during the life cycle of their programs, and use them in the 
final evaluations. 

We conducted this performance audit from March 2009 to September 2009, 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

[End of section] 

Appendix II: USAID Report to Congress: 

Note: This is USAID's Report to Congress dated December 31, 2008, as 
required by the Food for Peace Act. 

USAID: 
From The American People: 

December 31, 2008: 

The Honorable Tom Harkin: 
Chairman: 
Committee on Agriculture, Nutrition, and Forestry: 
U.S. Senate: 
Washington, DC 20510: 

Dear Mr. Chairman: 

Pursuant to Section 3012(f)(3) of the Food for Peace Act (Public Law 
480) dated June 18, 2008, we are pleased to submit the U.S. Agency for 
International Development's (USAID) report on efforts undertaken by the 
Administrator to conduct oversight of non-emergency food aid programs. 

If you have any questions concerning this report, please feel free to 
contact me. 

Sincerely, 

Signed by: 

[Illegible], for: 
Jeffrey J. Grieco: 
Assistant Administrator Bureau for Legislative and Public Affairs: 

Enclosure: a/s: 

Clearance Page For Letters to Congressmen Peterson, Harkin, Chambliss, 
Goodlatte, and Berman and Congresswoman Ros-Lehtinen, Regarding Report 
of Non-Emergency Food Aid Programs: 

Clearances: 
	
DCHA/FFP: Jeff Boms; cleared by email: Date 12/15/08. 

DCHA/FFP: Jonathan Dworken; cleared by email: Date 12/15/08. 

DCHA/PP	Susan Bradley [Signed by]: Date 12/15/08. 

AA/DCHA: Rob Jenkins [Signed by]: Date 12/15/08. 

LPA: Rick Marshall [Signed by]: Date 12/15/08. 

DCHA/FFP: Angela Stephens: AS: x2-0414: 12/15/2008 

P:\DCHA.SHARED\2009 Correspondence\FFP\Rpt to Congress re program 
oversight\FFP Program Oversight Report to Congress FINAL.doc. 

Report To Congress: 

Sec. 3012 of the Food, Conservation, and Energy Act of 2008 requires 
that not later than 180 days after the date of enactment of the Act, 
the Administrator of the U.S. Agency for International Development 
(USAID) submit to the appropriate committees of Congress a report on 
efforts undertaken by the Administrator to conduct oversight of non-
emergency programs under Title II of P.L. 480, the Food for Peace Act.
USAID is conducting oversight of its non-emergency food aid programs 
through: (1) a significant expansion of field monitors; (2) 
establishment of a three-year pilot project, the Bellmon Estimation for 
Title 11 (BEST) Project, which will conduct independent market analyses 
to ensure non-emergency program compliance with the Bellmon Amendment; 
(3) expansion of the Food and Nutrition Technical Assistance (FANTA) 
Project research to strengthen monitoring and evaluation for non-
emergency programs; (4) improving the quality of assessments and 
targeting approaches; and (5) upgrading the Office of Food for Peace 
(FFP) information technology system. 

1) Increase USAID Staff that Manage and Monitor Non-emergency Food Aid: 

Before the enactment of the 2008-2012 Farm Bill, FFP was precluded from 
using P.L. 480 Title II funds to hire personal services contractors 
(PSCs) primarily responsible for monitoring non-emergency programs. 
With this new authority, FFP has undertaken a thorough review of 
existing management and monitoring arrangements, and is expanding 
monitoring in approximately 10 countries with Title II non-emergency 
programs. Positions have already been advertised in three of the 
countries. 

2) Bellmon Estimation for Title II (BEST) Project: 

Prior to the approval of proposed food aid levels for monetization and 
distribution programs, P.L. 480 Title II (Section 403(A) A-D of P.L. 
480, as amended) requires that the United States Government (USG) make 
a positive Bellmon determination to ensure a) adequate storage 
faculties will be available in the recipient country at the time of the 
arrival of the commodities to prevent the spoilage or waste of the 
commodity, and b) the distribution and monetization of the commodity in 
the recipient country will not result in a substantial disincentive or 
interference with the domestic production or marketing in that country. 

In order to help USAID comply with the Bellmon Amendment and to make 
the Bellmon determination process more rigorous and consistent, FFP has 
established as of fiscal year 2009 a three-year pilot project, the 
Bellmon Estimation for Title II (BEST) Project, to conduct independent 
market analyses. Based on an established schedule, BEST will conduct an 
initial country situational study to develop a baseline market report 
(BMR). 

The BMR will include, but not be limited to, the following information: 

* Market and production system profile, summary; 

* Historical overview of the country's food aid program, covering 
USAID, USDA, WFP and other development, emergency and monetization food 
aid programs; 

* Distribution/logistics/storage profile; 

* Policy issues relevant to agriculture, markets and trade; and; 

* Product selection and analysis for key food security crops. 

In November 2008, USAID notified its Missions and FFP cooperating 
sponsors (CSs) about the BEST Project and CSs' continued obligation to 
contribute to market analyses and surveillance, and provided 
opportunity for them to comment on this new approach to compliance with 
the Bellmon Amendment. 

3) Food and Nutrition Technical Assistance (FANTA) Project FFP research 
activities, through FANTA, will include: 

* Food Security Programming Strategies: A solid understanding of 
national food security conditions is a crucial prerequisite to sound 
programming. In response to demand, FFP, through the FANTA Project, 
will support Missions and governments to plan and implement in-depth 
food security assessments as they relate to the programming of Title II 
non-emergency resources. Working with Missions, governments, and other 
stakeholders, FFP, through the FANTA Project, will apply assessment 
results to prepare food security strategies that define objectives, 
approaches, and institutional partnerships for effective use of Title 
II non-emergency resources to reduce food insecurity. 

* Exit Strategies: Factors that lead to success in sustaining program 
benefits after Title II non-emergency food assistance has ended will be 
examined, with the goal of deriving recommendations for effective Title 
II non-emergency exit strategies. 

* Preventive Malnutrition for Children Under Two Approach (PM2A): This 
study will identify key components of success in programs that utilize 
the PM2A approach, in order to improve the cost effectiveness, design 
and content of such programs. This study will include training and 
supporting CSs to adapt, design and implement PM2A, supporting 
formative research and baseline surveys/analyses, and performing 
operations research. 

* Layers: FFP monitoring of Title II non-emergency programs is an 
important mechanism for ensuring quality. FFP will use Layers, a 
computerized system for monitoring food aid programs, to improve 
monitoring speed and efficiency and the use of information. The 
innovative Layers approach is a computerized food aid program 
monitoring system (hardware/software solution) using handheld computers 
(PDAs). Layers helps USAID missions collect program data to assess the 
quality of implementing partner operations in the field. A wide variety 
of issues can he evaluated, from the conditions of storage in commodity 
warehouses to the quality of services delivered by Title II CSs in 
their maternal child and health nutrition activities, food for work, 
food for education, or other interventions. 

* Monitoring and Evaluation (M&E) Training: FFP will help CSs harmonize 
impact and output indicators, implement joint baseline assessments and 
coordinate other components of M&E systems as relates to Title II 
nonemergency programs. For example, in November 2008, two regional 
workshops in Africa were held for CS staff overseeing Title II non-
emergency programs. The workshops aimed to support CSs implementing new 
Title II programs by examining their results framework, harmonizing 
indicators, improving the usefulness and overall design of their M&E 
system, and reflecting on the design and planning of their baseline 
studies. Further M&E workshops arc planned. 

4) Improve the Quality of Assessments and Targeting Approaches: 

For more than 20 years, FFP has provided separate cooperative 
agreements in support of capacity building and technical assistance to 
Title II CSs. These agreements have addressed various aspects of Title 
II program management through workshops, training, assessments and 
specialty tools, along with support to key personnel. These 
achievements have been captured throughout the years. Under the new 
program oversight authority in the Farm Bill, FFP is now focused on the 
next phase of support to increase the effectiveness and efficiency of 
Title II food aid programs through the broader establishment and 
adoption of standard best practices across CSs. 

The principal objectives of the proposed agreement will be to identify 
best practices in Title 11 food aid program management and capacity 
building; design and undertake training in best practices in program 
management and capacity building; communicate and disseminate 
information on best practices in program management and capacity 
building; and strengthen the coordination and collaboration among 
current and prospective Title II CSs. 

In this regard, FFP has already issued a Request for Information 
(pursuant to FAR 15.201) to seek information from potential grantees or 
participants as to their interest, priorities and capacities to 
participate in or undertake a Leader/Associates Award, i.e., 
cooperative agreement, to increase the effectiveness and efficiency of 
Title II food aid programs. 

5) Food for Peace Information Technology System: 

The new Farm Bill provides USAID with the flexibility to use P.L. 480 
Title II resources to upgrade FFP's information technology systems in 
order to improve tracking, reporting and evaluation capabilities, 
including non-emergency programs. The new system will involve 
collaboration and testing with users in each business area in which the 
system is used-budgeting, proposal, document management, commodity 
purchase, commodity booking and shipping, tracking, monitoring and 
evaluation, reporting, closeout and interface. FFP is in the process of 
reviewing the functional components of the architectural design to 
ensure that they align with these requirements. It is expected that an 
award for the implementation of the new system will he made in early 
2009. 

[End of section] 

Appendix III Examples of Activities Supported with Nonemergency Food 
Aid: 

Figure 5 provides examples of activities we observed in Bangladesh and 
Haiti that are illustrative of the various activities supported with 
nonemergency food aid. 

Figure 5: Examples of Activities Supported with Nonemergency Food Aid: 

[Refer to PDF for image: 4 photographs] 

Direct distribution: 

Food provided in exchange for work; generally targeted seasonally and 
geographically to food insecure population groups. 

Photograph: Food-for-work: CRS Haiti provided temporary employment to 
234 laborers for 6 months for soil conservation and reforestation 
activities. 

Photograph: Food-for-work: For several years, CARE Bangladesh has been 
helping the Tangail district prepare for natural disasters through 
infrastructure activities, including raising homesteads to mitigate the 
effects of severe flooding on households. 

Monetized food aid: 

Proceeds from U.S. food aid sold locally support development projects, 
such as the rehabilitation and construction of new water points and 
mother and child health and nutrition programs. 

Photograph: Safe water and sanitation: Save the Children Bangladesh has 
installed a water point in the Barisal district to ensure household 
access to safe water. 

Photograph: Mother and child health and nutrition: World Vision Haiti 
provides pregnant and lactating women and children under the age of 2 
with monthly health checkups, immunizations, and counseling, at the 
Terre Casse clinic in Haiti’s Central Plateau region. 

Sources: GAO analysis; GAO and USAID (photos). 

[End of figure] 

[End of section] 

Appendix IV: Comments from USAID: 

Note: GAO comments supplementing those in the report text appear at the 
end of this appendix. 

USAID: 
From The American People: 
U.S. Agency for International Development: 
1300 Pennsylvania Avenue, NW: 
Washington, DC 20523: 
[hyperlink, http://www.usaid.gov] 

September 12, 2009: 

Mr. Thomas Melito: 
Director: 
International Affairs and Trade: 
U.S. Government Accountability Office: 
Washington, DC 20548: 

Dear Mr. Melito: 

I am pleased to provide the formal response of the U.S. Agency for 
International Development (USAID) to the GAO draft report 
"International Food Assistance: USAID is Taking Actions to Improve 
Monitoring and Evaluation of Nonemergency Food Aid, but Weaknesses in 
Planning Could Impede Efforts" (GAO-09-980). 

The enclosed USAID comments are provided for incorporation with this 
letter as an appendix to the final report. 

Thank you for the opportunity to respond to the GAO final report and 
for the courtesies extended by your staff in the conduct of this audit 
review. 

Sincerely, 

Signed by: 

Drew W. Luten: 
Acting Assistant Administrator: 
Bureau for Management: 

Enclosure: a/s: 

[End of letter] 

USAID Comments On GAO Draft Report 09-980: 

USAID's Office of Food for Peace (FFP) takes seriously its obligation 
to the federal government and the American people to ensure that Title 
II resources are used effectively and efficiently to achieve the best 
possible food security outcomes and that food aid programs continually 
learn from past experiences and improve their implementation. 

The GAO recommends that the USAID Administrator (1) prepare a concept 
of operations document (CONOPS) to reduce the risks associated with 
upgrading FFP's information technology system and (2) develop an 
integrated monitoring and evaluation plan that is linked to program 
goals, establishes a systematic process for determining budget and 
staffing levels, examines funding options, and sets implementation time 
frames. 

Related to recommendation (1) that FFP develop a CONOPS to reduce the 
risks associated with upgrading FFP's information technology system, 
under a current contract, FFP will receive a CONOPS that will be part 
of the recommendations deliverable that will be produced at the 
completion of the IT functional requirements gathering process. Through 
the functional requirements gathering, the process currently entails 
reviewing all of the Title 1 day-to-day activities which are carried 
out to meet mission needs, and assessing the interrelationships of all 
applicable information systems, and developing high-level descriptions 
for all business processes and workflow. Through the understanding and 
update of the current state of operations, redundant and manual 
processes can be identified and eliminated to provide a more efficient 
and effective process which then can be implemented through the new 
information system solution. The CONOPS will be utilized for the 
implementation phase of the system's development lifecycle. 

Related to recommendation (2), the report does not adequately take into 
account existing and on-going M&E processes currently carried out by 
USAID and FFP. As recognized in published FFP guidance, and as outlined 
in USAID's Automated Directives System (ADS), USAID and FFP already 
describe and outline key monitoring and evaluation (M&E) 
responsibilities of Title II Multi-Year Assistance Program (MYAP) 
awardees and potential awardees, providing additional detail to the 
existing policies, including, but not limited to the following: [See 
comment 1] 

1. ADS 200, Introduction to Program Policy, which provides an overview 
to the ADS Series 200. 

2. ADS 201, Planning, which explains how USAID manages for long-term 
results and development impact, retaining rigorous internal management 
and documentation standards while collaborating with Department of 
State/Office of the Director of Foreign Assistance and other U.S. 
Government agencies on implementing a new foreign assistance policy 
agenda and reporting system; it also summarizes the Foreign Assistance 
Framework and its standardized program structure. 

3. ADS 202, Achieving, which outlines the basic policy directives and 
required procedures used in achieving foreign assistance results with 
USAID-managed program and staff resources to help ensure that the 
development results that USAID seeks are obtained in a cost-effective 
and timely manner, consistent with applicable regulatory and 
accountability requirements. 

4. ADS 203, Assessing and Learning, which describes USAID practices and 
standards used to determine how well assistance objectives are 
achieving their intended results. Potential awardees are requested to 
consult ADS 203 closely for important information regarding the design 
of M&E plans. 

5. Evaluating Title II Development-oriented Multi-Year Assistance 
Projects at [hyperlink, 
http://www.fantaproject.org/downloads/pdfs/TN11_MYAP.pdf]. 

6. Monitoring and Evaluation Framework for Title II Development-
oriented Projects at [hyperlink, 
http://www.fantaproject.org/downloads/pdfs/TN10_MEFramework.pdf]. 

7. Title 11 Evaluation Scopes of Work at [hyperlink, 
http://www.fantaproject.org/publications/tn2.shtml]. 

8. (M&E) responsibilities of Title II Multi-Year Assistance Program 
awardees and potential awardees at [hyperlink, 
http://www.usaid.gov/our_work/humanitarian_assistance/ffp/ffpib.html]. 

In addition, in fiscal year (FY) 2008, FFP delivered 4-5 day M&E 
workshops to newly awarded MYAPs in order to improve the quality of 
Title II M&E data. The workshops help ensure that required indicators 
from FFP and USAID Missions are included in partner M&E plans and that 
data are collected in a consistent manner. In FY 2008, workshops were 
held in Mali and in Mozambique for 16 newly awarded MYAPs who improved 
their results frameworks, M&E plans, and the design of their baseline 
studies as a result of attending. USAID Mission staff who attended the 
workshops presented Mission indicators and had an opportunity to learn 
about the new Title II non-emergency programs in their country and 
about FFP M&E. Workshops have also been held in Ethiopia, Madagascar 
and Malawi Further M&E workshops are planned. 

The GAO recommends that USAID develop an integrated M&E plan that 
addresses four elements. All of those elements exist in some form 
already, and there is no evidence that there will be sufficient added 
value to integrating them all in a single, hybrid document that would 
contain a mix of a set of indicators, an evaluation plan, and a plan 
for the administration of monitoring and evaluation focused mostly on 
how to fund and staff M&E. 

For example, the existing FFP Strategic Plan and performance monitoring 
plan (PMP), which is not mentioned at all in the report, already links 
M&E to key USAID and FFP goals. The Strategic Plan discusses how FFP 
objectives fit into USAID goals and the PMP shows how FFP's indicators 
are linked to its objectives. The PMP lays out a set of indicators 
designed to measure the FFP objectives, explains the reasoning behind 
the selection of each indicator, and provides detailed information on 
indicator description, data collection methods, baselines and targets, 
and data quality assessment plans. [See comment 2] 

FFP's current evaluation plan contains a set of multi-year studies 
prioritized because they 1) build on previous investments (e.g. 
preventive malnutrition for under two[Footnote 45]), 2) are needed to 
operationalize strategic approaches under the Strategic Plan (e.g. FFP 
Trigger Indicators and Early Warning and Response Systems in Multi-Year 
Title II Assistance Programs Document[Footnote 464]), 3) take advantage 
of unique and time-bound opportunities (e.g. FFP is strengthening 
guidance for exit strategies working closely with Tufts University 
through FANTA-2), and 4) will form part of the analytic agenda to 
inform program management and the new strategy (e.g. FFP is now 
conducting a qualitative review of Title II non-emergency program 
activities). [See comment 3] 

In addition, referred to as Technical and Operational Performance 
Support (TOPS), FFP will soon provide a single award with the objective 
to increase the effectiveness and efficiency of the Title II food aid 
programs by improving the technical knowledge and operations and 
promoting food aid commodity program best practices through 
coordination and delivery of institutional strengthening and capacity 
building support for current Title II awardees and prospective awardees 
worldwide. 

Related to staffing, FFP's first priority is to ensure that there are 
in-country staff with Title II nonemergency program oversight 
responsibility in all countries that do not currently have them. This 
oversight responsibility includes M&E, but not exclusively. FFP does 
take into consideration the size and number of programs in the country 
and the difficulty of the conditions in determining where to place 
staff. [See comment 4] 

Field staff are hired based on an appropriate set of experience and 
skills to carry out their oversight duties, which include but are not 
restricted to M&E, and they are provided with training to gain and 
strengthen needed skills. While it is true that USAID does not have 
data on the numbers of field staff that have M&E competencies or an 
inventory of its staff's skills and abilities, the GAO has not made 
clear why this is necessary and what purpose it would serve, especially 
given the cost and effort to acquire this information and keep it 
updated. It would be nearly impossible to obtain accurate figures since 
M&E, especially monitoring, cannot and should not be disaggregated from 
other programmatic activities, especially as the practical use of this 
information is not evident. [See comment 5] 

USAID is and will continue to examine additional options for funding 
M&E at the headquarters and country level. 

The following are GAO's comments on the U.S. Agency for International 
Development letter dated September 17, 2009. 

GAO Comments: 

1. We reviewed the eight documents listed and acknowledged several of 
them in our report's section on policies and procedures. However, none 
of these documents constitutes a plan. Instead, they are policy or 
technical documents that describe how USAID and its awardees should 
plan for aspects of monitoring and evaluation. 

2. We considered FFP's Strategic Plan for 2006-2010 but determined it 
to be irrelevant to our analysis because it was prepared in 2005, prior 
to the passage of the 2008 Food for Peace Act, and covers only some 
aspects of monitoring. The Performance Management Plan is related to 
the strategic plan and provides a set of indicators to measure 
progress. Neither of these documents discusses funding, resources, 
staffing needs, or how results would be used to improve program 
management. 

3. We reported on the planned studies earlier in this report. However, 
USAID provided us not with a plan, but rather with a list of selected 
studies, their implementers, costs, and a high-level description. In 
the list, USAID did not discuss alternatives, outline priorities, or 
explain how the selected studies would address FFP's most pressing 
needs or be used for program management. As we noted, several studies 
do not have assured funding due to the legal restrictions on FFP's 
funding, which makes the need for integrated planning even more 
critical. 

4. We noted that FFP allocated the new staff positions in order to 
ensure coverage in all countries where it has programs. While this is 
one factor to consider in workforce planning, FFP did not consider 
other important factors, such as existing staff skills or country- 
specific needs for monitoring and evaluation because it did not collect 
data on them. Lacking a systematic process to collect such data, FFP is 
unable to determine if it has adequate staff with sufficient training 
and skills in monitoring and evaluation. 

5. FFP officials reported that they had not used almost $9 million of 
the $22 million available in fiscal year 2009. Better planning could 
ensure that the available funds are more fully used in future years for 
activities such as surveying FFP staff on their monitoring and 
evaluation skills and conducting an assessment of FFP monitoring and 
evaluation needs. 

[End of section] 

Appendix V: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Thomas Melito, (202) 512-9601, or melitot@gao.gov: 

Staff Acknowledgments: 

In addition to the individual named above, Phillip Thomas, Martin De 
Alteriis, and Michael LaForge, Assistant Directors; and Sada Aksartova; 
Carol Bray; Ming Chen; Debbie Chung; Lynn Cothern; Mark Dowling; Brian 
Egger; Ian Ferguson; Etana Finkler; Kendall Helm; Joy Labez; Jeff 
Miller; Yvonne Moss; Ulyana Panchishin; Lisa Reijula; Julia Ann 
Roberts; Barbara Shields; Stephanie Shipman; and George Taylor made key 
contributions to this report. 

[End of section] 

Related GAO Products: 

International Food Assistance: Key Issues for Congressional Oversight. 
[hyperlink, http://www.gao.gov/products/GAO-09-977SP]. Washington, 
D.C.: September 30, 2009. 

International Food Assistance: Local and Regional Procurement Provides 
Opportunities to Enhance U.S. Food Aid, but Challenges May Constrain 
Its Implementation. [hyperlink, 
http://www.gao.gov/products/GAO-09-757T]. Washington, D.C.: June 4, 
2009. 

International Food Assistance: Local and Regional Procurement Can 
Enhance the Efficiency of U.S. Food Aid, but Challenges May Constrain 
Its Implementation. [hyperlink, 
http://www.gao.gov/products/GAO-09-570]. Washington, D.C.: May 29, 
2009. 

International Food Security: Insufficient Efforts by Host Governments 
and Donors Threaten Progress to Halve Hunger in Sub-Saharan Africa by 
2015. [hyperlink, http://www.gao.gov/products/GAO-08-680]. Washington, 
D.C.: May 29, 2008. 

Foreign Assistance: Various Challenges Limit the Efficiency and 
Effectiveness of U.S. Food Aid. [hyperlink, 
http://www.gao.gov/products/GAO-07-905T]. Washington, D.C.: May 24, 
2007. 

Foreign Assistance: Various Challenges Impede the Efficiency and 
Effectiveness of U.S. Food Aid. [hyperlink, 
http://www.gao.gov/products/GAO-07-560]. Washington, D.C.: April 13, 
2007. 

Foreign Assistance: U.S. Agencies Face Challenges to Improving the 
Efficiency and Effectiveness of Food Aid. [hyperlink, 
http://www.gao.gov/products/GAO-07-616T]. Washington, D.C.: March 21, 
2007. 

Darfur Crisis: Progress in Aid and Peace Monitoring Threatened by 
Ongoing Violence and Operational Challenges. [hyperlink, 
http://www.gao.gov/products/GAO-07-9]. Washington, D.C.: November 9, 
2006. 

Maritime Security Fleet: Many Factors Determine Impact of Potential 
Limits of Food Aid Shipments. [hyperlink, 
http://www.gao.gov/products/GAO-04-1065]. Washington, D.C.: September 
13, 2004. 

United Nations: Observations on the Oil for Food Program and Iraq's 
Food Security. [hyperlink, http://www.gao.gov/products/GAO-04-880T]. 
Washington, D.C.: June 16, 2004. 

Foreign Assistance: Lack of Strategic Focus and Obstacles to 
Agricultural Recovery Threaten Afghanistan's Stability. [hyperlink, 
http://www.gao.gov/products/GAO-03-607]. Washington, D.C.: June 30, 
2003. 

Foreign Assistance: Sustained Efforts Needed to Help Southern Africa 
Recover from Food Crisis. [hyperlink, 
http://www.gao.gov/products/GAO-03-644]. Washington, D.C.: June 25, 
2003. 

Food Aid: Experience of U.S. Programs Suggest Opportunities for 
Improvement. [hyperlink, http://www.gao.gov/products/GAO-02-801T]. 
Washington, D.C.: June 4, 2002. 

Foreign Assistance: Global Food for Education Initiative Faces 
Challenges for Successful Implementation. [hyperlink, 
http://www.gao.gov/products/GAO-02-328]. Washington, D.C.: Feb. 28, 
2002. 

Foreign Assistance: U.S. Food Aid Program to Russia Had Weak Internal 
Controls. [hyperlink, 
http://www.gao.gov/products/GAO/NSIAD/AIMD-00-329]. Washington, D.C.: 
September 29, 2000. 

Foreign Assistance: U.S. Bilateral Food Assistance to North Korea Had 
Mixed Results. [hyperlink, 
http://www.gao.gov/products/GAO/NSIAD-00-175]. Washington, D.C.: June 
15, 2000. 

Foreign Assistance: Donation of U.S. Planting Seed to Russia in 1999 
Had Weaknesses. [hyperlink, 
http://www.gao.gov/products/GAO/NSIAD-00-91]. Washington, D.C.: March 
9, 2000. 

Foreign Assistance: North Korea Restricts Food Aid Monitoring. 
[hyperlink, http://www.gao.gov/products/GAO/NSIAD-00-35]. Washington, 
D.C.: October 8, 1999. 

Food Security: Factors That Could Affect Progress toward Meeting World 
Food Summit Goals. [hyperlink, 
http://www.gao.gov/products/GAO/NSIAD-99-15]. Washington, D.C.: March 
22, 1999. 

Food Security: Preparations for the 1996 World Food Summit. [hyperlink, 
http://www.gao.gov/products/GAO/NSIAD-97-44]. Washington, D.C.: 
November 7, 1996. 

[End of section] 

Footnotes: 

[1] Section 3001 of Pub. L. No. 110-246, the Food, Conservation, and 
Energy Act of 2008 changed the title of the underlying legislation from 
the Agricultural Trade Development Assistance Act of 1954, also known 
as Pub. L. 480, to the Food for Peace Act. Title II of the Food for 
Peace Act addresses donation of agricultural commodities for 
humanitarian purposes. 

[2] USAID FFP, Bureau for Democracy, Conflict, and Humanitarian 
Assistance, Strategic Plan for 2006-2010 (Washington, D.C.: May 2005). 

[3] For the purposes of this report, we consider whether agencies are 
monitoring (1) that the necessary inputs for programs (equipment, 
supplies, and personnel) are in place and that programs are being 
implemented as intended and (2) that programs are achieving their 
expected outputs and targets by regularly tracking performance 
indicators. See appendix I for more details. 

[4] For the purposes of this report, we consider whether agencies are 
evaluating on a periodic or ad hoc basis (1) the extent to which 
program objectives were achieved as well as the factors that influenced 
outcome achievement and (2) the degree to which outcomes and impacts 
can be attributed directly to programs, and the cost effectiveness of 
the programs. These studies are often considered cross-cutting because 
they look across projects, programs, and countries, among other things. 
See appendix I for more details. 

[5] This recommendation was also addressed to the U.S. Department of 
Agriculture. See GAO, Foreign Assistance: Various Challenges Impede the 
Efficiency and Effectiveness of U.S. Food Aid, [hyperlink, 
http://www.gao.gov/products/GAO-07-560] (Washington, D.C.: Apr. 13, 
2007). 

[6] In the same speech, the Secretary of State also noted that U.S. 
diplomacy and development demand "our best efforts and deepest 
commitment to achieving sustainable results" and called for an evidence-
based approach to foreign policy. 

[7] See appendix II for USAID's report to Congress issued on December 
31, 2008. 

[8] AEA is an international professional association of evaluators 
devoted to the application and exploration of program evaluation, 
personnel evaluation, technology, and other forms of evaluation. In 
February 2009, AEA's Task Force on Evaluation Policy issued An 
Evaluation Roadmap for a More Effective Government, which recommended a 
framework for each federal agency to guide the development and 
implementation of its evaluation programs. The framework offers a set 
of guiding principles intended to facilitate the integration of 
monitoring and evaluation activities with program management. We used 
the AEA framework in considering the extent to which USAID's practices 
for monitoring and evaluation are consistent with established 
principles. Additionally, GAO has issued guidance on aspects of 
monitoring and evaluation over the years in reports including 
Performance Measurement and Evaluation: Definitions and Relationships, 
[hyperlink, http://www.gao.gov/products/GAO-05-739SP] (Washington, 
D.C.: May 2005); The Results Act: An Evaluator's Guide to Assessing 
Agency Annual Performance Plans, [hyperlink, 
http://www.gao.gov/products/GAO/GGD-10.1.20] (Washington, D.C.: April 
1998); and Designing Evaluations, [hyperlink, 
http://www.gao.gov/products/GAO/PEMD-10.1.4] (Washington, D.C.: March 
1991). 

[9] IEEE is a nonprofit, technical professional association that 
develops standards for a broad range of global industries including the 
information technology and information assurance industries and is a 
leading source for defining best practices. 

[10] We selected Bangladesh and Haiti based on the size and scope of 
USAID's nonemergency food aid programs, the presence of field 
monitoring staff, and the differences in how food aid programs are 
monitored in these countries (For example, Layers, a computerized 
monitoring system for quality assurance, is used in Haiti but not in 
Bangladesh). 

[11] According to FFP, the selection of the 20 priority countries for 
nonemergency food aid programs, completed in 2009, was based on a 
weighted average of the country's ranking under three food insecurity 
indicators: (1) percentage of children stunted (utilization), (2) 
percentage of population living under $1 a day (access), and (3) 
percentage of population undernourished (availability). The utilization 
indicator contributes 60 percent to the needs-based country rank, 
whereas the access indicator and the availability indicator contribute 
30 and 10 percent, respectively. 

[12] On a case-by-case basis, SYAPs may be extended beyond the initial 
12 months approved for the program. 

[13] These development projects are typically supported by implementing 
partners that sell food aid commodities as a means to generate cash to 
fund these projects. Food-insecure populations, as defined in FFP's 
Strategic Plan for 2006-2010, are people at risk of food insecurity 
because of physiological, socioeconomic, and political status, physical 
security, and limited or weak governance or a population whose ability 
to cope has been temporarily hindered by a shock. Beginning in 2006, 
multiyear assistance programs, MYAPs, replaced Development Assistance 
Programs (DAP). DAPs focused on increasing food security, whereas 
MYAPs--reflecting FFP's main strategic objective for 2006-2010--focus 
on reducing food insecurity. 

[14] Monetization is a practice that involves the sale in a recipient 
country of food aid commodities that have been purchased in and shipped 
from the United States. 

[15] Following the AEA guidance, we are defining program management as 
the management of a program's key stages, from planning and initial 
development through start up, ongoing implementation, and 
reauthorization. 

[16] The FANTA-2 project was established through a cooperative 
agreement between USAID and AED. Under subagreements, FANTA-2 staff 
collaborate with partner organizations such as Cornell University, 
Tufts University, Food Aid Management, the International Food Policy 
Research Institute, Technical Assistance to NGOs International, and the 
World Health Organization. 

[17] See Michael Bamberger, Jim Rugh, and Linda Mabry, Real World 
Evaluation: Working under Budget, Time, and Data and Political 
Constraints (Thousand Oaks, Calif.: Sage, 2006). The authors reviewed 
evaluations of CARE International programs and projects. 

[18] The 9 filled positions are as follows: 1 in Burundi, 3 in 
Madagascar, 1 in Malawi, 1 in Mali, 1 in Niger, 1 in Zambia, as well as 
1 position in the Southern Africa Regional Office. 

[19] According to USAID, 4 new positions in Madagascar include 1 
administrative staff member and 1 driver. 

[20] The term "architectural diagram" includes a description of an 
organization's current or "as is" information technology environment 
and its target or "to be" information technology environment. The 
architecture also includes a transition or sequencing plan, which is 
based on an analysis of the gaps between the "as is" and "to be" 
environments. 

[21] IEEE Guide for Information Technology--System Definition--Concept 
of Operations Document, Std. 1362-1998. 

[22] Remote monitoring will be based upon a regular assessment of (1) a 
livelihoods-based analytic food security framework of food security, 
(2) weather and crop monitoring and assessment, and (3) markets and 
trade monitoring and assessment. 

[23] Layers was developed as part of the FANTA-2 project. FANTA staff 
conduct initial training on the use of Layers for FFP field staff and 
provide continuing technical assistance to FFP staff after Layers is 
introduced. 

[24] Federal grants and cooperative agreements are used to accomplish a 
public purpose of support or stimulation authorized by the federal 
statute. The award mechanisms of grants and cooperative agreements are 
the largest components of awards made by USAID. 

[25] Personal services contracts are characterized by the employer- 
employee relationship that exists between the government and the 
contractor. The contractor is treated very much like the civil service 
staff, frequently performing the same or similar work. USAID hires 
personal services contractors to work in a variety of fields both in 
USAID/Washington and in USAID missions worldwide. 

[26] In addition, the viability of Layers will depend on USAID 
missions' willingness to use their own funds to provide ongoing support 
for Layers--estimated by FFP at $60,000 annually--after the first year. 
FFP and FANTA-2 are optimistic that missions will want to do so both 
because the cost is small relative to improvements in the monitoring of 
the implementing partners' programs and because the system can be 
easily adapted for nonfood aid programs. 

[27] Prior to 2009, food security country frameworks had been prepared 
for Madagascar, Malawi, and Mozambique. 

[28] In 1977, Congress passed the Bellmon Amendment to Pub. L. No. 480 
(Pub. L. No. 480 was renamed the Food for Peace Act in 2008). As 
required by the Bellmon Amendment, before food aid can be supplied, the 
U.S. government must determine that (1) adequate storage facilities are 
available in the recipient country at the time of export of the 
commodity to prevent waste or spoilage and (2) the distribution of the 
commodities will not result in a substantial disincentive or 
interference with domestic production or marketing in that country. 

[29] These bulletins explain requirements regarding monitoring and 
evaluation, reporting, and performance indicators and reinforce the 
Automated Directives System (ADS), chapter 203, which details the 
agency's monitoring and evaluation requirements. 

[30] According to guidance issued by State's Office of the Director of 
Foreign Assistance in March 2009, independent evaluations tend to have 
a high degree of credibility because the evaluators conducting them are 
not responsible to managers in the projects that are being evaluated. 
However, USAID's capacity to perform independent evaluations in recent 
years has been limited because it lacked a central evaluation unit 
between 2006 and 2008. 

[31] In one instance, we were unable to determine whether the lead 
evaluator was independent due to a lack of information in the 
evaluation and, in another instance, it was not clear whether the 
external evaluators had worked as a team with the implementing 
partners. In addition to the 16 final evaluations we found in DEC, we 
looked at a final evaluation that the USAID mission in Bangladesh 
provided to us in hard copy and found that it had been conducted by an 
external evaluator. 

[32] AED defines "integrity" as striving for the highest ethical 
standards, transparency, and accountability in all aspects of its work 
as an independent organization free of political influence. 

[33] Tufts University, an approved subrecipient under the FANTA-2 
cooperative agreement, recently received a 3-year task order to assess 
the success of exit strategies for FFP's food aid programs in Bolivia. 

[34] According to USAID officials, there is an example of a cross- 
cutting evaluation that was funded in part by FFP and compared two 
approaches to prevent malnutrition in infants in Haiti. The first 
approach targeted the interventions on all children aged between 6 and 
23 months (the "preventive" model) and the second approach targeted 
underweight children aged between 6 and 60 months (the "recuperative" 
model). By the use of carefully selected comparison groups, the study 
concluded that the preventive early intervention model yielded the 
greatest benefits. However, the only other example FFP provided of a 
recent cross-cutting study was a FANTA Report on Food Aid and Food 
Security Assessment: A Review of the Title II Development Food Aid 
Program issued in 2002. 

[35] Guidance recently issued by FFP describes evaluation as answering 
the "why," "why not" and the "what else" of performance and states that 
it is used on a periodic basis to assess effects and impacts, as well 
as efficiency and effectiveness. However, the guidance only requires 
evaluations of the extent to which Food for Peace programs are meeting 
outcomes and objectives. 

[36] FFP plans for conducting future cross-cutting impact evaluations 
and other studies were discussed in this report. 

[37] According to USAID's most recent workforce plan, the agency has 
created competency models and conducts annual competency assessments 
for the human resources, information technology, and acquisitions 
occupations. It projects that it will complete competency models for 
the foreign service in fiscal year 2010, civil service in fiscal year 
2011, and foreign service nationals and U.S. personal services 
contractors in fiscal year 2013. USAID, USAID Five-Year Workforce Plan 
FY 2009-FY 2013. 

[38] During our fieldwork in Bangladesh and Haiti, we found that the 
number of staff devoted to monitoring and evaluation, and the amount of 
time they dedicated to those activities, varied considerably between 
the two missions. FFP in Haiti dedicated more staff to monitoring and 
evaluation, and they reported that they spent more of their time on 
monitoring and evaluation than did FFP staff in Bangladesh. 

[39] The Food for Peace Act authorizes the hiring of personal services 
contractors only in recipient countries. 

[40] USAID's central evaluation office was reestablished in 2008 but, 
as of August 2009, had only one employee, who is its chief. However, 
according to the chief, the office has begun the process of hiring 
about seven new staff. 

[41] At our request, the two FFP implementing partners that operate in 
Bangladesh both estimated that about 3 percent of their total project 
budgets were spent on monitoring and evaluation. However, these 
estimates were for all their monitoring and evaluation, including staff 
salaries. We were not able to independently verify these estimates. 

[42] For example, FFP implementing partners are to plan final 
evaluations for multiyear assistance programs. 

[43] USAID's central evaluation unit is in the process of preparing an 
agencywide plan for evaluation, which might be useful to units like FFP 
as they develop plans for their own programs. The central evaluation 
unit had hoped to complete its agencywide plan by summer 2009 but 
recently reported that the plan was unlikely to be finalized until a 
new USAID Administrator was confirmed. 

[44] We considered the field monitoring of USAID programs a form of 
input and implementation evaluation. 

[45] see PM2A guidance and research at [hyperlink, 
http://www.fanta.org]. 

[46] [hyperlink, 
http://www.usaid.gov/our_work/humanitarian_assistance/ffp/FFPtrigger_ind
icators.pdf]. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: