Human Trafficking: Monitoring and Evaluation of International Projects Are Limited, but Experts Suggest Improvements
Highlights
Human trafficking--a worldwide crime involving the exploitation of men, women, and children for others' financial gain--is a violation of human rights. Victims are often lured or abducted and forced to work in involuntary servitude. Since 2001, the U.S. government has provided about $447 million to combat global human trafficking. As GAO previously reported, estimates of the number of trafficking victims are questionable. In this report, GAO examines (1) collaboration among organizations involved in international antitrafficking efforts, (2) U.S. government monitoring of antitrafficking projects and difficulties in evaluating these projects, and (3) suggestions for strengthening monitoring and evaluation. GAO analyzed agency documents; convened an expert panel; interviewed officials; and conducted fieldwork in Indonesia, Thailand, and Mexico.
While governments, international organizations, and nongovernmental organizations have recognized the importance of collaborating and have established some coordination mechanisms and practices, they will need to overcome challenges that have impeded collaboration in the past for their efforts to be successful. In two of the three countries GAO visited, it found that host governments--which bear ultimate responsibility for combating trafficking within their borders--have passed national antitrafficking laws and enacted national action plans. However, organizations continue to face numerous challenges when collaborating to combat human trafficking, including varying levels of government commitment and capacity. For example, some governments treat foreign trafficking victims as illegal immigrants and deport rather than protect them. In addition, according to officials in two of the three countries GAO visited, the ministries responsible for coordinating antitrafficking efforts have limited authority and capacity. U.S. government-funded antitrafficking projects often lack some important elements that allow projects to be monitored, and little is known about project impact due to difficulties in conducting evaluations. Project documents GAO reviewed generally include monitoring elements, such as an overarching goal and related activities, but often lack other monitoring elements, such as targets for measuring performance. To oversee projects, State officials supplement their efforts with assistance from U.S. embassy staff, but have not established written guidance for oversight. Officials said that they are working to improve performance measures and develop monitoring guidance. Conducting impact evaluations of antitrafficking projects is difficult due to several factors, including questionable project-level estimates of the number of trafficking victims. These estimates are needed for baselines by which to evaluate how effectively specific interventions are reducing trafficking. Elements in the design of certain projects, such as objectives that are too broad, further impede evaluation. Because of these difficulties, few impact evaluations have been completed, and little is known about the impact of antitrafficking interventions. A GAO-convened panel of experts identified and discussed ways to address the factors that make it difficult to monitor and evaluate antitrafficking projects. Panelists' suggested approaches included improving information on the nature and severity of trafficking and addressing monitoring and evaluation in project design. To improve information on trafficking, panelists suggested methods that have been used to sample other hard-to-reach populations, including domestic violence victims. One suggested method is sampling of "hot spots"--an intensive search for victims in areas known to have high concentrations of victims. To address weaknesses in project design that impede monitoring and evaluation, panelists suggested that officials design projects that clearly link activities to intended outcomes, identify measurable indicators, and establish procedures for setting and modifying targets.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status Sort descending |
|---|---|---|
| Department of Labor | The Secretaries of State and Labor and the Administrator of USAID should improve the monitoring and evaluation of their projects to combat global human trafficking by considering monitoring and evaluating weaknesses in the design of antitrafficking projects, including developing a framework that clearly links activities with project-level goals, indicators, and targets, conducting "evaluability assessments" to determine whether a project is ready to be evaluated, and building monitoring and evaluation into project design before the project is implemented. |
Closed – Implemented
In its July 2007 comments on the report, Labor said that the report highlighted important areas for improving monitoring and evaluation of U.S.-funded antitrafficking programs. DOL has highlighted several steps DOL's Bureau of International Labor Affairs (ILAB), Office of Child Labor, Forced Labor, and Human Trafficking has taken in response to the recommendation to address weaknesses in the design of antitrafficking projects. In its August 2009 memo, DOL reported that it had (1) updated the procedures that staff should follow when planning evaluations and planned to train staff on these procedures, and (2) revised its standard terms of reference for contracting mid-term and final evaluations of international technical assistance projects, including antitrafficking projects. Further, in its August 2009 memo, DOL reported that, in fiscal year 2008, it had pilot-tested a new monitoring tool to strengthen, among other things, the reliability and validity of project performance data; beginning in fiscal year 2010, this tool was included in the project design phase.
|
| Department of Labor | The Secretaries of State and Labor and the Administrator of USAID should improve the monitoring and evaluation of their projects to combat global human trafficking by considering improving information about project impact on the nature and severity of human trafficking, including developing better data about the incidence of trafficking at the project level and applying rigorous evaluation methodologies. |
Closed – Implemented
In its July 2007 comments on the report, Labor commented that the report highlighted important areas for improving monitoring and evaluation of U.S.-funded antitrafficking programs. Labor reported that, in April 2010, its Bureau of International Labor Affairs (ILAB), Office of Child Labor, Forced Labor, and Human Trafficking (OCFT) began pilot-testing a randomized control test (RCT) evaluation to determine the impact of Labor's intervention on the use of child labor in the carpet industry in three countries. We consider this action to be responsive to the recommendation. These actions were in addition to other actions to improve data collection and apply rigorous evaluation methods taken by Labor in fiscal years 2007 and 2008 in response to GAO recommendations in this report.
|
| Department of State | The Secretaries of State and Labor and the Administrator of USAID should improve the monitoring and evaluation of their projects to combat global human trafficking by considering improving information about project impact on the nature and severity of human trafficking, including developing better data about the incidence of trafficking at the project level and applying rigorous evaluation methodologies. |
Closed – Implemented
In their July 2007 comments on the report, State agreed that the antitrafficking field is well-served by more information about the nature and severity of human trafficking and that State is committed to investigating methods of assessing the effectiveness of projects. In December 2010, State stated that, to help address the dearth of baseline data on the nature and severity of human trafficking in most countries, State has funded a number of grantees to collect baseline data through their programming efforts. One grantee in India, for example, has initiated a household survey to provide a baseline estimate of the number of people who are in debt bondage; which households are affected by trafficking; and the economic, social, and political status of the families in the designated area of these villages. To date the survey has been completed in one village, is nearing completion in two villages, and is underway in four more. In addition, as part of its effort to evaluate projects, State has developed a set of 26 common indicators to cover program outputs and outcomes and also measure systemic change. These indicators are organized according to a programmatic framework of prevention, protection, and prosecution oriented activities. By the end of FY 2011, State plans to be able to provide statistical information about the impact of State funding on common data questions, such as the number of victims of trafficking provided with discrete services (e.g. shelter, medical services, counseling) the number of traffickers prosecuted, and the estimated number of persons reached through preventative activities.
|
| U.S. Agency for International Development | The Secretaries of State and Labor and the Administrator of USAID should improve the monitoring and evaluation of their projects to combat global human trafficking by considering monitoring and evaluating weaknesses in the design of antitrafficking projects, including developing a framework that clearly links activities with project-level goals, indicators, and targets, conducting "evaluability assessments" to determine whether a project is ready to be evaluated, and building monitoring and evaluation into project design before the project is implemented. |
Closed – Implemented
In a September 2007 letter responding to the GAO recommendations, USAID stated that it would implement them to the greatest extent possible and would send the GAO report to its field missions to signal its commitment to good project design and evaluation. In December 2009, USAID published an evaluation framework for trafficking in persons (TIP) prevention and victim protection programs. The framework links anti-TIP interventions to program impact and provides guidance on evaluating anti-trafficking activities. USAID reported that it has distributed the evaluation framework to its regional bureaus, and in June 2011 provided examples of current programs that are incorporating monitoring and evaluation into program design. For example, during the design phase, a pilot project in Russia hired an independent monitoring and evaluation firm to develop appropriate indicators to evaluate the project and to link inputs and activities to program goals. Further, the agency has said that it is developing a Field Guide to Combat Trafficking, which includes monitoring and evaluation strategies, and to develop an anti-trafficking survey instrument to more effectively evaluate programs. We consider these actions to be responsive to the recommendation.
|
| Department of State | The Secretaries of State and Labor and the Administrator of USAID should improve the monitoring and evaluation of their projects to combat global human trafficking by considering monitoring and evaluating weaknesses in the design of antitrafficking projects, including developing a framework that clearly links activities with project-level goals, indicators, and targets, conducting "evaluability assessments" to determine whether a project is ready to be evaluated, and building monitoring and evaluation into project design before the project is implemented. |
Closed – Implemented
In its July 2007 comments on the report, State agreed that effective project design is critical to successful project implementation and program monitoring, and can lay the foundation for evaluation. In June 2010, GAO attended a State Department-sponsored conference on program evaluation, which included a session on conducting evaluability assessments of antitrafficking programs. The presenter, from State's Office to Monitor and Combat Trafficking in Persons, mentioned GAO in the context of her office hiring two organizations in 2009, Westat and The Urban Institute, to conduct an evaluability assessment of the anti-trafficking programs of 8 Global Trafficking in Persons grantees. The purpose was to determine whether these programs were ready for an effectiveness and impact evaluation. The presentation slides provide information on the methodology of both evaluability assessments. The slides also include a logic model that clearly links project objectives with activities, outputs, and outcomes. The presenter also mentioned that in order to build monitoring and evaluation into the project design, some recent G/TIP's grant agreements now include requirements for the grantee to collect data on specific performance indicators. In addition, in a December 2010 letter, State stated that it has funded Westat Inc. and the Urban Institute to develop two fact sheets each to be disseminated to grantees and made available to other anti-trafficking practitioners. Drawing from their experiences conducting the evaluability assessments, they will create worksheets on the following topics: identifying measures to reflect the impact of your program's activities; how to measure the effectiveness of prevention activities; assessing the effectiveness of shelter care; the necessary elements (pre-conditions) for conducting impact evaluations.
|
| U.S. Agency for International Development | The Secretaries of State and Labor and the Administrator of USAID should improve the monitoring and evaluation of their projects to combat global human trafficking by considering improving information about project impact on the nature and severity of human trafficking, including developing better data about the incidence of trafficking at the project level and applying rigorous evaluation methodologies. |
Closed – Not Implemented
In a 2010 response to GAO on this recommendation, USAID reported that it conducted assessments of anti-trafficking projects since 2000 in eight Asian countries and identified lessons learned and best practices for future anti-trafficking programming; hosted a symposium of U.S. government, NGO and private sector specialists to identify lessons learned; and is working with State to identify promising practices and indicators to improve anti-trafficking programming. In June 2011, USAID further responded that it is in the process of (1) developing a trafficking in persons program survey tool to identify priority countries to help guide program interventions, and (2) is creating a Trafficking in Persons Monitoring & Evaluation Committee that will provide guidance to missions and Washington operating units on best strategies for monitoring and evaluation. These responses on completed or planned actions do not provide sufficient support that USAID has taken action to improve information on project impact by developing better data on the incidence of trafficking or applying rigorous evaluation methodologies.
|