President's Emergency Plan for AIDS Relief:

Agencies Can Enhance Evaluation Quality, Planning, and Dissemination

GAO-12-673: Published: May 31, 2012. Publicly Released: May 31, 2012.

Additional Materials:

Contact:

David B. Gootnick
(202) 512-3149
GootnickD@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Department of State’s (State) Office of the U.S. Global AIDS Coordinator (OGAC), the Department of Health and Human Services’ (HHS) Centers for Disease Control and Prevention (CDC), and the U.S. Agency for International Development (USAID) have evaluated a wide variety of President’s Emergency Plan for AIDS Relief (PEPFAR) program activities, demonstrating a clear commitment to evaluation. However, GAO found that the findings, conclusions, and recommendations were not fully supported in many PEPFAR evaluations. Agency officials provided nearly 500 evaluations addressing activities ongoing in fiscal years 2008 through 2010 in all program areas relating to HIV/AIDS treatment, prevention, and care. GAO’s assessment of a selected sample of seven OGAC-managed evaluations found that they generally adhered to common evaluation standards, as did most of a selected sample of 15 evaluations managed by CDC and USAID headquarters. Based on this assessment, GAO determined that these evaluations generally contained fully supported findings, conclusions, and recommendations. However, based on a similar assessment of a randomly selected sample taken from 436 evaluations provided by PEPFAR country and regional teams, GAO estimated that 41 percent contained fully supported findings, conclusions, and recommendations, while 44 percent contained partial support and 15 percent were not supported.

State, OGAC, CDC, and USAID have established detailed evaluation policies, as recommended by the American Evaluation Association (AEA). However, PEPFAR does not fully adhere to AEA principles relating to evaluation planning, independence and qualifications of evaluators, and public dissemination of evaluation results. Specifically, OGAC does not require country and regional teams to include evaluation plans in their annual operational plans, limiting its ability to ensure that evaluation resources are appropriately targeted. Further, although OGAC, CDC, and USAID evaluation policies and procedures provide some guidance on how to ensure evaluator independence and qualifications, they do not require documentation of these issues. GAO found that most PEPFAR program evaluations did not fully address whether evaluators had conflicts of interest and some did not include detailed information on the identity and makeup of evaluation teams. Finally, although OGAC, CDC, and USAID use a variety of means to share evaluation findings, not all evaluation reports are available online, limiting their accessibility to the public and their usefulness for PEPFAR decision makers, program managers, and other stakeholders.

Why GAO Did This Study

PEPFAR, reauthorized by Congress in fiscal year 2008, supports HIV/AIDS prevention, treatment, and care overseas. The reauthorizing legislation, as well as other U.S. law and government policy, stresses the importance of evaluation for improving program performance, strengthening accountability, and informing decision making. OGAC leads the PEPFAR effort by providing funding and guidance to implementing agencies, primarily CDC and USAID. Responding to legislative mandates, GAO (1) identified PEPFAR evaluation activities and examined the extent to which evaluation findings, conclusions, and recommendations were supported and (2) examined the extent to which PEPFAR policies and procedures adhere to established general evaluation principles. GAO reviewed these principles as well as agencies’ policies and guidance; surveyed CDC and USAID officials in 31 PEFAR countries and 3 regions; and analyzed evaluations provided by OGAC, CDC, and USAID.

What GAO Recommends

GAO recommends that State work with CDC and USAID to (1) improve adherence to common evaluation standards, (2) develop PEPFAR evaluation plans, (3) provide guidance for assessing and documenting evaluators’ independence and qualifications, and (4) increase online accessibility of evaluation results. Commenting jointly with HHS’s CDC and USAID, State agreed with these recommendations and noted steps it will take to implement them.

For more information, contact David Gootnick at (202) 512-3149 or gootnickd@gao.gov.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: The Secretary of State should direct the U.S. Global AIDS Coordinator, in collaboration with CDC and USAID to enhance PEPFAR evaluations, to provide detailed guidance for implementing agencies and country and regional teams on assessing, ensuring, and documenting the independence and competence of PEPFAR program evaluators.

    Agency Affected: Department of State

    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Secretary of State should direct the U.S. Global AIDS Coordinator, in collaboration with CDC and USAID to enhance PEPFAR evaluations, to require implementing agency headquarters and country and regional teams to include evaluation plans in their annual operational plans.

    Agency Affected: Department of State

    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Secretary of State should direct the U.S. Global AIDS Coordinator, in collaboration with CDC and USAID to enhance PEPFAR evaluations, to develop a strategy to improve PEPFAR implementing agencies’ and country and regional teams’ adherence to common evaluation standards.

    Agency Affected: Department of State

    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Secretary of State should direct the U.S. Global AIDS Coordinator, in collaboration with CDC and USAID to enhance PEPFAR evaluations, to increase the online accessibility of PEPFAR program evaluation results.

    Agency Affected: Department of State

    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Jul 21, 2014

    Jul 9, 2014

    Jul 8, 2014

    Jun 26, 2014

    Jun 25, 2014

    Jun 24, 2014

    Jun 17, 2014

    Looking for more? Browse all our products here