Evidence-Based Policymaking:

Selected Agencies Coordinate Activities, but Could Enhance Collaboration

GAO-20-119: Published: Dec 4, 2019. Publicly Released: Dec 4, 2019.

Additional Materials:

Contact:

Michelle Sager
(202) 512-6806
sagerm@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Agencies can bring multiple sources of information together to determine whether federal programs are working as intended — a practice known as “evidence-building.” Agencies assess existing evidence, determine whether new evidence is needed, and set priorities to get decision makers the evidence they need.

Collaboration within an agency can help ensure that evidence-building efforts are effective. While the agencies we reviewed have processes to coordinate evidence-building, we recommended that the Corporation for National and Community Service and the Departments of Labor and Health and Human Services collaborate better on setting priorities.

People with documents around a conference table

People with documents around a conference table

Additional Materials:

Contact:

Michelle Sager
(202) 512-6806
sagerm@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Federal decision makers need evidence about whether federal programs and activities achieve intended results as they set priorities and consider how to make progress toward national objectives. The five agencies GAO reviewed took actions that align with direction from Congress and the Office of Management and Budget (OMB) to strengthen their evidence-building activities. The five agencies are: the Departments of Education, Health and Human Services (HHS), and Labor (DOL); the Corporation for National and Community Service (CNCS); and the U.S. Agency for International Development. For example, based on a statutory requirement, a majority of grant funding for HHS's Maternal, Infant, and Early Childhood Home Visiting program is to be used for home visiting models with sufficient evidence of their effectiveness. Consistent with this requirement, HHS annually assesses evidence, such as the results of program evaluations, to identify effective home visiting models that grantees can implement.

Evidence-building can involve assessing existing evidence, identifying any new evidence needs, and prioritizing when to fulfill those needs. These efforts are fragmented within each of the five agencies—that is, each has multiple organizational units with responsibilities for evidence-building. For example, DOL has established separate units responsible for different sources of evidence—evaluations, performance information, and statistics. Effective collaboration can help agencies manage this fragmentation, and lead to improved results.

GAO found that to assess existing evidence, each agency established a coordinated, agency-wide process that reflects leading practices for collaboration. Those leading practices are: (1) defining a leadership model; (2) involving relevant participants ; (3) clarifying roles and responsibilities ; and (4) documenting that information in written guidance . However, agencies' processes for determining which new evidence to generate, when, and how (i.e., prioritizing new evidence) did not always reflect the leading practices (see figure).

Highlights_v5-102782_119-alc 

Why GAO Did This Study

Congress and OMB have taken steps intended to strengthen federal evidence-building activities. In September 2017, a federal commission found that agencies had uneven capacity to support, or did not fully coordinate, a full range of evidence-building activities.

GAO was asked to examine the coordination of federal evidence-building activities. This report (1) describes selected agencies' actions that align with direction from Congress and OMB to strengthen evidence-building activities and (2) examines the extent to which selected agencies' processes for coordinating those activities reflect leading practices for collaboration.

To address these objectives, GAO reviewed documents and interviewed officials about federal evidence-building activities at five selected agencies. GAO selected these agencies based on the greater number of experiences they had in comparison to other agencies incorporating these activities into the design and implementation of certain programs. GAO assessed their coordination of these activities against four leading practices for collaboration identified in GAO's past work.

What GAO Recommends

GAO is making a total of seven recommendations to DOL, CNCS, and HHS to better reflect leading collaboration practices in their evidence prioritization processes. DOL concurred, CNCS neither agreed nor disagreed, and HHS did not concur with the recommendations. CNCS and HHS stated, but did not provide information to support, that each had already taken relevant actions. GAO continues to believe the recommendations are valid, as discussed in the report.

For more information, contact Michelle Sager at (202) 512-6806 or sagerm@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: In comments on a draft of this report, the Corporation for National and Community Service (CNCS) neither agreed nor disagreed with this recommendation. Subsequently, in a June 2020 letter, CNCS stated that it agreed with our recommendation and described planned actions to strengthen its agency-wide evidence prioritization process. Among these planned actions were proposed changes to CNCS's approach for involving key participants from across the agency in the process. However, as of October 2020, CNCS had not yet provided us with documentation that would allow us to (1) assess the extent to which they address our recommendation and (2) confirm that these proposed changes were approved. We will continue to monitor CNCS's actions to address this recommendation.

    Recommendation: The Chief Executive Officer of CNCS should develop an approach to ensure that all relevant participants are involved in the agency-wide process for prioritizing evidence needs. (Recommendation 1)

    Agency Affected: Corporation for National and Community Service

  2. Status: Open

    Comments: In comments on a draft of this report, the Corporation for National and Community Service (CNCS) neither agreed nor disagreed with this recommendation. Subsequently, in a June 2020 letter, CNCS stated that it agreed with our recommendation and described planned actions to strengthen its agency-wide evidence prioritization process. Among these planned actions were workforce planning and hiring based on the knowledge, skills, and abilities needed for CNCS to address its evidence priorities. The letter stated that once those actions were completed, CNCS would revisit the roles and responsibilities of participants involved in the agency's evidence-building activities. We will continue to monitor CNCS's actions to address this recommendation.

    Recommendation: The Chief Executive Officer of CNCS should define roles and responsibilities for all relevant participants involved in the agency-wide process for prioritizing evidence needs. (Recommendation 2)

    Agency Affected: Corporation for National and Community Service

  3. Status: Open

    Comments: In comments on a draft of this report, the Corporation for National and Community Service (CNCS) neither agreed nor disagreed with this recommendation. Subsequently, in a June 2020 letter, CNCS stated that it agreed with our recommendations and described planned actions to strengthen its agency-wide evidence prioritization process. However, as of October 2020, CNCS had not yet provided us with documentation that would allow us to (1) confirm that these proposed changes were approved and (2) assess the extent to which written guidance for the revised process addresses our recommendation. We will continue to monitor CNCS's actions to address this recommendation.

    Recommendation: The Chief Executive Officer of CNCS should revise written guidance for the agency-wide process for prioritizing evidence needs to ensure it identifies all relevant participants and their respective roles and responsibilities. (Recommendation 3)

    Agency Affected: Corporation for National and Community Service

  4. Status: Closed - Implemented

    Comments: In a November 2019 letter providing comments on our report, HHS stated that it was in the process of developing an approach for including all relevant participants in its evidence prioritization process, in line with our recommendation. Subsequently, in March 2020, HHS announced its new evidence prioritization process. As part of that process, HHS's operating and staff divisions are to consult with internal and external stakeholders when prioritizing evidence needs. This stakeholder engagement can help HHS meet the evidence needs of decision makers both within and outside the agency.

    Recommendation: The Secretary of Health and Human Services should develop an approach to ensure that all relevant participants are involved in the department-wide process for prioritizing evidence needs. (Recommendation 4)

    Agency Affected: Department of Health and Human Services

  5. Status: Closed - Implemented

    Comments: In a November 2019 letter providing comments on our report, HHS stated that it was in the process of developing an approach for including all relevant participants in its evidence prioritization process, in line with our recommendation. Subsequently, in March 2020, HHS announced a new evidence prioritization process, and its related guidance addresses our recommendation. Specifically, HHS's written guidance for that process identifies roles and responsibilities for key participants. For example, the guidance directs the heads of HHS's operating divisions to work with experts in research and evaluation to develop evidence priorities for their organizations. Furthermore, it directs HHS's Evaluation Officer, located within the Office of the Assistant Secretary for Planning and Evaluation, to coordinate these priorities across the department. By clearly documenting who should be involved, and how they should contribute, HHS has greater assurance that relevant participants are effectively collaborating to prioritize the agency's evidence needs.

    Recommendation: The Secretary of Health and Human Services should revise written guidance for the department-wide process for prioritizing evidence needs to ensure it identifies all relevant participants and their respective roles and responsibilities. (Recommendation 5)

    Agency Affected: Department of Health and Human Services

  6. Status: Open

    Comments: In a November 2019 letter providing comments on a draft of this report, the Department of Labor (DOL) agreed with this recommendation. The letter further described actions DOL planned to take to address it. DOL stated that in fiscal year 2020 it would update its written guidance to formalize the involvement of relevant participants in department-wide evidence-building activities. However, as of October 2020, DOL had not yet provided us with documentation that would allow us to assess the extent to which its updated guidance addresses our recommendation. We will continue to monitor DOL's actions to address this recommendation.

    Recommendation: The Secretary of Labor should develop an approach to ensure that all relevant participants are involved in the department-wide process for prioritizing evidence needs. (Recommendation 6)

    Agency Affected: Department of Labor

  7. Status: Open

    Comments: In a November 2019 letter providing comments on a draft of this report, the Department of Labor (DOL) agreed with this recommendation. The letter further described actions DOL planned to take to address it. DOL stated that in fiscal year 2020 it would update its written guidance to formalize the involvement of relevant participants in department-wide evidence-building activities. However, as of October 2020, DOL had not yet provided us with documentation that would allow us to assess the extent to which its updated guidance addresses our recommendation. We will continue to monitor DOL's actions to address this recommendation.

    Recommendation: The Secretary of Labor should revise written guidance for the department-wide process for prioritizing evidence needs to ensure it identifies all relevant participants and their respective roles and responsibilities. (Recommendation 7)

    Agency Affected: Department of Labor

 

Explore the full database of GAO's Open Recommendations »

Oct 1, 2020

Aug 21, 2020

Aug 13, 2020

Aug 10, 2020

Jul 29, 2020

Jul 9, 2020

Jun 17, 2020

May 7, 2020

Apr 13, 2020

Feb 26, 2020

Looking for more? Browse all our products here