Comparative Effectiveness Research:

HHS Needs to Strengthen Dissemination and Data-Capacity-Building Efforts

GAO-15-280: Published: Mar 3, 2015. Publicly Released: Mar 3, 2015.

Additional Materials:

Contact:

Linda T. Kohn
(202) 512-7114
kohnl@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Agency for Healthcare Research and Quality (AHRQ), an agency within the Department of Health and Human Services (HHS), has taken some steps to disseminate comparative clinical effectiveness research (CER), as required under the Patient Protection and Affordable Care Act (PPACA), but has not taken other actions to help it fully address its dissemination requirements. The steps it has taken include the creation of tools that organize and disseminate research findings to certain targeted stakeholder groups and the development of plans for a publicly available database that includes CER. For example, AHRQ's marketing plans—customized plans to help convey key messages about AHRQ's research—include various informational tools to disseminate CER, such as research summaries that communicate research findings to clinicians, consumers, caregivers, and policymakers. However, the agency has not clearly defined how to disseminate information to certain stakeholder groups specified in the law, nor has it identified and documented time frames to implement the marketing plans and distribute the associated informational tools, as would be consistent with federal internal control standards, which state that significant events need to be clearly documented to ensure management goals are carried out. Additionally, in order to implement PPACA's requirement for developing a publicly available database that contains CER evidence, AHRQ officials told GAO that they plan to create a web page to list and provide users with links to existing publicly available databases that could be used to search for CER, but they have not documented a specific implementation plan that includes time frames and strategies to address known potential limitations, such as difficulties that certain users may face in searching the databases for CER results.

HHS's Assistant Secretary for Planning and Evaluation (ASPE) has coordinated among various agencies to fund projects intended to build data capacity for CER, but its approach lacks key elements needed to ensure its effectiveness. For example, these projects include an effort to better standardize data that could be used in multiple research projects. However, HHS's approach to building data capacity for CER lacks key elements, such as defined objectives, milestones, and time frames, that are necessary to ensure effectiveness. ASPE officials worked with the Office of the National Coordinator for Health Information Technology to develop a strategic road map to guide both the identification and selection of ASPE's projects beginning in fiscal year 2014 through fiscal year 2019. Although the February 2014 strategic framework for the road map highlighted several priority objectives, such as enabling the collection of standardized clinical data, these objectives were broad and not clearly defined. For example, although ASPE identified and considered related, ongoing federal and non-federal data infrastructure projects in an attempt to identify needs or gaps, among other things, its strategic road map is unclear on the timing and level of coordination that would be necessary for its projects to work together with these related projects to improve data capacity. Standard practices for project management call for agencies to conceptualize, define, and document specific goals and objectives in the planning process, along with the appropriate steps, milestones, time frames, and resources needed to achieve those results.

Why GAO Did This Study

PPACA imposed new requirements on HHS related to CER—research that evaluates and compares health outcomes and the clinical effectiveness, risks, and benefits of two or more medical treatments or services. Among other things, PPACA required AHRQ to broadly disseminate findings from federally funded CER and the Secretary of HHS (who, by delegation, charged ASPE) to coordinate federal programs to build data capacity for CER. PPACA also mandated that GAO review HHS's CER activities. This report examines (1) AHRQ's activities to disseminate the results of federally funded CER and (2) ASPE's activities to coordinate federal programs to support CER by building the capacity to collect, link, and analyze data, among other objectives. GAO reviewed relevant legal requirements and HHS documentation; interviewed HHS officials; and obtained information from five stakeholder groups that AHRQ targeted to receive disseminated information or were otherwise involved in AHRQ's dissemination efforts.

What GAO Recommends

GAO recommends that HHS direct (1) AHRQ to take several actions related to its dissemination efforts, including identifying and documenting time frames for the implementation and distribution of marketing plans and informational tools, and (2) ASPE to include clearly defined objectives, milestones, and time frames, or other indicators of performance, in its strategic road map used to identify its CER-funded projects. HHS concurred with the recommendations.

For more information, contact Linda T. Kohn at (202) 512-7114 or kohnl@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In 2017, AHRQ provided documentation to show that each plan for marketing activities now includes timeframes for the activities, including for informational tools.

    Recommendation: To help ensure that HHS fully addresses its dissemination requirements under PPACA, the Secretary of Health and Human Services should direct AHRQ to identify and document time frames for the implementation and distribution of marketing plans and informational tools.

    Agency Affected: Department of Health and Human Services

  2. Status: Open

    Comments: HHS officials provided information in 2017 on dissemination efforts conducted as part of the Patient-Centered Outcomes Research Clinical Decision Support Learning Network. However, we have not yet identified the extent to which these efforts are specifically focused on federal and private health plans, or on health IT vendors, as specified in the GAO recommendation.

    Recommendation: To help ensure that HHS fully addresses its dissemination requirements under PPACA, the Secretary of Health and Human Services should direct AHRQ to expand dissemination efforts to federal and private health plans and vendors of health information technology focused on clinical decision support.

    Agency Affected: Department of Health and Human Services

  3. Status: Open

    Comments: HHS officials provided information in 2017 to describe AHRQ's approach to address dissemination requirements through a website that includes instructions and links for searching multiple different publicly available databases that have information on comparative effectiveness research. However, we have not yet identified the extent to which AHRQ has plans to meet the needs of various potential users in the general public, as specified in GAO's recommendation.

    Recommendation: To help ensure that HHS fully addresses its dissemination requirements under PPACA, the Secretary of Health and Human Services should direct AHRQ to document and complete plans to develop a publicly available database, including plans to meet the needs of various potential users in the general public.

    Agency Affected: Department of Health and Human Services

  4. Status: Open

    Comments: AHRQ officials provided information in 2017 on how AHRQ and NIH officials have been meeting with each other to collaborate on dissemination activities. However, we have not yet identified specific plans for this collaboration, as specified in GAO's recommendation.

    Recommendation: To help ensure that HHS fully addresses its dissemination requirements under PPACA, the Secretary of Health and Human Services should direct AHRQ to develop specific plans on how it will collaborate with the National Institutes of Health on its dissemination activities.

    Agency Affected: Department of Health and Human Services

  5. Status: Closed - Implemented

    Comments: HHS officials provided documentation in 2017 to show how the Assistant Secretary for Planning and Evaluation has implemented a standardized monitoring system to track progress toward milestones and deliverables. Each quarter, projects are required to report on performance towards project goals and objectives, among other requirements. The "individual dashboard" reports for these projects include objectives, milestones, and timeframes, as specified in GAO's recommendation.

    Recommendation: To ensure that HHS fully addresses the PPACA requirements to build data capacity for CER, the Secretary should direct ASPE to include clearly defined objectives, milestones, and time frames, or other indicators of performance, in its strategic road map that is used to identify its Patient-Centered Outcomes Research Trust Fund projects.

    Agency Affected: Department of Health and Human Services

 

Explore the full database of GAO's Open Recommendations »

Nov 15, 2017

Nov 14, 2017

Nov 13, 2017

Nov 6, 2017

Oct 31, 2017

Oct 18, 2017

Oct 4, 2017

Looking for more? Browse all our products here