Skip to main content

Comparative Effectiveness Research: HHS Needs to Strengthen Dissemination and Data-Capacity-Building Efforts

GAO-15-280 Published: Mar 03, 2015. Publicly Released: Mar 03, 2015.
Jump To:
Skip to Highlights

Highlights

What GAO Found

The Agency for Healthcare Research and Quality (AHRQ), an agency within the Department of Health and Human Services (HHS), has taken some steps to disseminate comparative clinical effectiveness research (CER), as required under the Patient Protection and Affordable Care Act (PPACA), but has not taken other actions to help it fully address its dissemination requirements. The steps it has taken include the creation of tools that organize and disseminate research findings to certain targeted stakeholder groups and the development of plans for a publicly available database that includes CER. For example, AHRQ's marketing plans—customized plans to help convey key messages about AHRQ's research—include various informational tools to disseminate CER, such as research summaries that communicate research findings to clinicians, consumers, caregivers, and policymakers. However, the agency has not clearly defined how to disseminate information to certain stakeholder groups specified in the law, nor has it identified and documented time frames to implement the marketing plans and distribute the associated informational tools, as would be consistent with federal internal control standards, which state that significant events need to be clearly documented to ensure management goals are carried out. Additionally, in order to implement PPACA's requirement for developing a publicly available database that contains CER evidence, AHRQ officials told GAO that they plan to create a web page to list and provide users with links to existing publicly available databases that could be used to search for CER, but they have not documented a specific implementation plan that includes time frames and strategies to address known potential limitations, such as difficulties that certain users may face in searching the databases for CER results.

HHS's Assistant Secretary for Planning and Evaluation (ASPE) has coordinated among various agencies to fund projects intended to build data capacity for CER, but its approach lacks key elements needed to ensure its effectiveness. For example, these projects include an effort to better standardize data that could be used in multiple research projects. However, HHS's approach to building data capacity for CER lacks key elements, such as defined objectives, milestones, and time frames, that are necessary to ensure effectiveness. ASPE officials worked with the Office of the National Coordinator for Health Information Technology to develop a strategic road map to guide both the identification and selection of ASPE's projects beginning in fiscal year 2014 through fiscal year 2019. Although the February 2014 strategic framework for the road map highlighted several priority objectives, such as enabling the collection of standardized clinical data, these objectives were broad and not clearly defined. For example, although ASPE identified and considered related, ongoing federal and non-federal data infrastructure projects in an attempt to identify needs or gaps, among other things, its strategic road map is unclear on the timing and level of coordination that would be necessary for its projects to work together with these related projects to improve data capacity. Standard practices for project management call for agencies to conceptualize, define, and document specific goals and objectives in the planning process, along with the appropriate steps, milestones, time frames, and resources needed to achieve those results.

Why GAO Did This Study

PPACA imposed new requirements on HHS related to CER—research that evaluates and compares health outcomes and the clinical effectiveness, risks, and benefits of two or more medical treatments or services. Among other things, PPACA required AHRQ to broadly disseminate findings from federally funded CER and the Secretary of HHS (who, by delegation, charged ASPE) to coordinate federal programs to build data capacity for CER. PPACA also mandated that GAO review HHS's CER activities. This report examines (1) AHRQ's activities to disseminate the results of federally funded CER and (2) ASPE's activities to coordinate federal programs to support CER by building the capacity to collect, link, and analyze data, among other objectives. GAO reviewed relevant legal requirements and HHS documentation; interviewed HHS officials; and obtained information from five stakeholder groups that AHRQ targeted to receive disseminated information or were otherwise involved in AHRQ's dissemination efforts.

Recommendations

GAO recommends that HHS direct (1) AHRQ to take several actions related to its dissemination efforts, including identifying and documenting time frames for the implementation and distribution of marketing plans and informational tools, and (2) ASPE to include clearly defined objectives, milestones, and time frames, or other indicators of performance, in its strategic road map used to identify its CER-funded projects. HHS concurred with the recommendations.

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Health and Human Services To help ensure that HHS fully addresses its dissemination requirements under PPACA, the Secretary of Health and Human Services should direct AHRQ to identify and document time frames for the implementation and distribution of marketing plans and informational tools.
Closed – Implemented
In 2017, AHRQ provided documentation to show that each plan for marketing activities now includes timeframes for the activities, including for informational tools.
Department of Health and Human Services To help ensure that HHS fully addresses its dissemination requirements under PPACA, the Secretary of Health and Human Services should direct AHRQ to expand dissemination efforts to federal and private health plans and vendors of health information technology focused on clinical decision support.
Closed – Implemented
HHS officials provided information in 2017 on dissemination efforts conducted as part of the Patient-Centered Outcomes Research Clinical Decision Support Learning Network. As described on the learning network's website, it includes participation from health plans and health IT vendors, consistent with GAO's recommendation.
Department of Health and Human Services To help ensure that HHS fully addresses its dissemination requirements under PPACA, the Secretary of Health and Human Services should direct AHRQ to document and complete plans to develop a publicly available database, including plans to meet the needs of various potential users in the general public.
Closed – Implemented
HHS officials provided information in 2017 to describe AHRQ's approach to address dissemination requirements through a website that includes instructions and links for searching multiple different publicly available databases that have information on comparative effectiveness research. Furthermore, in September 2018, HHS provided information on AHRQ's efforts to solicit feedback from potential users, as well as modifications that AHRQ is making based on this feedback to make the website more user-friendly.
Department of Health and Human Services To help ensure that HHS fully addresses its dissemination requirements under PPACA, the Secretary of Health and Human Services should direct AHRQ to develop specific plans on how it will collaborate with the National Institutes of Health on its dissemination activities.
Closed – Implemented
AHRQ officials provided information in 2017 on how AHRQ and NIH officials have been meeting with each other to collaborate on dissemination activities. They also provided documents that describe ongoing plans for this collaboration, such as workgroup meeting schedules and minutes, as well as processes for collaborating to identify and complete key dissemination activities.
Department of Health and Human Services To ensure that HHS fully addresses the PPACA requirements to build data capacity for CER, the Secretary should direct ASPE to include clearly defined objectives, milestones, and time frames, or other indicators of performance, in its strategic road map that is used to identify its Patient-Centered Outcomes Research Trust Fund projects.
Closed – Implemented
HHS officials provided documentation in 2017 to show how the Assistant Secretary for Planning and Evaluation has implemented a standardized monitoring system to track progress toward milestones and deliverables. Each quarter, projects are required to report on performance towards project goals and objectives, among other requirements. The "individual dashboard" reports for these projects include objectives, milestones, and timeframes, as specified in GAO's recommendation.

Full Report

Office of Public Affairs

Topics

Agency evaluationComparative analysisData collectionHealth care programsInternal controlsProgram evaluationPatient care servicesResearch grantsResearch programsStrategic planningCaregiversFederal assistance programsHealth information technology