Electronic Health Record Programs:

Participation Has Increased, but Action Needed to Achieve Goals, Including Improved Quality of Care

GAO-14-207: Published: Mar 6, 2014. Publicly Released: Mar 6, 2014.


Additional Materials:


Carolyn L. Yocom
(202) 512-4931


Office of Public Affairs
(202) 512-4800

What GAO Found

Based on the number of providers awarded incentive payments, participation in the Department of Health and Human Services' (HHS) Medicare and Medicaid Electronic Health Record (EHR) programs increased substantially from their first year in 2011 to 2012. For hospitals, participation increased from 45 percent of those eligible for 2011 to 64 percent of those eligible for 2012. For professionals, such as physicians, participation increased from 21 percent of those eligible for 2011 to 48 percent of those eligible for 2012. While increases occurred, a substantial percentage of providers that participated in 2011 did not participate in 2012. Officials who oversee the programs at the Centers for Medicare & Medicaid Services (CMS) noted there could be several reasons for this, such as challenges in demonstrating meaningful use, and are monitoring the issue. Various program changes make future participation difficult to estimate. For example, increased stringency of requirements for the programs' second phase beginning in 2014—Stage 2—may slow participation, while the introduction of penalties in 2015 for some providers may motivate participation.

Reporting on meaningful use for 2011 and 2012 indicates that providers who have already participated in the programs' first phase—Stage 1—used their certified EHR systems more often than required. For example, for both 2011 and 2012, Medicare hospitals reported using computerized provider order entry for over 84 percent of patients—in excess of the required threshold for Stage 1 of 30 percent. However, some meaningful use measures may be more challenging for providers, including measures involving the electronic exchange of information. For example, less than 15 percent of professionals reported on an optional Stage 1 measure to provide a summary of care document at each care transition or referral, which is mandatory in Stage 2. A CMS official said the agency is taking steps to help providers prepare for Stage 2 meaningful use measures.

The lack of a comprehensive strategy limits HHS's ability to ensure the department can reliably use the clinical quality measures (CQM) collected in certified EHRs for quality measurement activities. Reliability issues persist, although CMS and HHS's Office of the National Coordinator for Health Information Technology (ONC) have made efforts to address concerns. For example, different providers may report CQMs based on and tested to different requirements depending on whether their EHRs have incorporated technical updates. Without a comprehensive strategy, efforts to address reliability issues (in accordance with the internal control standard requiring relevant and reliable information) and improve quality and efficiency may be limited.

Consistent with law and GAO guidance on assessing agency performance, HHS, CMS, and ONC have established some performance measures for the EHR programs that are tied to strategic goals regarding adoption and meaningful use of EHRs; however, they have not established measures that would help them to track progress toward program outcomes such as health care quality, efficiency, and patient safety. Although HHS expects that the use of EHRs can help achieve improved outcomes and support other efforts that are also intended to improve care, that result is not yet assured. CMS and ONC may lack critical information necessary to establish program priorities and subsequently make program adjustments based on progress toward outcomes.

Why GAO Did This Study

The Health Information Technology for Economic and Clinical Health (HITECH) Act established the EHR programs, which provide incentive payments for—and later are expected to apply penalties to—certain providers, such as hospitals and professionals, to encourage them to demonstrate meaningful use of certified EHR technology and meet other program requirements. For example, one measure of meaningful use requires providers to implement checks for potential drug interactions with patients' other drugs and allergies.

As mandated by the HITECH Act, GAO (1) assessed the extent of current and expected participation in the EHR programs, (2) examined information reported by providers and others to measure meaningful use in the EHR programs, (3) evaluated HHS efforts to ensure that EHR data can be reliably used to measure quality of care, and (4) evaluated HHS efforts to assess the effect of the EHR programs on program goals related to adoption and meaningful use of EHRs and improved outcomes. GAO analyzed data from CMS and other sources; reviewed applicable statutes, regulations, and guidance; and interviewed officials from HHS and stakeholder groups.

What GAO Recommends

GAO recommends that HHS develop a comprehensive strategy to better ensure the reliability of CQM data collected using EHRs and develop and use outcome-oriented performance measures to monitor progress toward goals. HHS agreed data reliability and performance monitoring are important but neither agreed nor disagreed with GAO's recommendations.

For more information, contact Carolyn Yocom at  (202) 512-4931 or yocomc@gao.gov .

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Priority recommendation

    Comments: In November 2018, HHS described a number of actions that it has taken, which demonstrate that it has implemented our recommendation. Specifically, CMS requires new electronic CQMs under consideration for use in CMS programs to have testing data to support feasibility, validity, and reliability, and, in 2018, CMS began requiring measure developers to submit testing results for those measures designed for clinicians. For example, CMS has developed a scoring system to assess reliability, and the agency's goal is that selected measures meet a threshold set by the agency. While some older measures still reported by hospitals and clinicians have not been subject to the same requirements as newer measures, CMS's strategy includes using the annual update process-during which measure specifications are reviewed-to help mitigate reliability concerns. Additionally, CMS has processes in place to gradually phase out older measures and replace them with new measures that have undergone feasibility, validity, and reliability testing. In conjunction with other previously established ONC activities, these activities for developing new and reviewing existing electronic CQMs collectively address the intent of our recommendation.

    Recommendation: To more effectively use CQMs to assess provider performance, the Secretary of Health and Human Services should direct CMS and ONC to develop a comprehensive strategy for ensuring that CQM data collected and reported using certified EHR technology are reliable, including testing for and mitigation of reliability issues arising from variance in certified EHR systems tested to different CQM specifications.

    Agency Affected: Department of Health and Human Services

  2. Status: Open

    Comments: HHS neither agreed nor disagreed with our recommendation. In July 2015, CMS noted that the agency is working to align the programs to better enable monitoring using outcome-oriented performance measures and noted that the agency is collecting data that will help them to develop such measures. CMS did not indicate when HHS plans to develop such measures as GAO recommended. In June 2016, CMS also noted that it analyzed the results of the EHR programs as of October 2015, but did not indicate that it used performance measures that assess outcomes. In September 2017, HHS officials provided us a variety of publically available reports, which they indicated show how program participants are progressing in the EHR programs and the related impacts. In 2018, CMS changed the name of these programs to the Promoting Interoperability programs to focus on improving interoperability and patients' access to health information, and officials noted that the agency is working to develop related outcome-based measures. To fully implement this recommendation, CMS needs to develop performance measures that enable the agency to assess whether the Promoting Interoperability programs are improving outcomes, such as health care quality, efficiency, and patient safety, as we recommended.

    Recommendation: To ensure that CMS and ONC can effectively monitor the effect of the EHR programs and progress made toward goals, the Secretary of Health and Human Services should direct the agencies to develop performance measures to assess outcomes of the EHR programs--including any effects on health care quality, efficiency, and patient safety and other health care reform efforts that are intended to work toward similar outcomes.

    Agency Affected: Department of Health and Human Services

  3. Status: Open

    Comments: In written responses provided by HHS in February 2014, on a draft of the report, the agency indicated that it agrees that outcome-oriented performance measures will be useful to evaluating the extent that the EHR programs--enacted through legislation--achieve the expected results. However, HHS did not identify any specific actions that it might undertake to address our recommendation. In July 2015, CMS indicated that the agency is still working to develop additional performance measures, which is a necessary first step towards implementing our recommendation to HHS that CMS and ONC use the outcome-oriented performance measures to make program adjustments, as appropriate. In September 2017 and March 2018, HHS officials provided us documents, which they indicated show how information gathered through monitoring activities was used to inform the EHR programs. In November 2018, HHS officials noted that CMS is actively working to use data submitted for the Promoting Interoperability Programs to improve upon the outcomes of patients, and also indicated (as noted above) that CMS is collaborating with stakeholders to develop outcome-based measures for the Promoting Interoperability Programs. To fully implement this recommendation, CMS needs to develop outcome-oriented performance measures and then demonstrate it is using them to make appropriate program adjustments. We will update the status of this recommendation when we receive additional information.

    Recommendation: To ensure that CMS and ONC can effectively monitor the effect of the EHR programs and progress made toward goals, the Secretary of Health and Human Services should direct the agencies to use the information these performance measures provide to make program adjustments, as appropriate, to better achieve program goals.

    Agency Affected: Department of Health and Human Services


Explore the full database of GAO's Open Recommendations »