Nursing Homes:

CMS Needs Milestones and Timelines to Ensure Goals for the Five-Star Quality Rating System Are Met

GAO-12-390: Published: Mar 23, 2012. Publicly Released: Mar 23, 2012.

Additional Materials:

Contact:

Linda T. Kohn
(202) 512-7114
kohnl@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

CMS developed and implemented the Five-Star System largely during an 8-month period in 2008 with input from long-term care stakeholders, CMS’s Five-Star System contractor, and members of a technical expert panel—a panel composed of nine individuals that CMS identified as experts in long-term care research. CMS made numerous methodological decisions during the development of the Five-Star System, including three key methodological decisions. GAO defines key methodological decisions as those that at least six technical expert panel members—of the nine that GAO contacted—recalled as eliciting the most intense review and discussion during the development of the Five-Star System. One key methodological decision was how to combine the component ratings to create an overall rating. The other two key methodological decisions pertained to how to create ratings that account for variation in the type of care provided across nursing homes.

CMS generally considers modifying the Five-Star System in response to (1) methodological issues raised by stakeholders, (2) its routine monitoring of the system, and (3) the availability of new data sources. CMS officials explained that when a methodological issue is raised by long-term care stakeholders, they review the Five-Star System to determine whether modifications should be made. Officials said that each issue raised does not always result in modifications to the Five-Star System, although some minor modifications have been made. CMS also considers making modifications to the Five-Star System based on its periodic analyses of trends of the system; however, to date, no modifications have been made based on these analyses. Lastly, CMS is currently determining how to modify the staffing and quality measure ratings of the Five-Star System based on newly available data.

CMS has several planned efforts intended to improve the Five-Star System, including evaluating the usability of the system, adding nursing home capability information, revising the staffing component, and developing additional quality measures. However, CMS lacks GAO-identified leading strategic planning practices—the use of milestones and timelines to guide and gauge progress toward achieving desired results and the alignment of activities, resources, and goals—that could help the agency to more efficiently and effectively accomplish its planned efforts intended to improve the Five-Star System. While CMS officials have given us broad estimates for when they anticipate some of these efforts to be implemented, CMS does not have milestones and timelines associated with implementing the efforts, which could help ensure that appropriate progress is made towards implementation. In addition, CMS has not established, through planning documents, how its planned efforts to improve the Five-Star System will help CMS achieve the goals of the system—to inform consumers and improve provider quality. As a result, CMS may not be identifying and prioritizing its intended improvements in a manner that best ensures that the goals are being achieved.

Why GAO Did This Study

In 2008, in an effort to provide helpful information to consumers and improve provider quality, the Centers for Medicare & Medicaid Services (CMS) developed and implemented the Five-Star Quality Rating System (Five-Star System). The Five-Star System assigns each nursing home an overall rating and three component ratings—health inspections, staffing, and quality measures—based on the extent to which the nursing home meets CMS’s quality standards and other measures. The rating scale ranges from one to five stars, with more stars indicating higher quality.

The Patient Protection and Affordable Care Act directed GAO to review CMS’s Five-Star System. This report examines (1) how CMS developed and implemented the Five-Star System and what key methodological decisions were made during development, (2) the circumstances under which CMS considers modifying the Five-Star System, and (3) the extent to which CMS has established plans to help ensure it achieves its goals for the Five-Star System. To conduct this work, GAO reviewed CMS documents, interviewed CMS officials and others, and assessed whether CMS uses certain strategic planning practices.

What GAO Recommends

GAO recommends that the Administrator of CMS use strategic planning to establish how its planned efforts will help meet the goals of the Five-Star System, and develop milestones and timelines for each of its planned efforts. CMS agreed with these recommendations.

For more information, contact Linda Kohn at (202) 512-7114 or kohnl@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: CMS reported that it and other HHS operating divisions are undertaking initiatives to improve the usefulness of their websites, and Nursing Home Compare is included in this effort. CMS stated that, "As a part of this effort, CMS will work with other HHS agencies to develop a strategic plan for Nursing Home Compare that will help ensure that the site incorporates key goals and milestones that will let CMS measure its progress towards meeting those goals and accurately budget for the resources needed to meet those goals. As part of this strategic plan, CMS will develop a mechanism for receiving regular and systematic input from consumers, as well as from stakeholders, that is gathered according to accepted standards." On August 6, 2013, CMS released its strategic plan for Nursing Home Compare and the Five-Star System. This strategic plan is focused on one of CMS's four planned efforts for improving the Five-Star System--the effort to evaluate the usability of the Nursing Home Compare website, which includes the Five-Star System. This strategic plan outlines how evaluating usability aligns with the goals of the Five-Star System. For example, the plan contains intermediate goals that align to the overall goals of Nursing Home Compare and the Five-Star System. However, CMS has not developed similar planning documents for the other three efforts that CMS planned to undertake to improve the Five-Star System. Separately, a CMS official provided, through an e-mail to GAO, a summary of current activities related to these other three efforts; however, CMS did not provide evidence of any strategic planning practices or documents related to these efforts. CMS provided the following to GAO: 1. Regarding the effort to evaluate options to better distinguish among nursing homes' various care capabilities, CMS stated: "CMS has determined that there would be a significant level of effort associated with adequately distinguishing special capabilities of nursing homes and validating these capabilities on an ongoing basis. We do not currently have the resources to undertake this work. However, we are developing a tool for consumers to use when they are evaluating nursing homes for their needs or the needs of their loved ones. We plan to make this tool available for download on Nursing Home Compare. We envision that this tool will empower consumers when researching or visiting nursing homes." 2. Regarding the effort to evaluate options to include other types of nursing home staff in the staffing component rating, CMS stated: "In 2012 we began reporting on Nursing Home Compare the number of hours per resident day for physical therapists. These data, like the other staffing data currently included in the staffing component of the Five-Star Quality Rating System, are reported by the nursing homes during their recertification survey. We are also currently evaluating the possibility of including other therapy staffing, such as occupational, speech, and respiratory therapists. Moreover, we are evaluating the feasibility of reporting more real-time data in these therapy categories by using the reported amounts of therapy for each resident that is captured on the resident assessments (the Minimum Data Set). However, we have not yet determined how we might include these categories of staffing into the staffing component of the Five-Star Quality Rating System." 3. Regarding the effort to develop more quality measures for the quality measure component rating, CMS stated: "CMS is currently working on developing additional quality measures for public reporting on Nursing Home Compare. For instance, we will submit to the National Quality Forum this year a measure of rehospitalization for short-stay nursing home residents. We will also be developing a measure of hospitalization for all nursing home residents and a measure of discharge to community for short-stay residents. We also plan to develop measures related to dementia care and functional status."

    Recommendation: In order to strengthen CMS's efforts to improve the Five-Star System, the Administrator of CMS should use strategic planning practices to establish -- through planning documents -- how its planned efforts will help CMS achieve the goals of the Five-Star System.

    Agency Affected: Department of Health and Human Services: Centers for Medicare and Medicaid Services

  2. Status: Open

    Comments: CMS reported that it and other HHS operating divisions are undertaking initiatives to improve the usefulness of their websites, and Nursing Home Compare is included in this effort. CMS stated that, "As a part of this effort, CMS will work with other HHS agencies to develop a strategic plan for Nursing Home Compare that will help ensure that the site incorporates key goals and milestones that will let CMS measure its progress towards meeting those goals and accurately budget for the resources needed to meet those goals. As part of this strategic plan, CMS will develop a mechanism for receiving regular and systematic input from consumers, as well as from stakeholders, that is gathered according to accepted standards." On August 6, 2013, CMS released its strategic plan for Nursing Home Compare and the Five-Star System. This strategic plan is focused on one of CMS's four planned efforts for improving the Five-Star System--the effort to evaluate the usability of the Nursing Home Compare website, which includes the Five-Star System. While the strategic plan includes milestones for achieving this planned effort, it does not include timelines. In addition, the strategic plan does not include information on the other three efforts that CMS planned to undertake to improve the Five-Star System. Separately, a CMS official provided, through an e-mail to GAO, a summary of current activities related to these other three efforts, and while they include some milestones, there are no timelines. Additionally, CMS did not provide evidence of any strategic planning practices undertaken for these efforts. The summary CMS provided to GAO for the three efforts is as follows: 1.Regarding the effort to evaluate options to better distinguish among nursing homes' various care capabilities, CMS stated: "CMS has determined that there would be a significant level of effort associated with adequately distinguishing special capabilities of nursing homes and validating these capabilities on an ongoing basis. We do not currently have the resources to undertake this work. However, we are developing a tool for consumers to use when they are evaluating nursing homes for their needs or the needs of their loved ones. We plan to make this tool available for download on Nursing Home Compare. We envision that this tool will empower consumers when researching or visiting nursing homes." 2.Regarding the effort to evaluate options to include other types of nursing home staff in the staffing component rating, CMS stated: "In 2012 we began reporting on Nursing Home Compare the number of hours per resident day for physical therapists. These data, like the other staffing data currently included in the staffing component of the Five-Star Quality Rating System, are reported by the nursing homes during their recertification survey. We are also currently evaluating the possibility of including other therapy staffing, such as occupational, speech, and respiratory therapists. Moreover, we are evaluating the feasibility of reporting more real-time data in these therapy categories by using the reported amounts of therapy for each resident that is captured on the resident assessments (the Minimum Data Set). However, we have not yet determined how we might include these categories of staffing into the staffing component of the Five-Star Quality Rating System." 3. Regarding the effort to develop more quality measures for the quality measure component rating, CMS stated: "CMS is currently working on developing additional quality measures for public reporting on Nursing Home Compare. For instance, we will submit to the National Quality Forum this year a measure of rehospitalization for short-stay nursing home residents. We will also be developing a measure of hospitalization for all nursing home residents and a measure of discharge to community for short-stay residents. We also plan to develop measures related to dementia care and functional status."

    Recommendation: In order to strengthen CMS's efforts to improve the Five-Star System, the Administrator of CMS should use strategic planning practices to develop milestones and timelines for each of its planned efforts.

    Agency Affected: Department of Health and Human Services: Centers for Medicare and Medicaid Services

 

Explore the full database of GAO's Open Recommendations »

Sep 29, 2014

Sep 23, 2014

Sep 18, 2014

Sep 16, 2014

Sep 8, 2014

Sep 2, 2014

Aug 22, 2014

Aug 13, 2014

Aug 11, 2014

Looking for more? Browse all our products here