Medicaid Demonstrations:

Evaluations Yielded Limited Results, Underscoring Need for Changes to Federal Policies and Procedures

GAO-18-220: Published: Jan 19, 2018. Publicly Released: Feb 20, 2018.

Additional Materials:

Contact:

Katherine Iritani
(202) 512-7114
iritanik@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

About one-third of Medicaid's spending goes toward demonstrations, which allow states to test new approaches to delivering Medicaid services. Do they save money? Improve care?

The short answer is that states and the federal government don't fully know. We found that the federal government did not require complete and timely evaluations from the states, so conclusive results were not available. Moreover, the federal government wasn't making its evaluation results public—missing opportunities to inform federal and state Medicaid policy discussions.

We recommended ways for the Centers for Medicare & Medicaid Services to address these issues.

Percentage of federal Medicaid spending that is under demonstration projects in each state for fiscal year 2015

Map showing 11 states with no spending on demonstrations, 10 states where 75% or more of Medicaid spending is on demonstrations.

Map showing 11 states with no spending on demonstrations, 10 states where 75% or more of Medicaid spending is on demonstrations.

Additional Materials:

Contact:

Katherine Iritani
(202) 512-7114
iritanik@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Under section 1115 of the Social Security Act, the Secretary of Health and Human Services (HHS) may approve Medicaid demonstrations to allow states to test new approaches to providing coverage and for delivering services that can transform large portions of states' programs. However, GAO found that selected states' evaluations of these demonstrations often had significant limitations that affected their usefulness in informing policy decisions. The limitations included gaps in reported evaluation results for important parts of the demonstrations. (See table.) These gaps resulted, in part, from HHS's Centers for Medicare & Medicaid Services (CMS) requiring final, comprehensive evaluation reports after the expiration of the demonstrations rather than at the end of each 3- to 5-year demonstration cycle. CMS has taken a number of steps since 2014 to improve the quality of state-led evaluations, and in October 2017, officials stated that the agency planned to require final reports at the end of each demonstration cycle for all demonstrations. However, the agency has not established written procedures for implementing such requirements, which could allow for gaps to continue. CMS also plans to allow states to conduct less rigorous evaluations for certain types of demonstrations but has not established criteria defining under what conditions limited evaluations would be allowed.

Examples of Gaps in States' Evaluations of Medicaid Section 1115 Demonstrations

Arizona

The state was required to evaluate whether providing long-term services and supports under a managed care delivery model improved access and quality of care. The evaluation report lacked information on important measures of access and quality.

Arkansas

The state was required to evaluate the effects of using Medicaid funds to purchase private insurance for more than 200,000 beneficiaries. The evaluation did not address a key hypothesis that using private insurance would improve continuity of coverage for these beneficiaries, who were expected to have frequent changes in income that could lead to coverage gaps.

Massachusetts

The state was required to evaluate the effectiveness of its approach of providing up to $690 million in incentive payments to seven hospitals to improve quality of care and reduce per capita costs. Evaluation reports submitted after 5 years provided no conclusions on the impact of the payments in these areas.

Source: GAO. | GAO-18-220

Federal evaluations led by CMS have also been limited due to data challenges that have affected the progress and scope of the work. For example, delays obtaining data directly from states, among other things, led CMS to considerably reduce the scope of a large, multi-state evaluation, which was initiated in 2014 to examine the impact of state demonstrations in four policy areas deemed to be federal priorities. Though CMS has made progress in obtaining needed data, it is uncertain when results from the multi-state and other federal evaluations will be available to policymakers because CMS has no policy for making results public. By not making these results public in a timely manner, CMS is missing an opportunity to inform important federal and state policy discussions.

Why GAO Did This Study

Demonstrations—which represented roughly a third of the more than $300 billion in federal Medicaid spending in 2015—are a powerful tool to test new approaches to providing coverage and delivering Medicaid services that could reduce costs and improve beneficiaries' outcomes. Evaluations are essential to determining whether demonstrations are having their intended effects. States are required to evaluate their demonstrations and CMS can initiate its own federal evaluations of demonstrations.

GAO was asked to examine evaluations of demonstrations, including how the results have been used to inform Medicaid policy. This report examines (1) state-led evaluations and (2) federal evaluations. GAO reviewed evaluation documentation for eight states with high demonstration expenditures that varied in the number of years their demonstrations had been in effect and by geography. GAO also reviewed documentation for the ongoing federal evaluations and interviewed state and federal Medicaid officials. GAO assessed evaluation practices against federal standards for internal control and leading evaluation guidelines.

What GAO Recommends

GAO recommends that CMS: (1) establish written procedures for requiring final evaluation reports at the end of each demonstration cycle, (2) issue criteria for when it will allow limited evaluations of demonstrations, and (3) establish a policy for publicly releasing findings from federal evaluations of demonstrations. HHS concurred with these recommendations.

For more information, contact Katherine Iritani at (202) 512-7114 or iritanik@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Administrator of CMS should establish written procedures for implementing the agency's policy that requires all states to submit a final evaluation report after the end of each demonstration cycle, regardless of renewal status. (Recommendation 1)

    Agency Affected: Department of Health and Human Services: Centers for Medicare and Medicaid Services

  2. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Administrator of CMS should issue written criteria for when CMS will allow limited evaluation of a demonstration or a portion of a demonstration, including defining conditions, such as what it means for a demonstration to be longstanding or noncomplex, as applicable. (Recommendation 2)

    Agency Affected: Department of Health and Human Services: Centers for Medicare and Medicaid Services

  3. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Administrator of CMS should establish and implement a policy for publicly releasing findings from federal evaluations of demonstrations, including findings from rapid cycle, interim, and final reports; and this policy should include standards for timely release. (Recommendation 3)

    Agency Affected: Department of Health and Human Services: Centers for Medicare and Medicaid Services

 

Explore the full database of GAO's Open Recommendations »

Apr 13, 2018

Apr 9, 2018

Apr 5, 2018

Mar 29, 2018

Mar 28, 2018

Mar 23, 2018

Mar 19, 2018

Mar 8, 2018

Looking for more? Browse all our products here