Healthcare.gov:

CMS Has Taken Steps to Address Problems, but Needs to Further Implement Systems Development Best Practices

GAO-15-238: Published: Mar 4, 2015. Publicly Released: Mar 4, 2015.

Additional Materials:

Contact:

Valerie C. Melvin
(202) 512-6304
melvinv@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Several problems with the initial development and deployment of Healthcare.gov and its supporting systems led to consumers encountering widespread performance issues when trying to create accounts and enroll in health plans:

Inadequate capacity planning: The Centers for Medicare & Medicaid Services (CMS) did not plan for adequate capacity to support Healthcare.gov and its supporting systems.

Software coding errors: CMS and its contractors identified errors in the software code for Healthcare.gov and its supporting systems, but did not adequately correct them prior to launch.

Lack of functionality: CMS had not implemented all planned functionality prior to the initial launch of Healthcare.gov and its supporting systems.

Since the initial launch, CMS has taken steps to address these problems, including increasing capacity, requiring additional software quality reviews, and awarding a new contract to complete development and improve the functionality of key systems. After it took these actions, performance issues affecting Healthcare.gov and its supporting systems were significantly reduced.

In addition, CMS did not consistently apply recognized best practices for system development, which contributed to the problems with the initial launch of Healthcare.gov and its supporting systems.

Requirements were not effectively managed: Requirements management helps ensure that a project's plans and work products are aligned with the needs of users. However, CMS did not always ensure that requirements were approved and were linked to source and lower-level requirements. As a result, CMS was hindered in ensuring that expected functionality for the system was delivered.

System testing was inconsistent. Testing is essential for ensuring that a system operates as intended. However, Healthcare.gov and its supporting systems were not fully tested prior to launch, and test documentation was missing key elements such as criteria for determining whether a system passed a test. Thus, CMS's assurance that these systems would perform as intended was limited.

Project oversight was not effective. Oversight includes monitoring a project's progress and taking corrective actions when its performance deviates from what is planned. However, CMS's oversight was limited by an unreliable schedule, lack of estimates of work needed to complete the project, unorganized and outdated project documentation, and inconsistent reviews of project progress.

As it has undertaken further development, CMS has made improvements in some of these areas, by, for example, establishing new requirements management processes and improving test documentation. However, weaknesses remain in its application of requirements, testing, and oversight practices. In addition, the Department of Health and Human Services (HHS) has not provided adequate oversight of the Healthcare.gov initiative through its Office of the Chief Information Officer. The Office of Management and Budget's (OMB) oversight role was limited, and GAO has previously recommended that it improve oversight of IT projects' performance.

Why GAO Did This Study

The Patient Protection and Affordable Care Act required the establishment of health insurance marketplaces to assist individuals in obtaining health insurance coverage. CMS, a component of HHS, was responsible for establishing a federally facilitated marketplace for states that elected not to establish their own. This marketplace is supported by an array of IT systems, which are to facilitate enrollment in qualifying health plans. These include Healthcare.gov, the website that serves as the consumer portal to the marketplace, as well as systems for establishing user accounts, verifying eligibility, and facilitating enrollment.

GAO was asked to review CMS's management of the development of IT systems supporting the federal marketplace. Its objectives were to (1) describe problems encountered in developing and deploying systems supporting Healthcare.gov and determine the status of efforts to address deficiencies and (2) determine the extent to which CMS applied disciplined practices for managing and overseeing the development effort, and the extent to which HHS and OMB provided oversight. To do this, GAO reviewed program documentation and interviewed relevant CMS and other officials.

What GAO Recommends

GAO is recommending that CMS take seven actions to implement improvements in its requirements management, system testing, and project oversight, and that HHS improve its oversight of the Healthcare.gov effort. HHS concurred with all of the recommendations.

For more information, contact Valerie C. Melvin at (202) 512-6304 or melvinv@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In December 2016, we verified that, in response to our recommendation, CMS had documented an approval process for functional and technical design requirements. As a result of these actions, the agency is better positioned to ensure that requirements are understood and approved by system stakeholders.

    Recommendation: To improve requirements management for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to document the approval process for functional and technical design requirements documentation.

    Agency Affected: Department of Health and Human Services

  2. Status: Closed - Implemented

    Comments: In January 2017, we verified that, in response to our recommendation, CMS had implemented an approval process and obtained the signatures of key stakeholders when developing business, functional, and technical requirements. As a result of these actions, CMS is better positioned to ensure that requirements for Healthcare.gov systems are clearly defined, agreed upon, and approved before development begins.

    Recommendation: To improve requirements management for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to implement the CMS procedure to obtain signatures from the three key stakeholders--the CMS business owner, the CMS approval authority, and the contractor organization approving authority--to ensure that stakeholders have a shared understanding of all business, functional, and technical requirements for systems supporting Healthcare.gov prior to developing them.

    Agency Affected: Department of Health and Human Services

  3. Status: Closed - Implemented

    Comments: In May 2017, we verified that CMS met the intent of our recommendation by requiring the use of certain tools for systems testing and requiring stakeholder review of systems test documentation. As a result of these actions, the agency is better positioned to ensure that testing was properly carried out and the system will function as intended.

    Recommendation: To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to document and approve systems testing policy and procedures, including (1) the use of the system testing tool designed to integrate systems development and systems testing and (2) requirements for stakeholder review of systems test documentation that is intended to ensure proper test coverage and to validate the results.

    Agency Affected: Department of Health and Human Services

  4. Status: Closed - Implemented

    Comments: In May 2017, we verified that CMS met the intent of our recommendation by requiring review and approval of testing documentation and defining roles and responsibilities of those carrying out the testing. As a result of these actions, the agency is better positioned to ensure that testing will be consistently executed and of sufficient quality to validate the system will address requirements.

    Recommendation: To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to require key information in system test plans, as recommended by best practices, including the means by which the quality of testing processes will be assured, and the identification of responsibilities for individuals or groups carrying out testing.

    Agency Affected: Department of Health and Human Services

  5. Status: Closed - Implemented

    Comments: In December 2016, we verified that, in response to our recommendation, CMS had incorporated key elements defined in best practices, such as expected testing outputs, inputs required to execute the test cases, and information about whether each test item has passed or failed elements defined in best practices into their test cases we reviewed. As a result, the agency is better positioned to ensure the systems supporting Healthcare.gov are performing as intended.

    Recommendation: To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to require and ensure key information is included in test cases, as recommended by best practices, such as all outputs and exact values; test case dependencies; inputs required to execute each test case; and information about whether each test item has passed or failed testing.

    Agency Affected: Department of Health and Human Services

  6. Status: Closed - Implemented

    Comments: In October 2015, we verified that, in response to our recommendation, CMS had improved the development of schedules by, in general, logically sequencing all work activities, ensuring a valid critical path, and reflecting a reasonable total float. As a result, the agency's project schedules should now be a more effective tool for gauging project progress and estimates of timelines for completing project activities.

    Recommendation: To improve oversight processes for systems development activities related to systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to ensure schedules for the Healthcare.gov effort are well constructed by, among other things, (1) logically sequencing activities, (2) confirming the critical paths are valid, and (3) identifying reasonable total float.

    Agency Affected: Department of Health and Human Services

  7. Status: Closed - Implemented

    Comments: In September 2015, we verified that CMS, in response to our recommendation, had developed and documented procedures for estimating levels of effort and had defined how levels of effort are to be used to monitor system development progress. By taking these actions, the agency should be better positioned to ensure that level-of-effort estimations are applied in a consistent manner and provide for more accurate monitoring of progress.

    Recommendation: To improve oversight processes for systems development activities related to systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to develop and implement policy and procedures for estimating level of effort to ensure effort is estimated at the appropriate level (requirements or program area), and define how levels of effort will be used to monitor system development progress.

    Agency Affected: Department of Health and Human Services

  8. Status: Closed - Implemented

    Comments: In May 2017, we verified that the HHS IT Steering Committee, which serves as the department's IT investment review board for projects such as Healthcare.gov, was actively reviewing projects. As a result of these actions, the agency is better able to exercise their oversight role and may be better able to take corrective actions on poorly performing projects.

    Recommendation: To improve oversight for Healthcare.gov and its supporting systems, the Secretary of Health and Human Services should direct the Department of Health and Human Services' Chief Information Officer to carry out authorized oversight responsibilities. Specifically, the Chief Information Officer should ensure the department-wide investment review board is active and carrying out responsibilities for overseeing the performance of high-risk IT investments such as those related to Healthcare.gov.

    Agency Affected: Department of Health and Human Services

 

Explore the full database of GAO's Open Recommendations »

Nov 15, 2017

Nov 14, 2017

Nov 13, 2017

Nov 6, 2017

Oct 31, 2017

Oct 18, 2017

Oct 4, 2017

Looking for more? Browse all our products here