Healthcare.gov:

CMS Has Taken Steps to Address Problems, but Needs to Further Implement Systems Development Best Practices

GAO-15-238: Published: Mar 4, 2015. Publicly Released: Mar 4, 2015.

Additional Materials:

Contact:

Valerie C. Melvin
(202) 512-6304
melvinv@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Several problems with the initial development and deployment of Healthcare.gov and its supporting systems led to consumers encountering widespread performance issues when trying to create accounts and enroll in health plans:

Inadequate capacity planning: The Centers for Medicare & Medicaid Services (CMS) did not plan for adequate capacity to support Healthcare.gov and its supporting systems.

Software coding errors: CMS and its contractors identified errors in the software code for Healthcare.gov and its supporting systems, but did not adequately correct them prior to launch.

Lack of functionality: CMS had not implemented all planned functionality prior to the initial launch of Healthcare.gov and its supporting systems.

Since the initial launch, CMS has taken steps to address these problems, including increasing capacity, requiring additional software quality reviews, and awarding a new contract to complete development and improve the functionality of key systems. After it took these actions, performance issues affecting Healthcare.gov and its supporting systems were significantly reduced.

In addition, CMS did not consistently apply recognized best practices for system development, which contributed to the problems with the initial launch of Healthcare.gov and its supporting systems.

Requirements were not effectively managed: Requirements management helps ensure that a project's plans and work products are aligned with the needs of users. However, CMS did not always ensure that requirements were approved and were linked to source and lower-level requirements. As a result, CMS was hindered in ensuring that expected functionality for the system was delivered.

System testing was inconsistent. Testing is essential for ensuring that a system operates as intended. However, Healthcare.gov and its supporting systems were not fully tested prior to launch, and test documentation was missing key elements such as criteria for determining whether a system passed a test. Thus, CMS's assurance that these systems would perform as intended was limited.

Project oversight was not effective. Oversight includes monitoring a project's progress and taking corrective actions when its performance deviates from what is planned. However, CMS's oversight was limited by an unreliable schedule, lack of estimates of work needed to complete the project, unorganized and outdated project documentation, and inconsistent reviews of project progress.

As it has undertaken further development, CMS has made improvements in some of these areas, by, for example, establishing new requirements management processes and improving test documentation. However, weaknesses remain in its application of requirements, testing, and oversight practices. In addition, the Department of Health and Human Services (HHS) has not provided adequate oversight of the Healthcare.gov initiative through its Office of the Chief Information Officer. The Office of Management and Budget's (OMB) oversight role was limited, and GAO has previously recommended that it improve oversight of IT projects' performance.

Why GAO Did This Study

The Patient Protection and Affordable Care Act required the establishment of health insurance marketplaces to assist individuals in obtaining health insurance coverage. CMS, a component of HHS, was responsible for establishing a federally facilitated marketplace for states that elected not to establish their own. This marketplace is supported by an array of IT systems, which are to facilitate enrollment in qualifying health plans. These include Healthcare.gov, the website that serves as the consumer portal to the marketplace, as well as systems for establishing user accounts, verifying eligibility, and facilitating enrollment.

GAO was asked to review CMS's management of the development of IT systems supporting the federal marketplace. Its objectives were to (1) describe problems encountered in developing and deploying systems supporting Healthcare.gov and determine the status of efforts to address deficiencies and (2) determine the extent to which CMS applied disciplined practices for managing and overseeing the development effort, and the extent to which HHS and OMB provided oversight. To do this, GAO reviewed program documentation and interviewed relevant CMS and other officials.

What GAO Recommends

GAO is recommending that CMS take seven actions to implement improvements in its requirements management, system testing, and project oversight, and that HHS improve its oversight of the Healthcare.gov effort. HHS concurred with all of the recommendations.

For more information, contact Valerie C. Melvin at (202) 512-6304 or melvinv@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: On May 7, 2015, HHS officials stated that they had documented the approval process for business and functional requirements documentation and provided supporting documentation. However, officials added that "Similar documentation will be developed for System Design Documents and Technical Design Documents as key aspects of the project move from the Requirements Phase to the Design Phase." Therefore, this recommendation will remain open until HHS can provide documentation on the requirements for technical design documentation approval.

    Recommendation: To improve requirements management for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to document the approval process for functional and technical design requirements documentation.

    Agency Affected: Department of Health and Human Services

  2. Status: Open

    Comments: On May 7, 2015, HHS officials stated that they had completed instituting a process that requires CMS to obtain signatures from CMS business owners, as well as CMS system owners or other approving authority. They intend to improve the process by also requiring signatures from the contractor-approving authority. They provided documentation of the documented requirements for approving business requirements. However, HHS officials added that "Similar documentation will be developed for Functional Design Documents, System Design Documents, and Technical Design Documents as key aspects of the project move from the Requirements Phase to the Design Phase." Therefore, this recommendation will remain open until HHS provides evidence that they have instituted a process for functional and technical requirements.

    Recommendation: To improve requirements management for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to implement the CMS procedure to obtain signatures from the three key stakeholders--the CMS business owner, the CMS approval authority, and the contractor organization approving authority--to ensure that stakeholders have a shared understanding of all business, functional, and technical requirements for systems supporting Healthcare.gov prior to developing them.

    Agency Affected: Department of Health and Human Services

  3. Status: Open

    Comments: On May 7, 2015, HHS officials provided an updated status on this recommendation. They still consider this recommendation open.

    Recommendation: To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to document and approve systems testing policy and procedures, including (1) the use of the system testing tool designed to integrate systems development and systems testing and (2) requirements for stakeholder review of systems test documentation that is intended to ensure proper test coverage and to validate the results.

    Agency Affected: Department of Health and Human Services

  4. Status: Open

    Comments: On August 6, 2015 CMS officials provided a test plan that included information about how quality of testing processes would be assured and the identification of responsibilities for individuals or groups carrying out testing. However, the recommendation specifically stated that CMS should require this information in test plans and CMS officials did not provide a specific guide or procedure that requires all test plans include this information. Therefore, this recommendation will remain open.

    Recommendation: To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to require key information in system test plans, as recommended by best practices, including the means by which the quality of testing processes will be assured, and the identification of responsibilities for individuals or groups carrying out testing.

    Agency Affected: Department of Health and Human Services

  5. Status: Open

    Comments: On May 7, 2015, HHS officials stated that they consider this recommendation to be open.

    Recommendation: To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to require and ensure key information is included in test cases, as recommended by best practices, such as all outputs and exact values; test case dependencies; inputs required to execute each test case; and information about whether each test item has passed or failed testing.

    Agency Affected: Department of Health and Human Services

  6. Status: Open

    Comments: On August 6, 2015 CMS officials provided schedules that they believe are well constructed. GAO is currently analyzing the schedules.

    Recommendation: To improve oversight processes for systems development activities related to systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to ensure schedules for the Healthcare.gov effort are well constructed by, among other things, (1) logically sequencing activities, (2) confirming the critical paths are valid, and (3) identifying reasonable total float.

    Agency Affected: Department of Health and Human Services

  7. Status: Open

    Comments: On May 7, 2015, CMS officials provided procedures they believe detail requirements for estimating level of effort and how they intend to use the estimates to monitor system development progress. GAO is currently in the process of evaluating the documentation to determine if it addresses our recommendation.

    Recommendation: To improve oversight processes for systems development activities related to systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to develop and implement policy and procedures for estimating level of effort to ensure effort is estimated at the appropriate level (requirements or program area), and define how levels of effort will be used to monitor system development progress.

    Agency Affected: Department of Health and Human Services

  8. Status: Open

    Comments: GAO requested the status of this recommendation on July 13, 2015 and September 4, 2015. HHS has not yet responded, so this recommendation will remain open.

    Recommendation: To improve oversight for Healthcare.gov and its supporting systems, the Secretary of Health and Human Services should direct the Department of Health and Human Services' Chief Information Officer to carry out authorized oversight responsibilities. Specifically, the Chief Information Officer should ensure the department-wide investment review board is active and carrying out responsibilities for overseeing the performance of high-risk IT investments such as those related to Healthcare.gov.

    Agency Affected: Department of Health and Human Services

 

Explore the full database of GAO's Open Recommendations »

May 24, 2016

May 13, 2016

May 11, 2016

May 9, 2016

May 6, 2016

Apr 28, 2016

Apr 19, 2016

Apr 18, 2016

Looking for more? Browse all our products here