Healthcare.gov:

CMS Has Taken Steps to Address Problems, but Needs to Further Implement Systems Development Best Practices

GAO-15-238: Published: Mar 4, 2015. Publicly Released: Mar 4, 2015.

Additional Materials:

Contact:

Valerie C. Melvin
(202) 512-6304
melvinv@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Several problems with the initial development and deployment of Healthcare.gov and its supporting systems led to consumers encountering widespread performance issues when trying to create accounts and enroll in health plans:

Inadequate capacity planning: The Centers for Medicare & Medicaid Services (CMS) did not plan for adequate capacity to support Healthcare.gov and its supporting systems.

Software coding errors: CMS and its contractors identified errors in the software code for Healthcare.gov and its supporting systems, but did not adequately correct them prior to launch.

Lack of functionality: CMS had not implemented all planned functionality prior to the initial launch of Healthcare.gov and its supporting systems.

Since the initial launch, CMS has taken steps to address these problems, including increasing capacity, requiring additional software quality reviews, and awarding a new contract to complete development and improve the functionality of key systems. After it took these actions, performance issues affecting Healthcare.gov and its supporting systems were significantly reduced.

In addition, CMS did not consistently apply recognized best practices for system development, which contributed to the problems with the initial launch of Healthcare.gov and its supporting systems.

Requirements were not effectively managed: Requirements management helps ensure that a project's plans and work products are aligned with the needs of users. However, CMS did not always ensure that requirements were approved and were linked to source and lower-level requirements. As a result, CMS was hindered in ensuring that expected functionality for the system was delivered.

System testing was inconsistent. Testing is essential for ensuring that a system operates as intended. However, Healthcare.gov and its supporting systems were not fully tested prior to launch, and test documentation was missing key elements such as criteria for determining whether a system passed a test. Thus, CMS's assurance that these systems would perform as intended was limited.

Project oversight was not effective. Oversight includes monitoring a project's progress and taking corrective actions when its performance deviates from what is planned. However, CMS's oversight was limited by an unreliable schedule, lack of estimates of work needed to complete the project, unorganized and outdated project documentation, and inconsistent reviews of project progress.

As it has undertaken further development, CMS has made improvements in some of these areas, by, for example, establishing new requirements management processes and improving test documentation. However, weaknesses remain in its application of requirements, testing, and oversight practices. In addition, the Department of Health and Human Services (HHS) has not provided adequate oversight of the Healthcare.gov initiative through its Office of the Chief Information Officer. The Office of Management and Budget's (OMB) oversight role was limited, and GAO has previously recommended that it improve oversight of IT projects' performance.

Why GAO Did This Study

The Patient Protection and Affordable Care Act required the establishment of health insurance marketplaces to assist individuals in obtaining health insurance coverage. CMS, a component of HHS, was responsible for establishing a federally facilitated marketplace for states that elected not to establish their own. This marketplace is supported by an array of IT systems, which are to facilitate enrollment in qualifying health plans. These include Healthcare.gov, the website that serves as the consumer portal to the marketplace, as well as systems for establishing user accounts, verifying eligibility, and facilitating enrollment.

GAO was asked to review CMS's management of the development of IT systems supporting the federal marketplace. Its objectives were to (1) describe problems encountered in developing and deploying systems supporting Healthcare.gov and determine the status of efforts to address deficiencies and (2) determine the extent to which CMS applied disciplined practices for managing and overseeing the development effort, and the extent to which HHS and OMB provided oversight. To do this, GAO reviewed program documentation and interviewed relevant CMS and other officials.

What GAO Recommends

GAO is recommending that CMS take seven actions to implement improvements in its requirements management, system testing, and project oversight, and that HHS improve its oversight of the Healthcare.gov effort. HHS concurred with all of the recommendations.

For more information, contact Valerie C. Melvin at (202) 512-6304 or melvinv@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: In June 2016, HHS officials stated that they had documented the approval process for business and functional requirements documentation and provided us the supporting evidence. The officials stated that similar documentation is to be developed for system design documents and technical design documents as key aspects of the project move from the requirements phase to the design phase. Thus, this recommendation will remain open until HHS can provide documentation of the requirements for approval of technical design documents.

    Recommendation: To improve requirements management for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to document the approval process for functional and technical design requirements documentation.

    Agency Affected: Department of Health and Human Services

  2. Status: Open

    Comments: In June 2016, HHS officials stated that they had instituted a process that requires CMS to obtain signatures from CMS business owners, as well as CMS system owners or other approving authorities. The officials stated that they intend to improve the process by also requiring signatures from the contractor-approving authority. Further, the officials provided documentation of the requirements for approving business requirements and added that similar documentation is to be developed for functional design, system design, and technical design approval as key aspects of the project move from the requirements phase to the design phase. In light of these planned activities, this recommendation will remain open until HHS provides evidence that the department is requiring signatures from contractor-approving authorities and has instituted an approval process for functional and technical requirements.

    Recommendation: To improve requirements management for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to implement the CMS procedure to obtain signatures from the three key stakeholders--the CMS business owner, the CMS approval authority, and the contractor organization approving authority--to ensure that stakeholders have a shared understanding of all business, functional, and technical requirements for systems supporting Healthcare.gov prior to developing them.

    Agency Affected: Department of Health and Human Services

  3. Status: Open

    Comments: In a June 2016 update on the status on this recommendation, HHS officials stated that the agency still considered this recommendation to be open.

    Recommendation: To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to document and approve systems testing policy and procedures, including (1) the use of the system testing tool designed to integrate systems development and systems testing and (2) requirements for stakeholder review of systems test documentation that is intended to ensure proper test coverage and to validate the results.

    Agency Affected: Department of Health and Human Services

  4. Status: Open

    Comments: In June 2016, CMS officials provided a test plan that included information about how the quality of testing processes would be assured and the identification of responsibilities for individuals or groups carrying out the testing. However, CMS officials have not provided specific guidance or a documented procedure to indicate that the agency requires this information to be included in all test plans, although our recommendation specifically stated that CMS should include such information in test plans. Therefore, this recommendation will remain open for further update on the agency's actions in this regard.

    Recommendation: To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to require key information in system test plans, as recommended by best practices, including the means by which the quality of testing processes will be assured, and the identification of responsibilities for individuals or groups carrying out testing.

    Agency Affected: Department of Health and Human Services

  5. Status: Open

    Comments: As of June 2016, HHS had not completed its actions on this recommendation and agency officials stated that they considered this recommendation to be open.

    Recommendation: To improve systems testing processes for future development covering systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to require and ensure key information is included in test cases, as recommended by best practices, such as all outputs and exact values; test case dependencies; inputs required to execute each test case; and information about whether each test item has passed or failed testing.

    Agency Affected: Department of Health and Human Services

  6. Status: Closed - Implemented

    Comments: In October 2015, we verified that, in response to our recommendation, CMS had improved the development of schedules by, in general, logically sequencing all work activities, ensuring a valid critical path, and reflecting a reasonable total float. As a result, the agency's project schedules should now be a more effective tool for gauging project progress and estimates of timelines for completing project activities.

    Recommendation: To improve oversight processes for systems development activities related to systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to ensure schedules for the Healthcare.gov effort are well constructed by, among other things, (1) logically sequencing activities, (2) confirming the critical paths are valid, and (3) identifying reasonable total float.

    Agency Affected: Department of Health and Human Services

  7. Status: Closed - Implemented

    Comments: In September 2015, we verified that CMS, in response to our recommendation, had developed and documented procedures for estimating levels of effort and had defined how levels of effort are to be used to monitor system development progress. By taking these actions, the agency should be better positioned to ensure that level-of-effort estimations are applied in a consistent manner and provide for more accurate monitoring of progress.

    Recommendation: To improve oversight processes for systems development activities related to systems supporting Healthcare.gov, the Secretary of Health and Human Services should direct the Administrator of the Centers for Medicare & Medicaid Services to direct the Chief Information Officer to develop and implement policy and procedures for estimating level of effort to ensure effort is estimated at the appropriate level (requirements or program area), and define how levels of effort will be used to monitor system development progress.

    Agency Affected: Department of Health and Human Services

  8. Status: Open

    Comments: GAO requested that HHS provide updates on the status of this recommendation on July 13, 2015, September 4, 2015, and June 17, 2016. However, the agency has not yet responded to our requests. Thus, this recommendation will remain open.

    Recommendation: To improve oversight for Healthcare.gov and its supporting systems, the Secretary of Health and Human Services should direct the Department of Health and Human Services' Chief Information Officer to carry out authorized oversight responsibilities. Specifically, the Chief Information Officer should ensure the department-wide investment review board is active and carrying out responsibilities for overseeing the performance of high-risk IT investments such as those related to Healthcare.gov.

    Agency Affected: Department of Health and Human Services

 

Explore the full database of GAO's Open Recommendations »

Aug 26, 2016

Aug 5, 2016

Aug 1, 2016

Jul 7, 2016

Jun 23, 2016

Jun 20, 2016

Jun 15, 2016

Looking for more? Browse all our products here