Summary
In 2003, the Centers for Disease Control and Prevention (CDC), an agency within the Department of Health and Human Services (HHS), developed an electronic syndromic surveillance system called BioSense that uses health-related data to identify patterns of disease symptoms prior to specific diagnoses. In late 2007, CDC began to redesign the program to improve collaboration with stakeholders and address identified management weaknesses. Pursuant to House Report 110-231, GAO evaluated the BioSense program, focusing on the cost and timeline estimates and performance measures and benchmarks for implementing the program, among other objectives. To accomplish this, GAO analyzed relevant program documentation and interviewed CDC officials responsible for developing and implementing BioSense.
While CDC identified annual and long-term cost and timeline estimates and performance measures for the initial design of BioSense, these estimates and measures did not reflect the implementation of its redesigned program. CDC subsequently developed a draft plan for the redesigned program that described high-level cost and timeline estimates; however, the estimates are not reliable, and the plan did not include performance measures. According to best practices, cost estimates should be well-documented, comprehensive and accurate, and must be credible before they can be considered to be reliable. However, CDC's cost estimates for the redesigned program are not reliable because they are only partially documented, are not comprehensive and accurate, and therefore are not credible. Best practices for reliable timeline estimates include the identification of resources to complete each task, establishment of a critical path, and analysis of risks to the schedule. However, the agency has not implemented these practices, resulting in timelines for the redesigned program that are not reliable. The Office of Management and Budget directs agencies to define outcome-based performance measures to gauge program results early enough for stakeholder review, and industry experts describe the need for stakeholder input in developing performance measures in order to monitor performance. While CDC established performance measures and benchmarks for the initial implementation of the BioSense program, it has not yet developed outcome-based performance measures to monitor the progress of the redesigned program and does not intend to complete their development until the end of 2009. Until program officials develop reliable cost and timeline estimates and outcome-based performance measures for the redesigned BioSense program, they will lack key components needed to effectively manage the program, increasing the risk that the agency will perpetuate weaknesses identified in its initial implementation of the program and related system.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director:
Team:
Phone:
Valerie C. Melvin
Government Accountability Office: Information Technology
(202) 512-6304
Recommendations for Executive Action
Recommendation: To ensure that CDC defines reliable plans for effectively managing the development and implementation of the redesigned BioSense program and its related system, the Director of CDC should instruct the Director of the National Center for Public Health Informatics to develop reliable cost estimates for the program that are well-documented, comprehensive and accurate, and credible while in the planning phase of the redesigned BioSense program.
Agency Affected: Department of Health and Human Services: Public Health Service: Centers for Disease Control
Status: Open
Comments: CDC provided us with documentation describing its efforts to develop reliable cost estimates for the BioSense program. The documents provided evidence that the agency had followed practices outlined in our cost estimation guide in conducting activities relevant to the early phases of its cost estimation. Nonetheless, the agency has not yet completed reliable cost estimates for the redesigned BioSense program. Its officials reported that they plan to perform several iterations of cost estimation and provided us with high-level plans for completing the final cost estimation phases.
Recommendation: To ensure that CDC defines reliable plans for effectively managing the development and implementation of the redesigned BioSense program and its related system, the Director of CDC should instruct the Director of the National Center for Public Health Informatics to develop reliable timeline estimates for implementing the program while in the planning phase of the redesigned BioSense program.
Agency Affected: Department of Health and Human Services: Public Health Service: Centers for Disease Control
Status: Open
Comments: In September 2009, CDC provided us a work breakdown structure that identified specific timeline estimates for each of the major BioSense tasks and a critical path. BioSense program officials also provided assessments of risks to the completion of the tasks identified in the work breakdown structure. However, the risk assessments were not based on statistical analyses. Specifically, the risk assessments did not identify the probability of each risk event occurring, time variances to allow for contingencies, or the expected impact on the tasks that would be affected by the occurrence of each event. Until program officials have thoroughly analyzed risks to the timelines defined by the work breakdown structure, they will not have completed activities that are essential to the development of reliable timeline estimates for implementing the redesigned program.
Recommendation: To ensure that CDC defines reliable plans for effectively managing the development and implementation of the redesigned BioSense program and its related system, the Director of CDC should, with stakeholder input, instruct the Director of the National Center for Public Health Informatics to develop outcome-based performance measures that address all phases of the program and that focus the success of the program on CDC's federal, state, and local partners' use of the system for responding to public health events while in the planning phase of the redesigned BioSense program.
Agency Affected: Department of Health and Human Services: Public Health Service: Centers for Disease Control
Status: Open
Comments: In July 2009, CDC provided us a strategic plan for the BioSense program that included information on each of the program's goals and strategies for fiscal years 2008-2012. The strategic plan included milestones that matched up with each strategy, and a grid that showed the years in which each milestone would be relevant. However, the plan did not contain outcome-based performance measures for achieving the program's goals. CDC noted that the intent of the BioSense strategic plan was to provide direction for a future detailed program plan that is intended to include such performance measures.
