Information Technology:

Greater Use of Best Practices Can Reduce Risks in Acquiring Defense Health Care System

GAO-02-345: Published: Sep 26, 2002. Publicly Released: Sep 26, 2002.

Additional Materials:

Contact:

Randolph C. Hite
(202) 512-6256
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

This report examines the acquisition of the Composite Health Care System (CHCS) II. It is one in a series of reports reviewing the Department of Defense's use of best practices in acquiring information technology systems. CHCS II is expected to cost about $1 billion to deliver full capability to almost 1,100 health facilities worldwide by 2008. GAO's objectives were to determine (1) what progress has been made against project commitments, (2) whether the system has been economically justified, and (3) whether effective technical and management controls are in place.

CHCS II is envisioned as a state-of-the-art automated medical information system allowing improved health-care decisions and lower medical and system costs. An expected highlight is computer-based patient records that doctors and other health service providers will be able to access from any military treatment facility, irrespective of location. While early CHCS II progress was limited, clear improvement has been evident over the past 2 years, as Defense has begun to embrace industry best practices. The first incremental release of the system, with a deployment decision scheduled for September 2002, is set to contain more capability than was originally planned. The current schedule is nevertheless 3 years beyond the initial estimate, due in part to major program changes and a system redesign; benefits are in question since measurement has not yet begun; and costs to date are about 2= times the 1998 estimate for deploying the first increment to one region. Until recently, Defense's basis for investing in this system has been an outdated cost/benefit analysis that did not reflect important changes in assumptions and, further, justified all system increments beyond the first solely on an economic analysis of the entire system. Officials now, however, are finalizing an updated analysis and have stated they plan to adopt an incremental approach to justifying investment in CHCS II, a best practice that successful organizations have been following for years. The department is moving forward in ensuring that effective acquisition controls are in place, but further progress can be made. Technical and management controls are largely in effect in several areas, including testing and risk management. But performance-based contracting is not being followed, resulting in the risk that CHCS II will take longer to acquire and cost more than necessary.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: To strengthen CHCS II investment, risk, and contract management practices, and thereby increase the chances of the department's investments in CHCS II's producing mission value commensurate with system costs, the Secretary of Defense, through the Assistant Secretary of Health Affairs, should direct the MHS CIO to give expanded use of best practices in managing CHCS II the attention and priority they deserve. At a minimum, the Assistant Secretary should direct the MHS CIO to employ performance-based contracting practices on all future CHCS II delivery orders to the maximum extent possible, including (1) defining performance standards against which deliverables can be judged, (2) developing and using quality assurance plans that describes how contractor performance against the standards will be measured, and (3) defining and using contractor incentives and penalties tied to the quality plan.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The CHCS II program office has begun to utilize performance-based contracting (PBC) practices on CHCS II delivery orders. For example, DOD has employed performance standards, quality assurance plans, and contractor incentives on CHCS II delivery orders providing (1) an automated health information survey tool; and (2) additional functionality for CHCS II block 1.

    Recommendation: To strengthen CHCS II investment, risk, and contract management practices, and thereby increase the chances of the department's investments in CHCS II's producing mission value commensurate with system costs, the Secretary of Defense, through the Assistant Secretary of Health Affairs, should direct the MHS CIO to give expanded use of best practices in managing CHCS II the attention and priority they deserve. At a minimum, the Assistant Secretary should direct the MHS CIO to verify that the CHCS II inventory risks is complete and correct, and report this to the Assistant Secretary for Health Affairs every 6 months, along with a report on the status of all top priority risks, including each risk's probability of occurrence and impact on mission.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The program office updated the risk management plan (v 3.0) to require continuous risk management database updates and monthly risk reports. An initial 6-month report was provided to the Assistant Secretary in April 2003, that includes the status of all program risks, with details on priority 1 risks, including probability of occurrence and impact on mission.

    Recommendation: To strengthen CHCS II investment, risk, and contract management practices, and thereby increase the chances of the department's investments in CHCS II's producing mission value commensurate with system costs, the Secretary of Defense, through the Assistant Secretary of Health Affairs, should direct the MHS CIO to give expanded use of best practices in managing CHCS II the attention and priority they deserve. At a minimum, the Assistant Secretary should direct the MHS CIO to define and implement incremental investment management processes to include (1) modifying the CHCS II investment strategy to define how this approach will be implemented; (2) justifying investment in each system release before beginning detailed design and development of the release; (3) requiring that such justification be based on reliable estimates of costs, benefits, and risks; (4) measuring whether actual return-on-investment for each deployed release is in line with justification forecasts; and (5) using actual return-on investment results in deciding whether to begin detailed design and development of the next system release.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Department of Defense (DOD) has developed an incrementally-based cost/benefit analysis based on the total life-cycle of CHCS II. This expanded the use of best practices in DOD and (1) identified funds pertaining to CHCS II benefits that may be planned for other Departmental priorities, (2) helped ensure effective acquistion controls are in place for CHCS II, and (3) provided cost/benefit and return on investment information, which in turn (4) provides CHCS II managers a more reliable basis for justifying continued investment in the program and (5) making sound decisions.

    Recommendation: To strengthen CHCS II investment, risk, and contract management practices, and thereby increase the chances of the department's investments in CHCS II's producing mission value commensurate with system costs, the Secretary of Defense, through the Assistant Secretary of Health Affairs, should direct the Military Health System (MHS) chief information officer (CIO) to give expanded use of best practices in managing CHCS II the attention and priority they deserve. At a minimum, the Assistant Secretary should direct the MHS CIO to, as part of the CHCS II deployment decisions, including any request to the DOD CIO for deployment approval, consider the aggregate impact on defense health affairs mission performance caused by the workarounds needed to compensate for all unresolved defects affecting the system's operational efficiency.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: According to program officials, in late calendar year 2002, the program office produced a maintenance release for CHCS II that corrected many of the remaining bugs that required workarounds, and the limited deployment sites have that version. In addition, the Military Health System has put a standard operating procedure (SOP) in place to evaluate the effect of all workarounds required for new systems/versions before implementation. The SOP is part of the configuration control board procedures and the service components have agreed to these procedures. Finally, a test and evaluation master plan that addresses the aggregate impact of workarounds has been completed for block 2, and will be used as a template for future plans.

    Recommendation: Additionally, the Secretary of Defense should direct the Assistant Secretary of Defense for Command, Control, Communications, and Intelligence, who is the designated approval authority for CHCS II, to monitor the project's use of best practices, including implementation of each of the above recommendations, and use this information to oversee the project's movement through its acquisition cycle. To this end, the Assistant Secretary, or other designated CHCS II approval authority, should not grant any request for deployment approval of any CHCS II release that is not justified by reliable analysis of the release's costs, benefits, and risks.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The program office updated its cost-benefit analysis in September 2002, and the Naval Center for Cost Analysis validated the cost estimate. This was used to approve the (limited) deployment of release 1 in January 2003, and is available for use by the milestone decision authority for the full deployment decision.

    Jun 10, 2014

    May 22, 2014

    May 12, 2014

    May 8, 2014

    May 7, 2014

    Apr 2, 2014

    Feb 26, 2014

    Feb 12, 2014

    Looking for more? Browse all our products here