Information Technology: Greater Use of Best Practices Can Reduce Risks in Acquiring Defense Health Care System
GAO-02-345
Published: Sep 26, 2002. Publicly Released: Sep 26, 2002.
Skip to Highlights
Highlights
This report examines the acquisition of the Composite Health Care System (CHCS) II. It is one in a series of reports reviewing the Department of Defense's use of best practices in acquiring information technology systems. CHCS II is expected to cost about $1 billion to deliver full capability to almost 1,100 health facilities worldwide by 2008. GAO's objectives were to determine (1) what progress has been made against project commitments, (2) whether the system has been economically justified, and (3) whether effective technical and management controls are in place.
Recommendations
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | To strengthen CHCS II investment, risk, and contract management practices, and thereby increase the chances of the department's investments in CHCS II's producing mission value commensurate with system costs, the Secretary of Defense, through the Assistant Secretary of Health Affairs, should direct the Military Health System (MHS) chief information officer (CIO) to give expanded use of best practices in managing CHCS II the attention and priority they deserve. At a minimum, the Assistant Secretary should direct the MHS CIO to, as part of the CHCS II deployment decisions, including any request to the DOD CIO for deployment approval, consider the aggregate impact on defense health affairs mission performance caused by the workarounds needed to compensate for all unresolved defects affecting the system's operational efficiency. |
Closed – Implemented
According to program officials, in late calendar year 2002, the program office produced a maintenance release for CHCS II that corrected many of the remaining bugs that required workarounds, and the limited deployment sites have that version. In addition, the Military Health System has put a standard operating procedure (SOP) in place to evaluate the effect of all workarounds required for new systems/versions before implementation. The SOP is part of the configuration control board procedures and the service components have agreed to these procedures. Finally, a test and evaluation master plan that addresses the aggregate impact of workarounds has been completed for block 2, and will be used as a template for future plans.
|
| Department of Defense | To strengthen CHCS II investment, risk, and contract management practices, and thereby increase the chances of the department's investments in CHCS II's producing mission value commensurate with system costs, the Secretary of Defense, through the Assistant Secretary of Health Affairs, should direct the MHS CIO to give expanded use of best practices in managing CHCS II the attention and priority they deserve. At a minimum, the Assistant Secretary should direct the MHS CIO to define and implement incremental investment management processes to include (1) modifying the CHCS II investment strategy to define how this approach will be implemented; (2) justifying investment in each system release before beginning detailed design and development of the release; (3) requiring that such justification be based on reliable estimates of costs, benefits, and risks; (4) measuring whether actual return-on-investment for each deployed release is in line with justification forecasts; and (5) using actual return-on investment results in deciding whether to begin detailed design and development of the next system release. |
Closed – Implemented
The Department of Defense (DOD) has developed an incrementally-based cost/benefit analysis based on the total life-cycle of CHCS II. This expanded the use of best practices in DOD and (1) identified funds pertaining to CHCS II benefits that may be planned for other Departmental priorities, (2) helped ensure effective acquistion controls are in place for CHCS II, and (3) provided cost/benefit and return on investment information, which in turn (4) provides CHCS II managers a more reliable basis for justifying continued investment in the program and (5) making sound decisions.
|
| Department of Defense | To strengthen CHCS II investment, risk, and contract management practices, and thereby increase the chances of the department's investments in CHCS II's producing mission value commensurate with system costs, the Secretary of Defense, through the Assistant Secretary of Health Affairs, should direct the MHS CIO to give expanded use of best practices in managing CHCS II the attention and priority they deserve. At a minimum, the Assistant Secretary should direct the MHS CIO to verify that the CHCS II inventory risks is complete and correct, and report this to the Assistant Secretary for Health Affairs every 6 months, along with a report on the status of all top priority risks, including each risk's probability of occurrence and impact on mission. |
Closed – Implemented
The program office updated the risk management plan (v 3.0) to require continuous risk management database updates and monthly risk reports. An initial 6-month report was provided to the Assistant Secretary in April 2003, that includes the status of all program risks, with details on priority 1 risks, including probability of occurrence and impact on mission.
|
| Department of Defense | To strengthen CHCS II investment, risk, and contract management practices, and thereby increase the chances of the department's investments in CHCS II's producing mission value commensurate with system costs, the Secretary of Defense, through the Assistant Secretary of Health Affairs, should direct the MHS CIO to give expanded use of best practices in managing CHCS II the attention and priority they deserve. At a minimum, the Assistant Secretary should direct the MHS CIO to employ performance-based contracting practices on all future CHCS II delivery orders to the maximum extent possible, including (1) defining performance standards against which deliverables can be judged, (2) developing and using quality assurance plans that describes how contractor performance against the standards will be measured, and (3) defining and using contractor incentives and penalties tied to the quality plan. |
Closed – Implemented
The CHCS II program office has begun to utilize performance-based contracting (PBC) practices on CHCS II delivery orders. For example, DOD has employed performance standards, quality assurance plans, and contractor incentives on CHCS II delivery orders providing (1) an automated health information survey tool; and (2) additional functionality for CHCS II block 1.
|
| Department of Defense | Additionally, the Secretary of Defense should direct the Assistant Secretary of Defense for Command, Control, Communications, and Intelligence, who is the designated approval authority for CHCS II, to monitor the project's use of best practices, including implementation of each of the above recommendations, and use this information to oversee the project's movement through its acquisition cycle. To this end, the Assistant Secretary, or other designated CHCS II approval authority, should not grant any request for deployment approval of any CHCS II release that is not justified by reliable analysis of the release's costs, benefits, and risks. |
Closed – Implemented
The program office updated its cost-benefit analysis in September 2002, and the Naval Center for Cost Analysis validated the cost estimate. This was used to approve the (limited) deployment of release 1 in January 2003, and is available for use by the milestone decision authority for the full deployment decision.
|
Full Report
Office of Public Affairs
Topics
Best practicesComparative benchmarking productsComputerized patient record systemCost controlInformation technologyMedical information systemsSchedule slippagesSystems designSystems managementGovernment off the shelf