Chemical Demilitarization:
Actions Needed to Improve the Reliability of the Army's Cost Comparison Analysis for Treatment and Disposal Options for Newport's VX Hydrolysate
GAO-07-240R: Published: Jan 26, 2007. Publicly Released: Jan 26, 2007.
Additional Materials:
- Full Report:
- Accessible Text:
Contact:
(202) 512-3000
contact@gao.gov
Office of Public Affairs
(202) 512-4800
youngc1@gao.gov
The U.S. stockpile of 1,269 tons of VX nerve agent stored at the Newport Chemical Depot (Newport), Indiana, is one of nine stockpiles that the Department of Defense (DOD) must destroy in response to congressional direction initially provided in 1985. In addition, the stockpile must be destroyed to comply with the requirements of the Chemical Weapons Convention, which the United States became a party to in 1997. The stockpile at Newport is the first U.S. stockpile containing VX that will be destroyed by using neutralization--a process that mixes hot water and sodium hydroxide (a caustic chemical) with VX to change the chemical composition to a less toxic form. The resulting by-product is a liquid wastewater commonly referred to as hydrolysate that consists mostly of water but also has a caustic component and organic salts that need further treatment to meet Chemical Weapons Convention requirements and to meet federal and state environmental requirements for disposal. The Army, DOD's designated executive agent, began neutralizing Newport's VX stockpile on-site in May 2005 and, as of December 1, 2006, reports neutralizing about 34 percent of the stockpile. None of the generated hydrolysate--expected to be about 2 million gallons when the neutralization process is completed--has been treated. The hydrolysate is being stored on-site until a post-treatment plan can be implemented. The Army has been evaluating options for treating the hydrolysate since the mid-1990s. The John Warner National Defense Authorization Act for Fiscal Year 2007 mandated that GAO review the Army's Cost-Benefit Analysis of Off-Site Versus On-Site Treatment and Disposal of Newport Caustic Hydrolysate. Specifically, GAO (1) assessed the reasonableness of the Army's rationale to eliminate five of the eight technologies for treating Newport's hydrolysate; (2) determined what other options the Army considered, such as incineration; and (3) evaluated the adequacy of the cost comparison analysis presented for the three remaining technologies considered as alternatives to the Army's proposed plan. To meet the December 1, 2006, due date, GAO briefed or offered to brief your offices prior to that time. This report provides details of our findings and our conclusions and recommendations. GAO will also issue a separate letter on its assessment of the Army's cost-benefit analysis once DOD has completed its sensitivity review of the data in that letter.
The rationale that the Army used to eliminate five of the eight technologies for treating Newport hydrolysate appears reasonable. Based on our review of the supporting post-treatment estimate report and key National Research Council (NRC) reports referenced by the Army, there was evidence of significant difficulties associated with the five eliminated options that would make them less promising than the three others that were evaluated against the proposed DuPont option in the Army's cost comparisons. In addition to evaluating the eight alternatives discussed in its 2006 cost-benefit report, the Army previously evaluated off-site and on-site technical solutions for treating the hydrolysate, including incineration. The Army's evaluations concluded for various reasons that these alternatives would also be difficult to implement or not viable at this time. However, 5 of the 7 firms would use either of two methods: incineration or deep-well injection. Army officials believe that these two methods would garner higher levels of public concern than other methods. Moreover, while other on-site technologies were evaluated, those that were considered to be the most promising technologies in the 2001 and 2002 reports are very similar to the technologies identified in the Army's 2006 cost-benefit report. The use of any of the Army's four incinerators at its stockpile sites, while potentially technically feasible, has not been evaluated because it also has the potential for high levels of public concern, but could be evaluated if other options are no longer available. Based on our assessment of supporting documentation and analyses, we determined that the underlying cost estimates used in the Army's report were not reliable, and the impact of this on the Army's report finding that the DuPont plan had "significant cost savings" over the three considered alternatives is uncertain. Using OMB criteria and criteria approved by the cost estimating community, we determined that the estimates were unreliable because of (1) the quantity and magnitude of errors, (2) quality control weaknesses, (3) questionable or inadequate supporting source data and documentation, and (4) the undetermined sensitivity of key assumptions. Neither the Army nor the contractor has a system in place to perform cross-checks of the costs, underlying assumptions, or the technical parameters that went into the estimates. Moreover, we have determined that the results from the Army's programmatic risk analysis are unreliable because they were generated from the previously discussed unreliable cost estimates and because the Army attributed no risk to potential permitting, legal, or other challenges to the DuPont plan. Overall, we could not determine the cumulative effect of these problems on the outcome or results of the Army's analysis, in large part because we did not have confidence in much of the supporting data because of the problems that we have noted. Nevertheless, without reliable underlying cost estimates, the Army, the Congress, and the public cannot have confidence that the most cost-effective solution has been selected.
Recommendations for Executive Action
Status: Closed - Implemented
Comments: Based on our recommendation, DOD discarded its original cost-benefit analysis and agreed to conduct a new cost-benefit analysis, based on revised cost estimates, utilizing best practices to include the Association for the Advancement of Cost Engineering methods. While the revision process was underway, DOD changed its plans when the contractor associated with DOD's most favorable option dropped out of consideration for the project before the new analysis could be completed. Subsequently, DOD identified another contractor (Veolia Environmental Services) that used a technology (incineration) more proven than the ones considered in the original cost-benefit analysis. As such, it was considered cheaper than the remaining options, and had less schedule risk due to the contractor already having the necessary permits in place for disposal. DOD completed shipment to Veolia Environmental Services in Port Arthur, Texas, of all hydrolysate resulting from its neutralization process, and completed closure activities at the Newport facility in June 2010.
Recommendation: To ensure confidence in the reliability of the underlying cost estimates for the Army's decision to send hydrolysate from the Newport Chemical Depot, Indiana, off-site for treatment, which indicate significant cost and schedule savings compared to on-site disposal of the hydrolysate, the Secretary of Defense should direct the Secretary of the Army to conduct the Army's cost-benefit analysis again using best practices so that its data and conclusions are comprehensive, traceable, accurate, and credible.
Agency Affected: Department of Defense
Status: Closed - Not Implemented
Comments: DOD agreed to correct any errors when performing a new cost-benefit analysis, but a new cost-benefit analysis was not completed because a key contractor dropped out of consideration. When a new option emerged that used proven technology, this recommendation was no longer relevant.
Recommendation: To ensure confidence in the reliability of the underlying cost estimates for the Army's decision to send hydrolysate from the Newport Chemical Depot, Indiana, off-site for treatment, which indicate significant cost and schedule savings compared to on-site disposal of the hydrolysate, the Secretary of Defense should direct the Secretary of the Army to correct any technical and mathematical errors in the cost estimate.
Agency Affected: Department of Defense
Status: Closed - Not Implemented
Comments: DOD agreed to establish quality control and independent review processes when performing a new cost-benefit analysis, but a new cost-benefit analysis was not completed because a key contractor dropped out of consideration. When a new option emerged that used proven technology, this recommendation was no longer relevant.
Recommendation: To ensure confidence in the reliability of the underlying cost estimates for the Army's decision to send hydrolysate from the Newport Chemical Depot, Indiana, off-site for treatment, which indicate significant cost and schedule savings compared to on-site disposal of the hydrolysate, the Secretary of Defense should direct the Secretary of the Army to establish quality control and independent review processes that check data sources, calculations, and assumptions.
Agency Affected: Department of Defense
Status: Closed - Not Implemented
Comments: DOD agreed to perform a sensitivity analysis when performing a new cost-benefit analysis, but a new cost-benefit analysis was not completed because a key contractor dropped out of consideration. When a new option emerged that used proven technology, this recommendation was no longer relevant.
Recommendation: To ensure confidence in the reliability of the underlying cost estimates for the Army's decision to send hydrolysate from the Newport Chemical Depot, Indiana, off-site for treatment, which indicate significant cost and schedule savings compared to on-site disposal of the hydrolysate, the Secretary of Defense should direct the Secretary of the Army to perform a sensitivity analysis of key assumptions, including, at a minimum, (1) variations in the throughput rates for various options; (2) the technological uncertainty of options; and (3) for off-site treatment and disposal options, the risks associated with potential permitting, legal, and other challenges.
Agency Affected: Department of Defense
Explore the full database of GAO's Open Recommendations
»
Jan 13, 2021
-
Department of Energy Contracting:
Improvements Needed to Ensure DOE Assesses Its Full Range of Contracting Fraud RisksGAO-21-44: Published: Jan 13, 2021. Publicly Released: Jan 13, 2021.
Dec 16, 2020
-
Data Governance:
Agencies Made Progress in Establishing Governance, but Need to Address Key MilestonesGAO-21-152: Published: Dec 16, 2020. Publicly Released: Dec 16, 2020.
Dec 9, 2020
-
2020 Census:
The Bureau Concluded Field Work but Uncertainty about Data Quality, Accuracy, and Protection RemainsGAO-21-206R: Published: Dec 9, 2020. Publicly Released: Dec 9, 2020.
Dec 3, 2020
-
2020 Census:
Census Bureau Needs to Assess Data Quality Concerns Stemming from Recent Design ChangesGAO-21-142: Published: Dec 3, 2020. Publicly Released: Dec 3, 2020. -
2020 Census:
Census Bureau Needs to Ensure Transparency over Data QualityGAO-21-262T: Published: Dec 3, 2020. Publicly Released: Dec 3, 2020.
Nov 30, 2020
-
Federal Buying Power:
OMB Can Further Advance Category Management Initiative by Focusing on Requirements, Data, and TrainingGAO-21-40: Published: Nov 30, 2020. Publicly Released: Nov 30, 2020.
Nov 24, 2020
-
Disaster Response:
Agencies Should Assess Contracting Workforce Needs and Purchase Card Fraud RiskGAO-21-42: Published: Nov 24, 2020. Publicly Released: Nov 24, 2020.
Nov 23, 2020
-
Federal Contracting:
Actions Needed to Improve Department of Labor's Enforcement of Service Worker Wage ProtectionsGAO-21-11: Published: Oct 29, 2020. Publicly Released: Nov 23, 2020.
Nov 18, 2020
-
Federal Telework:
Key Practices That Can Help Ensure the Success of Telework ProgramsGAO-21-238T: Published: Nov 18, 2020. Publicly Released: Nov 18, 2020.
Oct 7, 2020
-
Unaccompanied Children:
Actions Needed to Improve Grant Application Reviews and Oversight of Care FacilitiesGAO-20-609: Published: Sep 15, 2020. Publicly Released: Oct 7, 2020.
Looking for more? Browse all our products here