Chemical Demilitarization:

Actions Needed to Improve the Reliability of the Army's Cost Comparison Analysis for Treatment and Disposal Options for Newport's VX Hydrolysate

GAO-07-240R: Published: Jan 26, 2007. Publicly Released: Jan 26, 2007.

Additional Materials:

Contact:

Davi M. Dagostino
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The U.S. stockpile of 1,269 tons of VX nerve agent stored at the Newport Chemical Depot (Newport), Indiana, is one of nine stockpiles that the Department of Defense (DOD) must destroy in response to congressional direction initially provided in 1985. In addition, the stockpile must be destroyed to comply with the requirements of the Chemical Weapons Convention, which the United States became a party to in 1997. The stockpile at Newport is the first U.S. stockpile containing VX that will be destroyed by using neutralization--a process that mixes hot water and sodium hydroxide (a caustic chemical) with VX to change the chemical composition to a less toxic form. The resulting by-product is a liquid wastewater commonly referred to as hydrolysate that consists mostly of water but also has a caustic component and organic salts that need further treatment to meet Chemical Weapons Convention requirements and to meet federal and state environmental requirements for disposal. The Army, DOD's designated executive agent, began neutralizing Newport's VX stockpile on-site in May 2005 and, as of December 1, 2006, reports neutralizing about 34 percent of the stockpile. None of the generated hydrolysate--expected to be about 2 million gallons when the neutralization process is completed--has been treated. The hydrolysate is being stored on-site until a post-treatment plan can be implemented. The Army has been evaluating options for treating the hydrolysate since the mid-1990s. The John Warner National Defense Authorization Act for Fiscal Year 2007 mandated that GAO review the Army's Cost-Benefit Analysis of Off-Site Versus On-Site Treatment and Disposal of Newport Caustic Hydrolysate. Specifically, GAO (1) assessed the reasonableness of the Army's rationale to eliminate five of the eight technologies for treating Newport's hydrolysate; (2) determined what other options the Army considered, such as incineration; and (3) evaluated the adequacy of the cost comparison analysis presented for the three remaining technologies considered as alternatives to the Army's proposed plan. To meet the December 1, 2006, due date, GAO briefed or offered to brief your offices prior to that time. This report provides details of our findings and our conclusions and recommendations. GAO will also issue a separate letter on its assessment of the Army's cost-benefit analysis once DOD has completed its sensitivity review of the data in that letter.

The rationale that the Army used to eliminate five of the eight technologies for treating Newport hydrolysate appears reasonable. Based on our review of the supporting post-treatment estimate report and key National Research Council (NRC) reports referenced by the Army, there was evidence of significant difficulties associated with the five eliminated options that would make them less promising than the three others that were evaluated against the proposed DuPont option in the Army's cost comparisons. In addition to evaluating the eight alternatives discussed in its 2006 cost-benefit report, the Army previously evaluated off-site and on-site technical solutions for treating the hydrolysate, including incineration. The Army's evaluations concluded for various reasons that these alternatives would also be difficult to implement or not viable at this time. However, 5 of the 7 firms would use either of two methods: incineration or deep-well injection. Army officials believe that these two methods would garner higher levels of public concern than other methods. Moreover, while other on-site technologies were evaluated, those that were considered to be the most promising technologies in the 2001 and 2002 reports are very similar to the technologies identified in the Army's 2006 cost-benefit report. The use of any of the Army's four incinerators at its stockpile sites, while potentially technically feasible, has not been evaluated because it also has the potential for high levels of public concern, but could be evaluated if other options are no longer available. Based on our assessment of supporting documentation and analyses, we determined that the underlying cost estimates used in the Army's report were not reliable, and the impact of this on the Army's report finding that the DuPont plan had "significant cost savings" over the three considered alternatives is uncertain. Using OMB criteria and criteria approved by the cost estimating community, we determined that the estimates were unreliable because of (1) the quantity and magnitude of errors, (2) quality control weaknesses, (3) questionable or inadequate supporting source data and documentation, and (4) the undetermined sensitivity of key assumptions. Neither the Army nor the contractor has a system in place to perform cross-checks of the costs, underlying assumptions, or the technical parameters that went into the estimates. Moreover, we have determined that the results from the Army's programmatic risk analysis are unreliable because they were generated from the previously discussed unreliable cost estimates and because the Army attributed no risk to potential permitting, legal, or other challenges to the DuPont plan. Overall, we could not determine the cumulative effect of these problems on the outcome or results of the Army's analysis, in large part because we did not have confidence in much of the supporting data because of the problems that we have noted. Nevertheless, without reliable underlying cost estimates, the Army, the Congress, and the public cannot have confidence that the most cost-effective solution has been selected.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: DOD agreed to establish quality control and independent review processes when performing a new cost-benefit analysis, but a new cost-benefit analysis was not completed because a key contractor dropped out of consideration. When a new option emerged that used proven technology, this recommendation was no longer relevant.

    Recommendation: To ensure confidence in the reliability of the underlying cost estimates for the Army's decision to send hydrolysate from the Newport Chemical Depot, Indiana, off-site for treatment, which indicate significant cost and schedule savings compared to on-site disposal of the hydrolysate, the Secretary of Defense should direct the Secretary of the Army to establish quality control and independent review processes that check data sources, calculations, and assumptions.

    Agency Affected: Department of Defense

  2. Status: Closed - Not Implemented

    Comments: DOD agreed to correct any errors when performing a new cost-benefit analysis, but a new cost-benefit analysis was not completed because a key contractor dropped out of consideration. When a new option emerged that used proven technology, this recommendation was no longer relevant.

    Recommendation: To ensure confidence in the reliability of the underlying cost estimates for the Army's decision to send hydrolysate from the Newport Chemical Depot, Indiana, off-site for treatment, which indicate significant cost and schedule savings compared to on-site disposal of the hydrolysate, the Secretary of Defense should direct the Secretary of the Army to correct any technical and mathematical errors in the cost estimate.

    Agency Affected: Department of Defense

  3. Status: Closed - Implemented

    Comments: Based on our recommendation, DOD discarded its original cost-benefit analysis and agreed to conduct a new cost-benefit analysis, based on revised cost estimates, utilizing best practices to include the Association for the Advancement of Cost Engineering methods. While the revision process was underway, DOD changed its plans when the contractor associated with DOD's most favorable option dropped out of consideration for the project before the new analysis could be completed. Subsequently, DOD identified another contractor (Veolia Environmental Services) that used a technology (incineration) more proven than the ones considered in the original cost-benefit analysis. As such, it was considered cheaper than the remaining options, and had less schedule risk due to the contractor already having the necessary permits in place for disposal. DOD completed shipment to Veolia Environmental Services in Port Arthur, Texas, of all hydrolysate resulting from its neutralization process, and completed closure activities at the Newport facility in June 2010.

    Recommendation: To ensure confidence in the reliability of the underlying cost estimates for the Army's decision to send hydrolysate from the Newport Chemical Depot, Indiana, off-site for treatment, which indicate significant cost and schedule savings compared to on-site disposal of the hydrolysate, the Secretary of Defense should direct the Secretary of the Army to conduct the Army's cost-benefit analysis again using best practices so that its data and conclusions are comprehensive, traceable, accurate, and credible.

    Agency Affected: Department of Defense

  4. Status: Closed - Not Implemented

    Comments: DOD agreed to perform a sensitivity analysis when performing a new cost-benefit analysis, but a new cost-benefit analysis was not completed because a key contractor dropped out of consideration. When a new option emerged that used proven technology, this recommendation was no longer relevant.

    Recommendation: To ensure confidence in the reliability of the underlying cost estimates for the Army's decision to send hydrolysate from the Newport Chemical Depot, Indiana, off-site for treatment, which indicate significant cost and schedule savings compared to on-site disposal of the hydrolysate, the Secretary of Defense should direct the Secretary of the Army to perform a sensitivity analysis of key assumptions, including, at a minimum, (1) variations in the throughput rates for various options; (2) the technological uncertainty of options; and (3) for off-site treatment and disposal options, the risks associated with potential permitting, legal, and other challenges.

    Agency Affected: Department of Defense

 

Explore the full database of GAO's Open Recommendations »

Jun 21, 2018

Jun 20, 2018

Jun 18, 2018

Jun 13, 2018

Jun 6, 2018

May 31, 2018

May 30, 2018

May 24, 2018

Looking for more? Browse all our products here