Customs Service Modernization:

Serious Management and Technical Weaknesses Must Be Corrected

AIMD-99-41: Published: Feb 26, 1999. Publicly Released: Feb 26, 1999.

Additional Materials:

Contact:

Randolph C. Hite
(202) 512-6256
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Pursuant to a congressional request, GAO reviewed the Customs Service's management of the Automated Commercial Environment (ACE), focusing on whether Customs has adequately justified ACE cost-effectiveness.

GAO noted that: (1) Customs is not managing ACE effectively, and it does not have a firm basis for concluding that ACE is a cost-effective solution to modernizing its commercial environment; (2) GAO found serious weaknesses relating to architectural definition, investment management, and software development and acquisition that must be corrected before further investment in ACE is justified; (3) Customs is not building ACE within the context of a complete and enforced systems architecture; (4) in May 1998, GAO reported that Customs' architecture was incomplete because it was not based on a complete understanding of its enterprisewide functional and information needs; (5) GAO also reported that Customs had not yet instituted effective procedures for ensuring compliance with the architecture once it is completed; (6) until its architecture is completed and effectively enforced, Customs will not have adequate assurance that information systems like ACE will optimally support its needs across all business areas; (7) further, Customs lacks a reliable estimate of what ACE will cost to build, deploy, and maintain; and Customs has neither adequately justified, nor is it effectively monitoring, ACE's cost-effectiveness; (8) specifically, Customs did not use rigorous cost estimating techniques in preparing its cost estimate, and did not disclose the inherent imprecision of the estimate; (9) additionally, Customs omitted costs and inflated benefits in preparing its cost-benefit analysis; (10) moreover, Customs is not using effective incremental investment management practices; (11) while Customs plans to develop/acquire ACE in 21 increments, these increments are not individually cost-benefit justified, and Customs is not determining what benefits each increment, once operational, actually provides; (12) as a result, Customs will not know if ACE's expected return-on-investment is actually being realized until it has already spent hundreds of millions of dollars developing/acquiring the entire system; and (13) GAO found that Customs has neither the capability to effectively develop nor acquire ACE and that its processes for doing both, according to widely accepted and proven software capability maturity models, are ad hoc, immature, and ineffective.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: Customs agreed and has completed actions in response to this recommendation. Specifically, Customs has considered alternative approaches to building ACE (i.e., build in-house vs. contract out) and selected an approach that entails hiring a prime contractor who will be responsible for building ACE. Additionally, the Department of the Treasury transferred responsibility for the International Trade Data System (ITDS) to Customs in November 1999. Subsequently, Customs incorporated ITDS' planned functionality into ACE's planned functionality, thus, eliminating the potential for duplication between the two planned systems.

    Recommendation: In addition to previous recommendations to improve Customs' management of information technology, GAO recommends that Customs correct the management and technical weaknesses discussed in this report before building ACE. To accomplish this, the Commissioner of Customs should, with the support of Customs' chief information officer (CIO), ensure that Customs rigorously analyze alternative approaches to building ACE, including International Trade Data System as an alternative to developing ACE entirely within Customs.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security

  2. Status: Closed - Implemented

    Comments: Customs (now known as the Bureau of Customs and Border Protection or CBP) agreed with this recommendation and has substantially completed the actions GAO recommended. Specifically, CBP has prepared an Automated Commercial Environment (ACE) life cycle cost estimate that included a discussion of the estimate's uncertainty. CBP has also prepared benefit expectations for the initial ACE releases. GAO has reviewed these benefit expectations and determined them to be realistic and supportable. Further, CBP established a requirement for a favorable return-on-investment and compliance with its enterprise architecture before making any ACE investment. Lastly, CBP's IT investment management process, which the agency plans to follow for each ACE release, requires demonstration of actual costs and benefits in post-implementation reviews. Customs plans to validate the actual costs and benefits of the initial ACE functionality after it is deployed in September 2003.

    Recommendation: In addition to previous recommendations to improve Customs' management of information technology, GAO recommends that Customs correct the management and technical weaknesses discussed in this report before building ACE. To accomplish this, the Commissioner of Customs should, with the support of Customs CIO, ensure that Customs make investment decisions incrementally, i.e., for each increment: (1) use disciplined processes to prepare a rigorous life-cycle cost estimate, including an explicit discussion of its inherent uncertainty; (2) prepare realistic and supportable benefit expectations; (3) require a favorable return-on-investment and compliance with Customs' architecture before making any investment; and (4) validate actual costs and benefits once an increment is piloted, compare these with estimates, use the results in making further decisions on subsequent increments, and report the results to Customs' House and Senate appropriations and authorizing committees.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security

  3. Status: Closed - Implemented

    Comments: Customs (now known as the Bureau of Customs and Border Protection or CBP) agreed with this recommendation and has taken steps to correct Automated Commercial Environment (ACE) software acquisition process weaknesses. Specifically, CBP hired MITRE to help ensure mature system acquisition processes were followed in awarding the ACE prime contract, conducted multiple interim assessments that were the basis for revisions to CBP's software acquisition processes, and demonstrated progress toward bringing ACE acquisition processes to at least Software Engineering Institute (SEI) level 2. A May 2003 interim assessment led by the SEI found that CBP's modernization office was performing the vast majority of SA-CMM level 2 processes. CBP plans to have their few remaining process weaknesses resolved in time for a formal assessment in September 2003. CBP has also established a policy requiring that all ACE software contractors have at least SEI level 2 processes, and is requiring that its prime modernization contractor maintain level 3 processes.

    Recommendation: In addition to previous recommendations to improve Customs' management of information technology, GAO recommends that Customs correct the management and technical weaknesses discussed in this report before building ACE. To accomplish this, the Commissioner of Customs should, with the support of Customs CIO, ensure that Customs strengthen ACE software acquisition management by: (1) establishing an effective process improvement program and correcting weaknesses in ACE software acquisition processes identified in this report, thereby bringing ACE processes to at least Software Engineering Institute (SEI) level 2; and (2) requiring at least SEI level 2 processes of all ACE software contractors.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security

 

Explore the full database of GAO's Open Recommendations »

Oct 10, 2014

Sep 30, 2014

Sep 22, 2014

Jul 9, 2014

May 14, 2014

Apr 30, 2014

Mar 26, 2014

Jan 13, 2014

Dec 9, 2013

Dec 6, 2013

Looking for more? Browse all our products here