This is the accessible text file for GAO report number GAO-02-908 
entitled 'Customs Service Modernization: Third Expenditure Plan Meets 
Legislative Conditions, but Cost Estimating Improvements Needed' which 
was released on August 09, 2002.

This text file was formatted by the U.S. General Accounting Office 

(GAO) to be accessible to users with visual impairments, as part of a 

longer term project to improve GAO products’ accessibility. Every 

attempt has been made to maintain the structural and data integrity of 

the original printed product. Accessibility features, such as text 

descriptions of tables, consecutively numbered footnotes placed at the 

end of the file, and the text of agency comment letters, are provided 

but may not exactly duplicate the presentation or format of the printed 

version. The portable document format (PDF) file is an exact electronic 

replica of the printed version. We welcome your feedback. Please E-mail 

your comments regarding the contents or accessibility features of this 

document to Webmaster@gao.gov.



Report to Congressional Committees:



August 2002:



Customs Service Modernization:



Third Expenditure Plan Meets Legislative Conditions, but Cost 

Estimating Improvements Needed:



GAO-02-908:



Contents:



Letter:



Agency Comments:



Appendixes:



Appendix I: Custom’s Third Automated Commercial Evironment (ACE):



Appendix II: Comments from the U.S. Customs Service:



Abbreviations:



ACE: Automated Commercial Environment:



ACS: Automated Commercial System:



CMM: Capability Maturity Model:



CMO: Customs Modernization Office:



COTS: commercial, off the shelf:



eCP: e-Customs Partnership:



FMCSA: Federal Motor Carrier Safety Administration:



GAO: General Accounting Office:



IBM: International Business Machines:



ITDS: International Trade Data System:



OMB: Office of Management and Budget:



SA-CMM: Software Acquisition Capability Maturity Model:



SEI: Software Engineering Institute:



Highlights:



August 2002:



DRAFT United States General Accounting Office DRAFT:



CUSTOMS SERVICE MODERNIZATION:



Third Expenditure Plan Meets Legislative Conditions, but Cost 

Estimating Improvements Needed:



Why GAO Did This Study:



The U.S. Customs Service has begun a multiyear, multibillion-dollar 

project: the Automated Commercial Environment (ACE), a new import 

processing system that is planned to support effective and efficient 

movement of goods into the United States. By congressional mandate, 

Customs’ expenditure plans for ACE must meet certain conditions, 

including being reviewed by GAO. This study addresses whether Customs’ 

latest plan satisfies these conditions and provides observations about 

the plan and Customs’ efforts to implement GAO’s open recommendations 

for improving ACE management.



Highlights of GAO-02-908, a report to the Subcommittee on Treasury and 

General Government, Senate Committee on Appropriations, and the 

Subcommittee on Treasury, Postal Service, and General Government, House 

Committee on Appropriations.



What GAO Found:



Customs’ May 2002 ACE expenditure plan is the third in a series of 

legislatively required plans. This plan provides for the design, 

development, and deployment of the second release of the first of four 

planned ACE increments. The plan also meets the legislative conditions 

governing investment in ACE that Congress imposed on Customs.



Since 1999, GAO has reported on Customs’ management of ACE and made a 

series of recommendations to correct deficiencies. Customs currently 

has efforts under way to respond to all of GAO’s recommendations. One 

of these deficiencies that affects the third expenditure plan is 

Customs’ lack of effective cost estimating capabilities. Specifically, 

the cost estimate in the third expenditure plan is questionable because 

of limitations in how it was derived and inconsistencies between this 

estimate and an independent estimate developed by a Customs contractor. 

Customs is currently determining the cause of the variance in the 

estimates.



The following table compares Customs’ expenditure plan cost estimate 

with the Software Engineering Institute’s seven questions for 

determining the reliability of an estimate:



Table:



Question: Are the objectives of the estimate clear and correct?; Yes: 
No: 

[Empty].



Question: Has the task been appropriately sized?; Yes: [Empty]; No:



Question: Are the estimated cost and schedule consistent with 

demonstrated accomplishments on other projects?; Yes: [Empty]; No:



Question: Have the factors that affect the estimate been identified and 

explained?; Yes: [Empty]; No:



Question: Have steps been taken to ensure the integrity of the 

estimating process?; Yes: [Empty]; No:



Question: Is the organization’s historical evidence capable of 

supporting a reliable estimate?; Yes: [Empty]; No:



Question: Has the situation changed since the estimate was prepared?; 

Yes: [Empty]; No:



[End of table]



Source: GAO analysis.



What GAO Recommends:



Because GAO has an open recommendation to Customs for correcting its 

cost estimating deficiencies, which Customs reports will be fully 

implemented in December 2002, GAO is not making another recommendation 

at this time. Customs concurred with GAO’s report and described actions 

that it is taking to improve its expenditure plan cost estimating.



This is a test for developing Highlights for a GAO report. The full 

report, including GAO’s objectives, scope, methodology, and analysis is 

available at www.gao.gov/cgi-bin/getrpt?GAO-02-908. For additional 

information about the report, contact Randolph C. Hite (202-512-3439). 

To provide comments on this test highlights, contact Keith Fultz (202-

512-3200) or E-mail HighlightsTest@gao.gov.



Letter:



August 9, 2002:



The Honorable Byron L. Dorgan 

Chairman 

The Honorable Ben Nighthorse Campbell 

Ranking Minority Member 

Subcommittee on Treasury and General Government 

Committee on Appropriations 

United States Senate:



The Honorable Ernest J. Istook, Jr. 

Chairman 

The Honorable Steny H. Hoyer 

Ranking Minority Member 

Subcommittee on Treasury, Postal Service, 

 and General Government 

Committee on Appropriations 

House of Representatives:



In May 2002, the U.S. Customs Service submitted to Congress its third 

expenditure plan seeking release of $190.2 million for its Automated 

Commercial Environment (ACE) project, pursuant to Customs’ fiscal year 

2002 appropriation.[Footnote 1] ACE is to be Customs’ new import 

processing system and the first project under the Customs Modernization 

Program. As required by the act, we reviewed the expenditure plan. Our 

objectives were to (1) determine whether the third ACE expenditure plan 

satisfies the legislative conditions, (2) determine whether the plan is 

consistent with our open ACE recommendations, and (3) provide 

observations about the plan and Customs’ management of ACE.



On June 28, 2002, we briefed your offices on the results of this 

review. This report officially transmits the results of our work. The 

full briefing, including our scope and methodology, is reprinted as 

appendix I.



Concerning our first objective, Customs’ expenditure plan satisfies the 

legislative conditions specified in the appropriations act. That is, 

the plan provides for (1) meeting the capital planning and investment 

control review requirements of the Office of Management and Budget 

(OMB); (2) complying with Customs’ enterprise architecture;[Footnote 2] 

and (3) complying with federal acquisition rules, requirements, 

guidelines, and systems acquisition management practices. Further, the 

plan was reviewed and approved by the Joint Capital Investment Review 

Board[Footnote 3] and OMB.



Customs is also making progress in addressing our open recommendations, 

which are as follows:



1. Justify and make ACE investment decisions incrementally.



2. Before building each ACE release, certify to Customs’ appropriations 

subcommittees that Customs’ enterprise architecture has been 

sufficiently extended and updated.



3. Develop and implement a Customs Modernization Office (CMO) human 

capital management strategy.



4. Develop and implement a rigorous and analytically verifiable cost 

estimating program that embodies the tenets of effective estimating as 

defined in the institutional and project-specific estimating guidance 

of the Software Engineering Institute (SEI).



5. Limit future expenditure plan requests for management reserve funds 

to 10 percent of the total funds requested for the ACE program, or 

adequately justify any management reserve requests in excess of 10 

percent.



6. Implement certain process controls embodied primarily in the second 

level of SEI’s Software Acquisition Capability Maturity Model (SA-

CMM),[Footnote 4] and by September 30, 2002, assess and report to 

Customs’ appropriations subcommittees on the CMO’s software acquisition 

process maturity.



7. Address the risks associated with the accelerated ACE acquisition 

strategy and report to Customs’ appropriations subcommittees on the 

strategy and plans for mitigating the risks associated with this 

strategy.



Finally, we determined that the cost estimate in the third expenditure 

plan is questionable because Customs is still in the process of 

addressing our recommendation (see recommendation 4) for having an 

effective cost estimating program. While the full impact of the 

weaknesses in the process used to derive the estimate was 

indeterminable, at least one limitation (an inflated labor rate) led to 

an overestimate of about $36 million for designing, developing, and 

deploying ACE increment 1 release 2. Because the expenditure plan 

includes a management reserve of 10 percent of the funding requirement, 

this means that the management reserve is also overstated by about $3.6 

million.



According to an ACE prime contractor executive, the overstatement is 

offset by a corresponding understatement in the estimated number of 

labor hours, which the official attributed to Customs’ not having a 

complete set of requirements at the time that the plan’s estimate was 

derived. However, Customs also arranged for an independent cost 

estimate by another contractor, which led to an estimate that was $77.9 

million lower than the expenditure plan cost estimate. Customs and 

support contractor officials stated that this independent estimate is 

incomplete and reflects a misunderstanding by the cost estimate 

contractor of the scope of work to be performed. Customs is currently 

determining the cause of the variance.



Because we have an open recommendation to Customs for correcting its 

cost estimating deficiencies, which Customs reports will be fully 

implemented in December 2002, we are not making a recommendation at 

this time.



Agency Comments:



In written comments on a draft of this report, the Acting Director, 

Office of Planning, U.S. Customs Service, concurred with the report and 

described actions that are being taken to improve expenditure plan cost 

estimating. These comments are reprinted in appendix II.



We are sending copies of this report to the Chairmen and Ranking 

Minority Members of other Senate and House committees and subcommittees 

that have authorization and oversight responsibilities for the Customs 

Service. We are also sending copies to the Secretary of the Treasury, 

the Commissioner of the Customs Service, and the Director of OMB. We 

also will make copies available to others upon request. In addition, 

the report will be available at no charge on the GAO Web site at http:/

/www.gao.gov.



Should you or your staff have any questions on matters discussed in 

this report, please contact me at (202) 512-3439. I can also be reached 

by E-mail at HiteR@gao.gov. Key contributors to this report were Mark 

Bird, Barbara Collier, Tamra Goldstein, Randolph Tekeley, Scott Pettis, 

and Aaron Thorne.



Randolph C. Hite

Director, Information Technology Architecture 

and Systems Issues:



Signed by Randolph C. Hite: 



[End of section]



Appendix I: Custom’s Third Automated Commercial Environment (ACE) 

Expenditure Plan:



[See PDF for image]



[End of figure]



[End of section]



Appendix II: Comments from the U.S. Customs Service:



U.S. Customs Service:



Memorandum:



DATE: July 31, 2002:



MEMORANDUM FOR RANDOLPH C. HITE:



U.S. GENERAL ACCOUNTING OFFICE:



FROM: Acting Director Office of Planning:



SUBJECT: Draft Audit Report on the United States Customs Service’s 
Third 

Automated Commercial Environment (ACE) Expenditure Plan:



Thank you for providing us with a copy of your draft report entitled 

“Customs Service Modernization: Third Expenditure Plan Meets 

Legislative Conditions, but Cost Estimating Improvements Needed” and 

the opportunity to discuss the issues in this report.



We are pleased that the General Accounting Office (GAO) recognizes the 

progress we are making to satisfy the recommendations from their 

previous reviews. We will continue to address all open recommendations 

aggressively.



GAO gives particular attention to the cost estimation methodology 

supporting the third expenditure plan, noting that we are still in the 

process of addressing their prior recommendation on establishing an 

effective cost estimating program. We continue to pursue efforts to 

improve our cost estimates supporting expenditure plans. Our fourth 

expenditure plan will be supported by a detailed internal review of the 

eCP inputs and independent expenditure plan cost estimates. Each of 

these efforts will be based on the Systems Engineering Institute’s 

(SEI) guidance for validating the reliability of an estimate. We are 

confident that these efforts will provide our stakeholders with greater 

confidence in the cost estimates supporting expenditure plan 4.



Further, Customs is proceeding with acquiring the services of an 

independent contractor to develop and maintain a Modernization Life 

Cycle Cost Model which will then support all planning activities. This 

model will also be based on the SEI estimating guidance.



If you have any questions regarding these comments, please contact Ms. 

Michele Donahue at (202) 927-0957.



Christine E. Gaugler:



Signed by Christine E. Gaugler:



[End of Section]



FOOTNOTES



[1] Pub. L. No. 107-67, 115 Stat. 514, 520 (2001).



[2] An enterprise architecture is an institutional blueprint for 

guiding and constraining investments in business process change and 

systems.



[3] To expedite reviews of ACE expenditure plans, the Customs and 

Treasury Investment Review Boards were consolidated to form the Joint 

Capital Investment Review Board, which first met in November 2001.



[4] Capability Maturity ModelSM (CMM) is a service mark of Carnegie 

Mellon University, and CMM is registered in the U.S. Patent and 

Trademark Office. The SA-CMM identifies key process areas that are 

necessary to effectively manage software-intensive system 

acquisitions. Level 2 is the second level of the SA-CMM’s five-level 

scale; achieving this level means that an organization has the software 

acquisition rigor and discipline to repeat project successes.



GAO’s Mission:



The General Accounting Office, the investigative arm of Congress, 

exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability 

of the federal government for the American people. GAO examines the use 

of public funds; evaluates federal programs and policies; and provides 

analyses, recommendations, and other assistance to help Congress make 

informed oversight, policy, and funding decisions. GAO’s commitment to 

good government is reflected in its core values of accountability, 

integrity, and reliability.



Obtaining Copies of GAO Reports and Testimony:



The fastest and easiest way to obtain copies of GAO documents at no 

cost is through the Internet. GAO’s Web site (www.gao.gov) contains 

abstracts and full-text files of current reports and testimony and an 

expanding archive of older products. The Web site features a search 

engine to help you locate documents using key words and phrases. You 

can print these documents in their entirety, including charts and other 

graphics.



Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its 

Web site daily. The list contains links to the full-text document 

files. To have GAO e-mail this list to you every afternoon, go to 

www.gao.gov and select “Subscribe to daily E-mail alert for newly 

released products” under the GAO Reports heading.



Order by Mail or Phone:



The first copy of each printed report is free. Additional copies are $2 

each. A check or money order should be made out to the Superintendent 

of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. 

Orders should be sent to:



U.S. General Accounting Office

441 G Street NW, Room LM

Washington, D.C. 20548:



To order by Phone: 	Voice: 	(202) 512-6000 

TDD: 	(202) 512-2537

Fax: 	(202) 512-6061



To Report Fraud, Waste, and Abuse in Federal Programs:



Contact:



Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470



Public Affairs:



Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548: