Information Technology: Customs Automated Commercial Environment Program Progressing, but Need for Management Improvements Continues
GAO-05-267
Published: Mar 14, 2005. Publicly Released: Mar 14, 2005.
Skip to Highlights
Highlights
The Department of Homeland Security (DHS) is conducting a multiyear, multibillion-dollar acquisition of a new trade processing system, planned to support the movement of legitimate imports and exports and strengthen border security. By congressional mandate, plans for expenditure of appropriated funds on this system, the Automated Commercial Environment (ACE), must meet certain conditions, including GAO review. This study addresses whether the fiscal year 2005 plan satisfies these conditions, describes the status of DHS's efforts to implement prior GAO recommendations for improving ACE management, and provides observations about the plan and DHS's management of the program.
Recommendations
Recommendations for Executive Action
Agency Affected | Recommendation | Status |
---|---|---|
United States Customs and Border Protection | To strengthen accountability for the ACE program and better ensure that future ACE releases deliver promised capabilities and benefits within budget and on time, the DHS Secretary, through the Under Secretary for Border and Transportation Security, should direct the Commissioner, Customs and Border Protection, to define and implement an ACE accountability framework that ensures coverage of all program commitment areas, including key expected or estimated system (1) capabilities, use, and quality; (2) benefits and mission value; (3) costs; and (4) milestones and schedules. |
Custom and Border Protection's Automated Commercial Environment (ACE) program has defined and implemented an accountability framework that covers program commitment areas and measures progress against costs, milestones, schedules, and risks for selected releases. In May 2007, we reported that the program office had developed an initial version of an accountability framework for measuring several program commitments, such as capabilities and milestones, but that other commitments, such as benefits, had not been as well defined. During the past year, the program office has completed its coverage of program commitments in key program documents--the ACE accountability framework, ACE Program Plan (August 2006), the CBP/ACE performance reference model, periodic Program Management Review (PMR) reports, and the fiscal year 2007 expenditure plan. The accountability framework now captures and integrates data on ACE functional and performance capabilities, benefits, and mission value, estimated and actual costs, milestones and schedule, accomplishments, and earned value management status. Further, the framework provides visibility into each ACE release at several levels of detail, and it is being used by the ACE Executive Director and others as the means for monitoring program progress, issues, and decisions. Data and examples from the accountability framework are included in both the periodic PMR reports and the fiscal year 2007 ACE expenditure plan.
|
United States Customs and Border Protection | To strengthen accountability for the ACE program and better ensure that future ACE releases deliver promised capabilities and benefits within budget and on time, the DHS Secretary, through the Under Secretary for Border and Transportation Security, should direct the Commissioner, Customs and Border Protection, to define and implement an ACE accountability framework that ensures currency, relevance, and completeness of all such commitments made to the Congress in expenditure plans. |
Automated Commercial Environment (ACE) program officials established an accountability framework that is used to inform both its annual expenditure plans and its quarterly congressional reports. However, as with prior expenditure plans, the fiscal year 2007/ 2008 expenditure plans did not reflect the most current program information. In particular, status information on milestones, earned value management (EVM) values, and risks was more than 3 months old when the plans were submitted to the appropriations committees. For example, the fiscal year 2007 expenditure plan listed the milestone for the Release 5/Drop A1 Production Readiness Review (PRR) as March 1, 2007, when this milestone had already slipped by more than 4 months (to July 12, 2007) a month before the plan was submitted to the appropriations committees. ACE program officials continue to use the quarterly ACE status reports to provide the appropriations committees with more current, relevant, and complete information about the program than could be provided in the expenditure plan. However, the quarterly reports were generally submitted to Congress 3 to 4 months after the end of each quarter, thus limiting their currency and relevance. For example, the December 2006 quarterly report was not provided until February 26, 2007, and it contained the same outdated PRR milestone as the expenditure plan. Nevertheless, in its fiscal year 2008 ACE expenditure plan, DHS committed to providing future ACE quarterly reports within 45 days of the end of each quarter. Our review of the last three ACE quarterly reports showed that each was submitted more than 3 months after the end of the applicable quarter. A DHS official told us that intend to continue to work on implementing the recommendation.
|
United States Customs and Border Protection | To strengthen accountability for the ACE program and better ensure that future ACE releases deliver promised capabilities and benefits within budget and on time, the DHS Secretary, through the Under Secretary for Border and Transportation Security, should direct the Commissioner, Customs and Border Protection, to define and implement an ACE accountability framework that ensures reliability of data relevant to measuring progress against commitments. |
In May 2006, we reported that defects for the Automated Commercial Environment (ACE) program are tracked with two separate information systems: one for trouble tickets and another for problem trouble reports. Information about defects that are tracked in both systems may be inconsistent and needed to be manually reconciled to ensure consistency. As a result, the true status of ACE defects -- an important measure of system quality -- was not known. The Bureau of Customs and Border Protection has taken steps to ensure the reliability of defect data used to measure progress against quality commitments for ACE. Specifically, the ACE program office has integrated the two tools using a cross-referencing function and has instituted manual processes for reconciling the data in each tool. The December 2006 quarterly congressional report stated that these efforts have improved the ability to record, assess, and report on system quality and performance.
|
United States Customs and Border Protection | To strengthen accountability for the ACE program and better ensure that future ACE releases deliver promised capabilities and benefits within budget and on time, the DHS Secretary, through the Under Secretary for Border and Transportation Security, should direct the Commissioner, Customs and Border Protection, to define and implement an ACE accountability framework that ensures reporting in future expenditure plans of progress against commitments contained in prior expenditure plans. |
Automated Commercial Environment (ACE) expenditure plans, including the fiscal year 2007 plan, have not adequately reported on progress against previous plan commitments. In May 2006, we reported that the fiscal year 2006 expenditure plan contained several such reporting gaps, such as whether funding amounts were actually obligated and expended as planned and whether releases met schedules. In particular, we reported that the plan did not address whether the design and development for Release 5 was actually accomplished as planned. Similarly, we reported the next year that the fiscal year 2007 expenditure plan did not adequately describe progress against commitments that were made in the fiscal year 2006 plan. For example, the 2007 expenditure plan did not: report progress against milestones in the fiscal year 2006 plan or explain why these milestones were not achieved; report actual obligation or expenditure of funds relative to the planned uses of these funds in prior expenditure plans; or address progress against the milestone dates for each stage of a release that was included in the prior year's plan. As a case in point, the fiscal year 2006 expenditure plan described specific functionality planned for Release 5/Drop A1, such as online registration for trade representatives, but the fiscal year 2007 expenditure plan does not include information on whether these functions were delivered. According to program officials, the quarterly congressional reports provide more current information on the program's progress against prior commitments. However, we found that these reports have also not fully addressed the commitments made in prior expenditure plans. For example, recent quarterly reports described progress against developmental milestones for each release, but did not report whether key functionality associated with each release was delivered. Our analysis of the fiscal year 2009 ACE expenditure plan shows that progress being reported against commitments in prior expenditure plans continues to be limited. For example, progress against cost and schedule commitments were not provided or were vague. Program officials stated that they intend to improve its progress reporting in their quarterly status reports.
|
United States Customs and Border Protection | To strengthen accountability for the ACE program and better ensure that future ACE releases deliver promised capabilities and benefits within budget and on time, the DHS Secretary, through the Under Secretary for Border and Transportation Security, should direct the Commissioner, Customs and Border Protection, to define and implement an ACE accountability framework that ensures use of criteria for exiting key readiness milestones that adequately consider indicators of system maturity, such as severity of open defects. |
The Automated Commercial Environment (ACE) program has demonstrated the use of appropriate exit criteria for several key readiness milestones. Specifically, the system life cycle gate review process for Office of Information and Technology projects, including ACE, has been amended to include milestone readiness decisions based on an assessment and acceptance of risks (including the risks related to unresolved defects). Further, the ACE Risk and Issue Management Process guidance provides for the systematic identification, analysis, prioritization, planning, execution, evaluation, and documentation of program risks and issues and requires that any risks associated with going forward should be identified as part of each life cycle gate review. This process has been applied in the following two major gate reviews: Release A1 (ESAR) Critical Design Review (May 2006) and Release M1 (E-Manifest: Rail and Sea Manifest) Critical Design Review (August 2006). In the case of A1, program officials reported that no severe defects existed, but that all known risks associated with proceeding past the milestone had been entered into the program's risk database and documented in the certification package submitted to the Department of Homeland Security's Chief Information Officer. Program officials reported these steps in the quarterly reports to Congress. Going forward, officials stated that similar assessments will be part of future gate reviews.
|
United States Customs and Border Protection | To strengthen accountability for the ACE program and better ensure that future ACE releases deliver promised capabilities and benefits within budget and on time, the DHS Secretary, through the Under Secretary for Border and Transportation Security, should direct the Commissioner, Customs and Border Protection, to define and implement an ACE accountability framework that ensures clear and unambiguous delineation of the respective roles and responsibilities of the government and the prime contractor. |
The August 2006 version of the Automated Commercial Environment (ACE) program plan describes general roles and responsibilities for the government and the prime contractor. More detailed roles and responsibilities of Customs and Border Protection (CBP), the prime contractor, and the support contractors have been documented in a roles and responsibilities matrix that assigns primary responsibility for each activity, such as testing, training, and maintenance. In addition, the task orders describe the specific responsibilities and expectations of the contractors.
|
Full Report
Public Inquiries
Topics
AccountabilityAppropriated fundsBorder securityBudget outlaysComponent-based software engineeringCost overrunsCustoms administrationFederal fundsFederal procurementFunds managementHomeland securityInternational tradePerformance measuresProcurement planningProgram evaluationRoot cause analysisSchedule slippagesStrategic planningTrade policies