Information Technology: Customs Has Made Progress on Automated Commercial Environment System, but It Faces Long-Standing Management Challenges and New Risks
GAO-06-580
Published: May 31, 2006. Publicly Released: May 31, 2006.
Skip to Highlights
Highlights
The Department of Homeland Security (DHS) is conducting a multiyear, multibillion-dollar acquisition of a new trade processing system, planned to support the movement of legitimate imports and exports and strengthen border security. By congressional mandate, plans for expenditure of appropriated funds on this system, the Automated Commercial Environment (ACE), must meet certain conditions, including GAO review. This study addresses whether the fiscal year 2006 plan satisfies these conditions; it also describes the status of DHS's efforts to implement prior GAO recommendations for improving ACE management, and provides observations about the plan and DHS's management of the program.
Recommendations
Recommendations for Executive Action
| Agency Affected Sort descending | Recommendation | Status |
|---|---|---|
| Department of Homeland Security | To assist CBP in managing ACE--and increasing the chances that it will deliver required capabilities on time and within budget, demonstrating promised mission benefits and results--the Secretary of Homeland Security should direct the appropriate departmental officials to fully address those legislative conditions associated with having an approved privacy impact assessment and ensuring architectural alignment. |
Closed – Implemented
Regarding the ACE Privacy Impact Assessment (PIA), our review of DHS's 2007 ACE Expenditure Plan (GAO 08-46, October 2007) reported that the ACE PIA did not cover recently completed screening releases S1 and S2. Program officials told us that S1 and S2 are covered by Automated Targeting System (ATS) PIA. However, our analysis of the ATS PIA showed that it addressed screening and targeting functions but did not specifically address releases S1 and S2. DHS subsequently implemented this recommendation by completing and documenting a PIA for releases S1 and S2. Regarding enterprise architecture alignment, our 2007 review also reported that DHS had not sufficiently documented the criteria and methodology used to determine how ACE aligned with the DHS Enterprise Architecture (EA), and did not address other relevant aspects of program alignment to an EA, such as data alignment. DHS subsequently implemented this recommendation by documenting ACE EA alignment criteria, methodology, and data architecture alignment and reported to the Congress in 2008 that the ACE Data Reference Model(DRM) was aligned to the CBP and DHS Data Reference Models (DRM).
|
| Office of Information and Technology | The Secretary of Homeland Security, through CBP's Acting Commissioner, should direct the Assistant Commissioner for Information and Technology to fully address those legislative conditions associated with measuring ACE performance and results and employing effective independent verification and validation practices. |
Closed – Implemented
Our review of DHS's 2007 ACE Expenditure Plan (GAO 08-46, October 2007) reported that CBP had implemented our 2006 recommendation to measure ACE performance and results by developing a range of ACE performance measures, such as user satisfaction; had taken steps to align the measures with program, CBP, and DHS strategic goals and outcomes; and planned to refine the measures and include them in the ACE accountability framework. We also reported that CBP had implemented our recommendation to employ effective independent validation and verification (IV&V) practices by certifying that an IV&V agent was under contract; developing an IV&V Implementation Management Plan that required an IV&V program consistent with the industry standard; providing a set of objectives, guidelines, and expectations for IV&V activities, including periodic independent reports on status, observations, recommendations and activities; and addressing satisfaction of quality standards for ACE products and user needs with the program. Program officials told us that these IV&V improvements allowed early identification and correction of program process and product weakness.
|
| Office of Information and Technology | The Secretary of Homeland Security, through CBP's Acting Commissioner, should direct the Assistant Commissioner for Information and Technology to accurately report to the appropriations committees on CBP's progress in implementing our prior recommendations. |
Closed – Implemented
Our review of DHS's FY 2007 ACE Expenditure Plan (GAO-08-46, October 2007) reported that DHS had submitted quarterly reports from November 2002 through the second quarter of fiscal year 2007 to the House and the Senate Appropriations Subcommittees documenting CBP's efforts to address open GAO recommendations for ACE. We also reported that by taking these actions, CBP had demonstrated that full implementation of this recommendation.
|
| Office of Information and Technology | The Secretary of Homeland Security, through CBP's Acting Commissioner, should direct the Assistant Commissioner for Information and Technology to include in the June 30, 2006, quarterly update report to the appropriations committees a strategy for managing ACE human capital needs and the ACE framework for managing performance and ensuring accountability. |
Closed – Not Implemented
The Bureau of Customs and Border Protection (CBP) included a summary of its information technology (IT) human capital strategy and its accountability framework for the Automated Commercial Environment (ACE) in its fiscal year 2006 second quarter report (March 2006). However, our review of DHS's 2007 ACE Expenditure Plan (GAO 08-46, October 2007) determined that CBP's June 30, 2006, report to the Congress described ACE human capital goals, but did not meet basic tenets of strategic human capital management. For instance, the goals were not developed using a gap analysis that assessed existing staffing and skills against expected staffing needs. At that time, DHS officials reported that a new human capital strategy and implementation plan was in development. In November 2008, CBP assessed its ACE staffing by job function and labor category and determined that the gap between available and needed staff was within tolerable limits, and therefore concluded that no strategic changes in IT human capital were needed. However, CBP's assessment did not address future human capital needs due to changes in technology (such as training and hiring) or projected attrition (such as retirements and employee turnover), and thus did not address key aspects of an information technology human capital strategy. Furthermore, the ACE quarterly reports to the Congress since June 2006 have not reported on IT skill gaps and or plans to address the gaps. As a result, CBP has not yet developed a credible IT human capital strategy for ACE or provided the appropriations committees with updates on ACE IT human capital status and plans.
|
| Office of Information and Technology | The Secretary of Homeland Security, through CBP's Acting Commissioner, should direct the Assistant Commissioner for Information and Technology to document key milestone decisions in a way that reflects the risks associated with proceeding with unresolved severe defects and provides for mitigating these risks. |
Closed – Implemented
Our review of DHS's 2007 ACE Expenditure Plan (GAO-08-46, October 2007) reported that CBP had completed implementation of this recommendation by amending the system life cycle gate review process for Office of Information and Technology projects, including ACE, to include milestone readiness decisions based on an assessment and acceptance of risks (including the risks related to unresolved defects). We further reported that the ACE Risk and Issue Management Process guidance -- which provides for the systematic identification, analysis, prioritization, planning, execution, evaluation, and documentation of program risks and issues and requires that any risks associated with going forward should be identified as part of each life cycle gate review -- was applied to two major gate reviews for ACE in 2006: Release A1 and Release M1. CBP plans to continue this process for future gate reviews. Quarterly reports to Congress on ACE stated that risks related to gate review decisions and the associated impacts were entered into a database to ensure visibility and mitigation and were included in the package submitted to the DHS CIO for review and certification to pass a given milestone. By taking these actions, CBP has demonstrated implementation of the recommendation.
|
| Office of Information and Technology | The Secretary of Homeland Security, through CBP's Acting Commissioner, should direct the Assistant Commissioner for Information and Technology to minimize the degree of overlap and concurrence across ongoing and future ACE releases, and capture and mitigate the associated risks of any residual concurrence. |
Closed – Implemented
Our review of DHS's FY 2007 ACE Expenditure Plan (GAO-08-46, October 2007) reported that the ACE program office had reduced overlap and concurrence in ACE releases by, for example decoupling and reducing dependencies among releases, reducing contention for hardware, dividing releases into drops for more flexible scheduling, and identifying overlap risk in a database. However, we reported that vague and incomplete mitigation strategies made it difficult to determine their effectiveness in reducing concurrency risks among releases and that challenges remained in managing scheduling slips. Since our 2007 report, DHS has taken two additional actions to address this recommendation. First it has stopped work on ACE drops M1 and M2, thus reducing concurrence with these drops. The Cargo Systems Program Office reported that the two remaining active projects, Entry Summary, Accounts, and Revenue (ESAR A2.3.1a) and Cargo Control and Release (CCR M1), do not contend for resources and that future work will be scheduled to avoid resource contention. Secondly, an Environments Working Group (EWG) convenes weekly to coordinate resources among all project teams and mitigate contention in the ACE environment using a chart of projects and resource requirements. By taking these actions, the agency has demonstrated that it has implemented the recommendation by minimizing the overlap among ACE releases -- thereby reducing resource contention -- and by establishing a mechanism to monitor and manage the residual risks of resource contention.
|
| Office of Information and Technology | The Secretary of Homeland Security, through CBP's Acting Commissioner, should direct the Assistant Commissioner for Information and Technology to use earned value management in the development of all existing and future releases. |
Closed – Implemented
Our review of DHS's 2007 ACE Expenditure Plan (GAO-08-46, October 2007) reported this recommendation as implemented because CBP had established baselines to implement EVM for Release 5/Drops A1 and A2, Screening 3, and Release 6/Drop M1 and included EVM data in its accountability framework. By establishing this framework, DHS has demonstrated its commitment to using EVM in future releases and is positioned to execute EVM throughout the life of ACE.
|
| Office of Information and Technology | The Secretary of Homeland Security, through CBP's Acting Commissioner, should direct the Assistant Commissioner for Information and Technology to develop the range of realistic ACE performance measures and targets needed to support an outcome-based, results-oriented accountability framework, including user satisfaction with ACE. |
Closed – Implemented
Our review of DHS's FY 2007 ACE Expenditure Plan (GAO-08-46, October 2007) reported that CBP had implemented this recommendation by establishing a) performance measures tied to program objectives and performance goals; b) processes for collecting, analyzing, and integrating performance data for each measure; and c) a life cycle process and database tool to manage these performance measures. These performance measures address user satisfaction, efficiency, and productivity, among other things, and are integral to the ACE accountability framework, which was approved by the CBP Commissioner and DHS CIO.
|
| Office of Information and Technology | The Secretary of Homeland Security, through CBP's Acting Commissioner, should direct the Assistant Commissioner for Information and Technology to explicitly align ACE program goals, benefits, desired business outcomes, and performance measures. |
Closed – Implemented
Our review of DHS's 2007 ACE Expenditure Plan (GAO-08-46, October 2007) reported that CBP had implemented this recommendation by developing ACE program measures tied to program goals and an ACE Performance Reference Model that aligns CBP strategic goals and objectives with performance measures for ACE releases. For example, CBP aligned the DHS strategy of preventing terrorists and terrorist weapons from entering the U. S. through improved information and targeting with the objective of using advanced passenger and cargo information from various ACE and non-ACE sources to pre-screen, target, and identify potential terrorists, terrorist shipments, and related activities. These were linked to CBP's strategy for increased use of targeting, desired results for the number of security-focused selections generated by ACE, and performance measures for capturing the actual selection numbers. Further, this model links the measures to specific ACE releases. This comprehensive alignment across DHS, CBP, and ACE provides the structure to ensure that ACE meets department and agency mission needs.
|
Full Report
Office of Public Affairs
Topics
Appropriated fundsBorder securityCustoms administrationExportingFederal procurementFederal procurement policyImportingPerformance measuresProcurement planningProgram evaluationProgram managementInformation sharing