This is the accessible text file for GAO report number GAO-06-591T entitled 'Cargo Container Inspections: Preliminary Observations on the Status of Efforts to Improve the Automated Targeting System' which was released on March 30, 2006. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Statement for the Record to the Permanent Subcommittee on Investigations, Committee on Homeland Security and Governmental Affairs, U.S. Senate: United States Government Accountability Office: GAO: For Release on Delivery Expected at 9:30 a.m. EST: Thursday, March 30, 2006: Cargo Container Inspections: Preliminary Observations on the Status of Efforts to Improve the Automated Targeting System: Statement for the Record by Richard M. Stana, Director, Homeland Security and Justice Issues: GAO-06-591T: GAO Highlights: Highlights of GAO-06-591T, a statement for the record to the Permanent Subcommittee on Investigations, Committee on Homeland Security and Governmental Affairs, United States Senate: Why GAO Did This Study: U.S. Customs and Border Protection’s (CBP) Automated Targeting System (ATS)—a computerized model that CBP officers use as a decision support tool to help them target oceangoing cargo containers for inspection— is part of CBP’s layered approach to securing oceangoing cargo. GAO reported in February 2004 on challenges CBP faced in targeting oceangoing cargo containers for inspection and testified before this subcommittee in March 2004 about the findings in that report. The report and testimony outlined recommendations aimed at (1) better incorporating recognized modeling practices into CBP’s targeting strategy, (2) periodically adjusting the targeting strategy to respond to findings that occur during the course of its operation, and (3) improving implementation of the targeting strategy. This statement for the record discusses preliminary observations from GAO’s ongoing work related to ATS and GAO’s 2004 recommendations addressing the following questions: * What controls does CBP have in place to provide reasonable assurance that ATS is effective at targeting oceangoing cargo containers with the highest risk of smuggled weapons of mass destruction? * How does CBP systematically analyze security inspection results and incorporate them into ATS? * What steps has CBP taken to better implement the rest of its targeting strategy at the seaports? What GAO Found: CBP has not yet put key controls in place to provide reasonable assurance that ATS is effective at targeting oceangoing cargo containers with the highest risk of containing smuggled weapons of mass destruction. To provide assurance that ATS targets the highest-risk cargo containers as intended, CBP is (1) working to develop and implement performance measures related to the targeting of cargo containers, (2) planning to compare the results of its random inspections with its ATS inspection results, (3) working to develop and implement a testing and simulation environment, and (4) addressing recommendations contained in a 2005 peer review of ATS. CBP expects to begin using performance measures in June 2006 and enter the final phase of software development for its testing and simulation environment at the same time. However, to date, none of these four initiatives has been fully implemented. Thus, CBP does not yet have key internal controls in place to be reasonably confident that ATS is providing the best information to allocate resources for targeting and inspecting containers that are the highest risk and not overlook inspecting containers that pose a threat to the nation. CBP does not yet have a comprehensive, integrated system in place to analyze security inspection results and incorporate them into ATS. CBP currently adjusts ATS based on intelligence information it receives and has initiated a process to track suggestions submitted by CBP targeting officers at the seaports for modifying ATS. However, CBP has not yet implemented plans to refine ATS based on findings from routine security inspections. Without a more comprehensive feedback system, CBP is limited in refining ATS, a fact that could hinder the overall effectiveness of the targeting strategy. CBP has taken steps to improve implementation of the targeting strategy at the seaports. It has implemented a testing and certification process for its officers who complete the Sea Cargo Targeting Course that should provide better assurance of effective targeting practices. CBP has also made a good faith effort to address longshoremen’s safety concerns regarding radiation emitted by nonintrusive inspection equipment by taking actions such as working with longshoremen’s unions and other maritime organization to develop public radiation tests on the nonintrusive inspection equipment. Nevertheless, CBP has not been able to persuade one longshoremen’s union to permit changes in the procedure for staging containers to increase inspection efficiency at some West Coast seaports where the union’s members work. www.gao.gov/cgi-bin/getrpt?GAO-06-591T. To view the full product, including the scope and methodology, click on the link above. For more information, contact Richard Stana at (202) 512-8777 or stanar@gao.gov. [End of section] Mr. Chairman and Members of the Subcommittee: I appreciate the opportunity to comment on our ongoing work on the U.S. Customs and Border Protection's (CBP) Automated Targeting System (ATS)- -a computerized model that CBP officers use as a decision support tool to help them target oceangoing cargo containers for inspection.[Footnote 1] In the aftermath of the terrorist attacks of September 11, 2001, there is heightened concern that terrorists will attempt to smuggle a weapon of mass destruction (e.g., a nuclear, biological, or radiological explosive device) into the United States using one of the 11 million cargo containers that arrive at our nation's seaports. Because of the large volume of imported containers, CBP maintains that it is unable to physically inspect all oceangoing containers without disrupting the flow of commerce. Thus, CBP uses a multilayered strategy for addressing the threat posed by the movement of oceangoing containers, of which ATS is a key component.[Footnote 2] CBP uses ATS to review documentation and assign a risk score for all containers destined for U.S. ports. CBP officers located at domestic ports or at 1 of the 40 foreign ports that participate in the Container Security Initiative (CSI) then use these scores to help them make decisions on the extent of additional documentary review and possible physical inspection that will be conducted at the seaport. We previously reported in February 2004 on the challenges CBP faced in targeting oceangoing cargo containers for inspection[Footnote 3] and testified before this Subcommittee in March 2004 about the findings in that report.[Footnote 4] The report and testimony outlined recommendations aimed at (1) incorporating recognized modeling practices into its targeting strategy, such as conducting simulated events and initiating an external peer review,[Footnote 5] (2) periodically adjusting the targeting strategy to respond to findings that occur during the course of its operation, and (3) improving implementation of the targeting strategy at domestic seaports. This subcommittee and other congressional requesters asked that we ascertain whether CBP had implemented the recommendations we made to improve the targeting strategy. Our work, in response to this request, has been under way since last October, and we expect to complete the work and provide this subcommittee and our other requesters with a report on the final results later this year. In this statement, I will discuss our preliminary observations on the status of these recommendations as part of the following questions: * What controls does CBP have in place to provide reasonable assurance that ATS is effective at targeting oceangoing cargo containers with the highest risk of smuggled weapons of mass destruction? * How does CBP systematically analyze security inspection results and incorporate them into ATS? * What steps has CBP taken to better implement the rest of its targeting strategy at the seaports? To address these questions, we interviewed CBP officials in headquarters and visited six seaports: Baltimore, Charleston, Los Angeles-Long Beach, Miami, New York-New Jersey, and Savannah. Because we did not select a random sample of ports to visit, the results from these visits cannot be generalized to ports nationwide. We also met with CBP's contractor responsible for conducting CBP's peer review of ATS and longshoremen's union representatives. We reviewed CBP's policies and procedures for targeting and inspecting shipments, and its documentation on intelligence gathering and dissemination, targeting strategies, random inspections, training, and radiation safety as well as its peer review report. We also examined information on officers trained and certified in CBP's Sea Cargo Targeting Training course. We did not independently validate the reliability of CBP's targeting results or test the effectiveness of ATS. We conducted our work in response to this request from October 2005 through March 2006 in accordance with generally accepted government auditing standards. Appendix I contains more detailed information on our scope and methodology. Summary: CBP has not yet put key controls in place to provide reasonable assurance that ATS is effective at targeting oceangoing cargo containers with the highest risk of containing smuggled weapons of mass destruction.[Footnote 6] To provide assurance that ATS targets the highest-risk cargo containers as intended, CBP is (1) working to develop and implement performance measures related to the targeting of cargo containers, (2) planning to compare the results of its random inspections with its ATS inspection results, (3) working to develop and implement a testing and simulation environment, and (4) addressing recommendations contained in a 2005 peer review of ATS. CBP expects to begin using performance measures in June 2006 and enter the final phase of software development for its testing and simulation environment at the same time. However, to date, none of these four initiatives has been fully implemented. Thus, CBP does not yet have key internal controls in place to be reasonably confident that ATS is providing the best information to allocate resources for targeting and inspecting containers that are the highest risk and not overlook inspecting containers that pose a threat to the nation.[Footnote 7] CBP does not yet have a comprehensive, integrated system in place to analyze security inspection results and incorporate them into ATS. An integrated system would allow any of the various systems that CBP uses to manage cargo inspection data to communicate with one another for the purpose of analyzing combined data. CBP currently adjusts ATS based on intelligence information it receives and has initiated a process to track suggestions submitted by CBP targeting officers at the seaports for modifying ATS. However, CBP has not yet implemented plans to refine ATS based on findings from routine security inspections. Without a more comprehensive feedback system, CBP is limited in refining ATS, a fact that could hinder the overall effectiveness of the targeting strategy. CBP has taken steps to improve implementation of the targeting strategy at the seaports. It has implemented a testing and certification process for its officers who complete the Sea Cargo Targeting Course that should provide better assurance of effective targeting practices. CBP has also made a good faith effort to address longshoremen's safety concerns regarding radiation emitted by nonintrusive inspection equipment by taking actions such as working with longshoremen's unions and other maritime organization to develop public radiation tests on the nonintrusive inspection equipment.[Footnote 8] Nevertheless, CBP has not been able to persuade one longshoremen's union to permit changes in the procedure for staging containers to increase inspection efficiency at some West Coast seaports where the union's members work. Background: Oceangoing cargo containers have an important role in the movement of cargo between global trading partners. Approximately 90 percent of the world's trade is transported in cargo containers. In the United States almost half of incoming trade (by value) arrives by containers aboard ships. If terrorists smuggled a weapon of mass destruction into the nation using a cargo container and detonated such a weapon at a seaport, the incident could cause widespread death and damage to the immediate area, perhaps shut down seaports nationwide, cost the U.S. economy billions of dollars, and seriously hamper international trade. The Department of Homeland Security and CBP are responsible for addressing the threat posed by terrorist smuggling of weapons in oceangoing containers. To carry out this responsibility, CBP uses a layered security strategy. One key element of this strategy is ATS. CBP uses ATS to review documentation, including electronic manifest information submitted by the ocean carriers on all arriving shipments, to help identify containers for additional inspection.[Footnote 9] CBP requires the carriers to submit manifest information 24 hours prior to a United States-bound sea container being loaded onto a vessel in a foreign port. ATS is a complex mathematical model that uses weighted rules that assign a risk score to each arriving shipment in a container based on manifest information. As previously discussed, CBP officers use these scores to help them make decisions on the extent of documentary review or physical inspection to be conducted. ATS is an important part of other layers in the security strategy. Under its CSI program, CBP places staff at designated foreign seaports to work with foreign counterparts to identify and inspect high-risk containers for weapons of mass destruction before they are shipped to the United States. At these foreign seaports, CBP officials use ATS to help target shipments for inspection by foreign customs officials prior to departing for the United States. Approximately 73 percent of cargo containers destined for the United States originate in or go through CSI ports. ATS is also an important factor in the Customs-Trade Partnership Against Terrorism (C-TPAT) program. C-TPAT is a cooperative program linking CBP and members of the international trade community in which private companies agree to improve the security of their supply chains in return for a reduced likelihood that their containers will be inspected. Specifically, C-TPAT members receive a range of benefits, some of which could change the ATS risk characterization of their shipments, thereby reducing the probability of extensive documentary and physical inspection. CBP Currently Does Not Have Reasonable Assurance That ATS Is Effective: CBP does not yet have key controls in place to provide reasonable assurance that ATS is effective at targeting oceangoing cargo containers with the highest risk of containing smuggled weapons of mass destruction. To address this shortcoming, CBP is (1) developing and implementing performance metrics to measure the effectiveness of ATS, (2) planning to compare the results of randomly conducted inspections with the results of its ATS inspections, (3) developing and implementing a simulation and testing environment, and (4) addressing recommendations contained in a 2005 peer review. To date, none of these control activities have been fully completed or implemented.[Footnote 10] Thus, CBP does not yet have key internal controls in place to be reasonably certain that ATS is providing the best available information to allocate resources for targeting and inspecting containers that are the highest risk and thus not overlook inspecting containers that pose a high threat to the nation. CBP Does Not Yet Have Performance Measures to Gauge the Effectiveness of ATS in Targeting Cargo Containers, But is Working to Develop Them: CBP does not yet have performance measures in place to help it determine the effectiveness of ATS at targeting oceangoing cargo containers with the highest risk of smuggled weapons of mass destruction. The Comptroller General's internal control standards include the establishment and review of performance measures as one example of a control activity to help an entity ensure it is achieving effective results.[Footnote 11] In July 2005, CBP contracted with a consulting firm to develop such performance metrics. CBP officials and personnel from this consulting firm told us that the firm's personnel analyzed shipment information in ATS over a 2-year period to obtain additional insights into ATS's performance and to determine whether ATS is more effective at targeting cargo containers for terrorism related risk than a random sampling inspection approach. CBP officials told us that the consulting firm's personnel prepared a draft of the results of their analyses and that, as of March 21, 2006, CBP officials are reviewing these analyses. They also said that the consulting firm's personnel are documenting the methodology for their analyses and related performance measures that CBP can use in the future. CBP officials expect to receive this methodology and the performance measures in April 2006, and told us that they expect to begin using the measures in June 2006. CBP officials also told us that they initially planned to have performance measures developed by August 31, 2005, but that this process has taken longer than expected because of delays in (1) obtaining security clearances for the consulting firm's personnel, (2) obtaining workspace for the firm's staff, and (3) arranging for the appropriate levels of access to CBP's information systems. CBP Is Not Yet Using the Results of Random Inspections to Assess ATS Effectiveness: Currently, CBP is not using the results of its random sampling program to assess the effectiveness of ATS. As part of its Compliance Measurement Program, CBP plans to randomly select 30,000 shipments based on entry information submitted by the trade community and examine those shipments to ensure compliance with supply chain security during fiscal year 2006.[Footnote 12] At this time, CBP is unable to compare the examination results from its random sampling program with its ATS inspection results, as we recommended in our 2004, report because CBP does not yet have an integrated, comprehensive system in place to compare multiple sets of data--like results of random inspections with results of routine ATS inspections that were triggered by ATS scores and other operational circumstances. Such a comparison would allow examination of if and why the outcomes of ATS's weighted rule sets are not consistent with the expected outcomes possible in the universe of cargo containers, based on sample projections. Furthermore, the Comptroller General's standards for internal control state that information should be recorded and communicated to management and others within the entity who need it in a form that enables them to carry out their responsibilities.[Footnote 13] CBP Has Not Yet Tested the Effectiveness of ATS in Targeting Cargo Containers for Inspection but Has Plans to Do So: Currently, CBP does not conduct simulated events (e.g., covert tests and computer-generated simulations)--a key control activity--to test and validate the effectiveness of ATS in targeting oceangoing cargo containers with the highest risk of containing smuggled weapons of mass destruction and has not yet implemented a dedicated simulation and testing environment. Without testing and validation, CBP lacks a vital mechanism for evaluating ATS's ability to identify high-risk containers. In July 2005, CBP contracted with a consulting firm to obtain assistance in the development of a computer-generated simulation and testing environment. CBP officials report that they have the simulation environment infrastructure in place and have processed mock manifest data to simulate cargo linked to terrorism in the new environment. CBP is currently reviewing the results of this test. Further, CBP officials told us that the consulting firm is continuing to work with CBP to develop system requirements so that officers can effectively use the simulation environment. CBP expects to receive the consulting firm's final input for the simulation and testing environment by June 2006. CBP officials said that they cannot estimate when this simulation and testing environment will be fully operational until CBP receives the consulting firm's final product. As with the development of performance measures, CBP officials also told us that this process has taken longer than expected because of delays in (1) obtaining security clearances for the consulting firm's personnel, (2) obtaining workspace for the firm's staff, and (3) arranging for the appropriate levels of access to CBP's information systems. As we reported in 2004, terrorism experts suggested that testing ATS by covertly simulating a realistic event using probable methods of attack would give CBP an opportunity to examine how ATS would perform in an actual terrorist situation.[Footnote 14] CBP officials told us that although they are considering implementing this kind of practice, they do not currently have a program in place to conduct such tests. The Director of CBP's Management Inspections and Integrity Assurance office told us that in mid-April 2006, his office will be presenting a proposal to the Acting Commissioner and other senior management to request initiation of a program to conduct testing of the CSI program that will include testing ATS to help ensure that it is appropriately targeting the highest-risk cargo in the CSI program. CBP Is Working to Address Peer Review Recommendations: In response to our 2004 recommendation that CBP initiate an external peer review of ATS, CBP contracted with a consulting firm to evaluate CBP's targeting methodology and recommend improvements.[Footnote 15] Specifically, the contractor identified strengths of the CBP targeting methodology and compared ATS with other targeting methodologies. However, the peer review did not evaluate the overall effectiveness of ATS because CBP did not have the systems in place to allow the contractor to do so. The contractor's final report, issued in April 2005, identified many strengths in the ATS targeting methodology, such as a very capable and highly dedicated team and the application of a layered approach to targeting. It also made several recommendations to improve the targeting methodology that included control activities, such as (1) the development of performance measures, (2) the development of a simulation and testing environment, (3) the development and implementation of a structured plan for continual rules enhancement, and (4) an evaluation and determination of the effectiveness of the ATS targeting rules, several of which reinforced the recommendations we made in our 2004 report.[Footnote 16] CBP issued a detailed plan, which projected delivery dates, for responding to the recommendations made in the contractor's final report. However, about half of these dates have not been met. For example, CBP projected that it would have its testing and simulation environment in place by September 30, 2005. Although CBP has been working on this effort, the environment has not yet been implemented. As previously discussed, CBP officials said that they cannot provide a current estimate of when this simulation and testing environment will be fully operational. Although CBP Strives to Refine ATS for Intelligence Information and Officer Feedback, It Is Not Yet Positioned to Use Inspection Results: CBP strives to refine ATS to include intelligence information it acquires and feedback it receives from its targeting officers at the seaports, but it is not able to systematically adjust ATS for inspection results. CBP does not have a comprehensive, integrated system in place to report details on security inspections nationwide that will allow management to analyze those inspections and refine ATS. CBP officials said that they are developing a system that will allow them to do so but did not know when it will be fully operational. CBP officials cautioned that because an inspection does not identify any contraband or a weapon of mass destruction or its components, it may not necessarily indicate that a particular rule is not operating as intended. They noted that terrorist incidents may happen infrequently, and the rule therefore might operate only when weapons, materials, or other dangerous contraband is actually shipped. However, without analyzing and using security inspection results to adjust ATS, CBP is limited in refining ATS, a fact that could hinder the effectiveness of CBP's overall targeting strategy. CBP Adjusts ATS for Targeting Cargo Containers for Inspection Based on Intelligence: CBP adjusts ATS's rules and weights for targeting cargo containers for inspection in response to intelligence received on an ongoing basis. CBP's Office of Intelligence (OINT) is responsible for acquiring, reviewing, analyzing, and disseminating intelligence. OINT officials told us they receive information from the intelligence community, which includes federal agencies such as the Central Intelligence Agency and the Federal Bureau of Investigation.[Footnote 17] According to OINT officials, OINT disseminates information to CBP's offices at the seaports to, among other things, support these offices' targeting efforts related to cargo containers. For example, the targeting officers may use information provided by OINT to search ATS for information about shipments and containers. OINT officials said they also disseminate information to CBP's senior management to inform them about risks associated with cargo containers. CBP uses intelligence information to refine its targeting of cargo containers for inspection by incorporating the intelligence information into ATS to readily identify containers whose manifest information may match or be similar to data contained in the intelligence information. CBP documentation and our observations showed that CBP headquarters personnel incorporate intelligence information into ATS by adjusting ATS's existing rules and weights and creating new rules and weights that result in a higher risk score being assigned to a container whose manifest information may match or be similar to data contained in the intelligence information. CBP officers can also conduct queries or create lookouts in ATS that will search all manifest data in the system to identify those containers whose manifest information may match or be similar to data contained in the intelligence information.[Footnote 18] Once ATS identifies these containers, CBP officers are to then designate these containers for inspection. When CBP receives credible intelligence information that requires immediate action, CBP officials also report that they can initiate a special operation to address specific concerns identified in the intelligence data. CBP officials at the six seaports we visited reported that they sometimes receive intelligence information from local sources such as state and local law enforcement. Officials at five of these seaports reported that they will use such information to help them make decisions regarding targeting efforts. Additionally, officials at five of the six seaports we visited said that if the information they receive has national implications, they will notify CBP headquarters personnel, who will make a determination regarding potential adjustments to ATS. CBP Targeting Officers at the Seaports Have Provided Few Suggestions for Adjusting ATS: In the late summer of 2005, CBP headquarters initiated a process to formally track its targeting officers' suggestions to enhance ATS for targeting cargo containers for inspection. Targeting officers at all six seaports we have visited are aware of the process for providing suggestions to CBP headquarters. According to documentation maintained by headquarters, CBP officers at the seaports have provided few suggestions to date. CBP headquarters officials said that although they have received few suggestions for modifying ATS, they do not believe this is an indication of ATS's effectiveness. These officials stated that overall the feedback they have received from CBP targeting officers at the seaports related to the operation and usefulness of ATS has been positive. We reviewed the report CBP uses to track these suggestions and found that since it was established, CBP headquarters has received 20 suggestions for enhancing the ATS component responsible for targeting oceangoing cargo containers for inspection. Some of these suggestions relate to modifying ATS's rules, while others focused on other aspects of ATS such as enhancing the organization and presentation of ATS screens by changing the size of an icon and the fonts or text used. CBP Is Not Using Inspection Results to Systematically Adjust ATS, but It Is Developing a System to Allow it to Do So: CBP is not using inspection results to systematically adjust ATS for targeting cargo containers for inspection because CBP does not yet have a comprehensive, integrated system in place that can report sufficient details for analyzing inspection results. CBP officials said that although they can analyze inspection results on a case-by-case basis to identify opportunities to refine ATS, such as when an inspection results in a seizure of some type of contraband, they currently do not have a reporting mechanism in place that will allow them to view inspection results nationwide to identify patterns for systematically adjusting ATS. CBP is developing the Cargo Enforcement Reporting Tracking System (CERTS) to document, among other things, all cargo examinations so that documentation substantiating the examinations will be available for analysis by management to adjust ATS. CBP officials said they will begin testing CERTS in the spring of 2006. CBP officials told us that once testing of CERTS is complete, they will be in a better position to estimate when CERTS can be fully implemented. CBP officials cautioned that because an inspection does not identify any contraband or a weapon of mass destruction or its components, it may not necessarily indicate that a particular rule is not operating as intended. They noted that terrorist incidents may happen infrequently and the rule therefore might operate only when weapons, materials, or other dangerous contraband is actually shipped. However, without using inspection results to adjust ATS, CBP may not be targeting and inspecting containers with the highest risk of containing smuggled weapons of mass destruction. CBP Has Taken Steps to Better Implement the Targeting Strategy at the Seaports: CBP has implemented a testing and certification process for its officers who complete the Sea Cargo Targeting Course that should provide better assurance of effective targeting practices. CBP has also made a good faith effort to address longshoremen's safety concerns regarding radiation emitted by nonintrusive inspection equipment. Nevertheless, it has not been able to persuade one longshoremen's union to permit changes in the procedure for staging containers to increase inspection efficiency. CPB Has Implemented a Testing and Certification Process for Officers Who Target Cargo Containers for Inspection: In our 2004 report, we recommended that CBP establish a testing and certification process for CBP staff who complete the national targeting training to provide reasonable assurance that they have sufficient expertise to perform targeting work. [Footnote 19] CBP has implemented such a testing and certification process. CBP conducted two evaluations that assessed its targeting training program--a job performance assessment and a job task analysis. With the results of these evaluations, CBP concluded that a certification component should be added to the training program and the Sea Cargo Targeting Training course content should remain unchanged. CBP officials then updated the course materials to encompass the inclusion of the certification component. In October 2004, CBP began certifying officers who successfully completed the Sea Cargo Targeting Training course. Since the establishment of the testing and certification component for the Sea Cargo Targeting Training course, CBP data indicate that it has trained and certified 278 of its officers responsible for targeting cargo as of March 24, 2006.[Footnote 20] While CBP has conducted a job performance assessment prior to the incorporation of a certification program for Sea Cargo Targeting Training, it has not yet formally assessed the impact that revised training and certification has had on officers' targeting of oceangoing cargo containers. However, a CBP official said that CBP has recently initiated planning efforts to begin such an evaluation and expects to complete the evaluation in May 2006. Nevertheless, supervisory officers from five of the six CBP offices at the seaports we visited said that the mandatory training and certification program has been beneficial. These supervisory officers told us that the training and certification improves the confidence of targeters, provides the ability for officers to improve their targeting productivity, and provides an opportunity for officers to gain a broader perspective into the targeting environment by examining passenger and outbound targeting. Despite CBP Action to Address Longshoremen's Safety Concerns, Efficiency Concerns Remain on the West Coast: In our 2004 report,[Footnote 21] we discussed concerns that longshoremen had regarding the safety of driving cargo containers through the gamma ray imaging system, one type of nonintrusive inspection equipment used to examine containers to detect potential contraband or weapons of mass destruction. Because this equipment emits radiation as it takes images of the inside of cargo containers, some longshoremen expressed concerns about the health effects of this radiation. As a result of these safety concerns, the longshoremen's union representing West Coast longshoremen established a policy that prevents its members from driving containers through the gamma ray imaging system. In response, CBP altered its procedures at ports affected by this policy. For example, at some West Coast ports, CBP allows longshoremen to stage cargo containers away from the dock, in rows at port terminals, so that CBP officers can then drive the gamma ray imaging system over a group of containers. However, this procedure can be space-intensive and time-consuming compared to the procedure utilized at East and Gulf Coast ports, whereby the gamma ray imaging system machinery is operated by a CBP officer and parked in place while longshoremen drive the cargo containers through the machinery.[Footnote 22] At other West Coast ports, the longshoremen get out of the trucks after transporting the cargo containers so that CBP officials can drive the gamma ray imaging system cargo over the container. This is also time-consuming compared to the procedure utilized at the East and Gulf Coast ports. In response to our recommendation that CBP work with longshoremen to address their safety concerns, CBP engaged in two efforts: (1) establishing CBP's radiation threshold in accordance with the Nuclear Regulatory Commission's (NRC) federal guidelines for public radiation exposure and advertising this threshold to longshoremen through the unions, and (2) working with longshoremen's unions and other maritime organizations to develop public radiation tests on nonintrusive inspection equipment. Officials from the West Coast union that prohibits its members from driving through the gamma ray imaging system told us that the union is satisfied with CBP efforts to operate the gamma ray imaging system in an alternative format, to comply with the union's policy of receiving no amount of man-made radiation. Despite CBP efforts to assure this union that the amount of radiation emitted by the gamma ray imaging system is within safe levels, a union representative told us that CBP will not convince the union to change its policy unless it eliminates radiation emission from inspection equipment. In closing, ATS is an integral part of CBP's layered security strategy. A well-functioning ATS is crucial to the effective screening of cargo containers at domestic and CSI foreign ports, as well as cargo shipped by the trade community participating in C-TPAT. While CBP is working to make improvements to ATS, our ongoing work indicates that it is not yet in a position to gauge the effectiveness of ATS. We are continuing to review CBP's plans and actions to improve ATS and will report to this subcommittee and the other requesters later this year. GAO Contacts and Acknowledgments: Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. For further information about this testimony, please contact me at 202-512-8777 or at stanar@gao.gov. Debra Sebastian, Assistant Director; Chan-My J. Battcher; Lisa L. Berardi; Wayne A. Ekblad; and Jessica A. Evans made key contributions to this report. Additional assistance was provided by Frances Cook, Kathryn E. Godfrey, Nancy A. Hess, Arthur L. James, Jr., Stanley J. Kostyla, and Vanessa R. Taylor. [End of section] Appendix I: Scope and Methodology: To address each of our objectives, we met with U.S. Customs and Border Protection (CBP) officials in headquarters and six seaports including, Baltimore, Charleston, Los Angeles-Long Beach, Miami, New York-Newark, and Savannah. These seaports were selected based on the number of cargo containers arriving at the seaport and their geographic dispersion as reported by the U.S. Department of Transportation. At these locations, we also observed targeting and inspection operations. Because we did not select a random, probability sample of ports to visit, the results from these visits cannot be generalized to ports nationwide. We also spoke with CBP's contractor responsible for conducting CBP's peer review and longshoremen's union representatives. To evaluate how CBP provides assurance that the Automated Targeting System (ATS) targets the highest-risk oceangoing cargo containers for inspection, we reviewed CBP documentation and prior GAO work on performance measures. Additionally, we reviewed CBP's peer review report. To gain an understanding of CBP's random sampling program, we met with CBP officials responsible for this program and reviewed and analyzed CBP documentation, including procedures for examining the randomly selected shipments and documenting the results of the inspections completed for those shipments. We did not independently validate the reliability of CBP's targeting results. To assess how CBP adjusts ATS to respond to findings that occur during the course of its operational activities, we met with CBP officials responsible for gathering and disseminating intelligence and for incorporating intelligence into CBP's targeting operations. Further, we reviewed CBP policies and procedures on intelligence gathering and disseminating as well as intelligence received and resulting changes to ATS rules and weights. We did not assess the quality of intelligence received or the appropriateness of adjusted rules and weights. To determine how targeting officers' feedback and inspection results are used to adjust ATS rules and weights, we met with CBP officials responsible for collecting and maintaining data on suggestions provided by targeting officers and reviewed CBP data on the suggestions received over a 7 month period. Regarding inspection results, we reviewed CBP's policies and procedures for documenting inspection results. Additionally, we reviewed CBP's manuals identifying the specific details of an inspection completed and observed officers entering inspection results into the ATS findings module during our site visits. Further, during these visits, we discussed how CBP offices at the seaports may use inspection results to enhance their targeting efforts. Last, we met with CBP officials and reviewed CBP documentation on its current and planned findings module. To determine the status of recommendations from GAO's February 2004 report to (1) establish a testing and certification process for CBP staff who complete the national targeting training to provide assurance that they have sufficient expertise to perform targeting work and (2) work with longshoremen's unions to address fully their safety concerns so that the nonintrusive inspection equipment can be used to conduct inspections efficiently and safely, we reviewed and analyzed data on the number of officers trained and certified in sea cargo targeting. We also reviewed CBP's Sea Cargo Training Manual as well as CBP evaluations assessing the quality of its Sea Cargo Training course. We did not assess the quality of this training. Regarding longshoremen's union concerns, we reviewed scientific literature related to radiation safety and the Nuclear Regulatory Commission guidelines on radiation levels. We also spoke with longshoremen's representatives to discuss whether CBP had addressed their concerns since we issued our 2004 report. Last, we also met with CBP's Radiation Safety Officer to gain a further understanding of the potential risks associated with CBP's inspection equipment and actions he took to address longshoremen's concerns. We did not assess the appropriateness of radiation safety levels used by CBP. We conducted our work from October 2005 through March 2006 in accordance with generally accepted government auditing standards. FOOTNOTES [1] A model is a physical, mathematical, or otherwise logical representation of a system, entity, phenomenon, or process. [2] In addition to ATS, CBP's multilayered strategy includes the (1) Compliance Measurement Program, which randomly selects additional containers to be physically examined; (2) the Container Security Initiative, whereby CBP places staff at foreign seaports to work with foreign counterparts to inspect high-risk containers before they are shipped to the United States; and (3) the Customs-Trade Partnership Against Terrorism which is a cooperative program between CBP and members of the international trade community in which private companies agree to improve the security of their supply chains. A supply chain consists of all stages involved in fulfilling a customer request, including stages conducted by manufacturers, suppliers, transporters, retailers, and customers. [3] GAO, Homeland Security: Challenges Remain in the Targeting of Oceangoing Cargo Containers for Inspection, GAO-04-352SU (Washington, D.C.: Feb. 20, 2004). [4] GAO, Homeland Security: Summary of Challenges Faced in Targeting Oceangoing Cargo Containers for Inspection, GAO-04-557T, (Washington, D.C.: Mar. 31, 2004). [5] External peer review is a process that includes an assessment of the model by independent and qualified external peers. [6] For purposes of this statement, when we state that CBP uses ATS to target oceangoing cargo containers to identify weapons of mass destruction, we are also including the different components that could be used to create a weapon of mass destruction. [7] Internal control is an integral component of an organization's management that provides reasonable assurance that the following objectives are achieved: (1) effectiveness and efficiency of operations, (2) reliability of financial reporting, and (3) compliance with applicable laws and regulations. [8] Nonintrusive inspection equipment uses technology to help determine the contents of a container without opening it. [9] Cargo manifests are prepared by the ocean carrier to describe the contents of a container. [10] The Comptroller General's internal control standards state that internal control activities help ensure that management's directives are carried out. Further, they state that the control activities should be effective and efficient in accomplishing the agency's control objectives. GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1, p. 11 (Washington, D.C.: November 1999). [11] See GAO/AIMD-00-21.3.1, pps. 11 and 14. [12] Entry information is documentation to declare items arriving in the United States. Entry information allows CBP to determine what is included in a shipment. Entry information provides more detail on a container's contents than manifest information. [13] See GAO/AIMD-00-21.3.1, p. 18. [14] See GAO-04-352SU. [15] See GAO-04-352SU. [16] See GAO-04-352SU. [17] The intelligence community is a federation of executive branch agencies and organizations that work separately and together to conduct intelligence activities necessary for the conduct of foreign relations and the protection of the national security of the United States. [18] A query is a search an individual officer creates to seek information from ATS about shipments and containers based on specific criteria to assist in the officer's targeting decisions. A lookout is a query that CBP headquarters or officers at the seaports can create that will notify all officers making targeting decisions when a shipment's manifest data are similar to or match the search criteria. . [19] See GAO-04-352SU. [20] A CBP official estimated that CBP has approximately 300 officers responsible for targeting oceangoing cargo containers. However, CBP is currently surveying its offices to determine a more precise estimate and will have this information available within the next month. [21] See GAO-04-352SU. [22] See GAO-04-352SU.