This is the accessible text file for GAO report number GAO-15-294 entitled 'Supply Chain Security: CBP Needs to Enhance Its Guidance and Oversight of High-Risk Maritime Cargo Shipments' which was released on January 27, 2015. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to the Ranking Member, Committee on Homeland Security, House of Representatives: January 2015: Supply Chain Security: CBP Needs to Enhance Its Guidance and Oversight of High-Risk Maritime Cargo Shipments: GAO-15-294: GAO Highlights: Highlights of GAO-15-294, a report to the Ranking Member, Committee on Homeland Security, House of Representatives. Why GAO Did This Study: The U.S. economy is dependent on a secure global supply chain. In fiscal year 2013, approximately 12 million maritime cargo shipments arrived in the United States. Within the federal government, CBP is responsible for administering cargo security, to include identifying “ high-risk” maritime cargo shipments with the potential to contain terrorist contraband. GAO was asked to review CBP's disposition of such shipments. This report discusses (1) how many maritime shipments CBP determined to be high risk and the extent to which CBP has accurate data on the disposition of such shipments, (2) the extent to which CBP consistently applies criteria and documents reasons for waiving examinations, and (3) the extent to which CBP ensures its policies on the disposition of high-risk shipments are being followed. GAO analyzed CBP data on maritime shipments arriving in the United States during fiscal years 2009 through 2013. GAO also visited four CBP targeting units selected on the basis of the percentage of maritime shipments they waived, among other factors. What GAO Found: From fiscal years 2009 through 2013, less than 1 percent of maritime shipments arriving in the United States were identified as high risk by U.S. Customs and Border Protection (CBP), but CBP does not have accurate data on their disposition (i.e., outcomes). CBP officials (targeters) are generally required to hold high-risk shipments for examination unless evidence shows that an examination can be waived per CBP policy. In particular, targeters at Advance Targeting Units (targeting units)—-responsible for reviewing shipments arriving at ports within their respective regions—-can waive an examination if they determine through research that (1) the shipment falls within a predetermined category (standard exception), or (2) they can articulate why the shipment should not be considered high risk (articulable reason), such as an error in the shipment's data. GAO found that CBP examined the vast majority of high-risk shipments, but CBP's disposition data are not accurate because of various factors— such as the inclusion of shipments that were never sent to the United States—and that the data overstate the number of high-risk shipments. On the basis of GAO's analyses and findings, CBP is taking steps to correct its data. When determining the disposition of high-risk shipments, CBP's targeting units are inconsistently applying criteria to make waiver decisions and are incorrectly documenting the reasons for some waivers. CBP policy lacks definitions for standard exception waivers. As a result, targeters are inconsistently applying and recording standard exception waivers. Because of these inconsistencies, some targeting units may be unnecessarily holding shipments for examination, while others may be waiving shipments that should be examined. Developing definitions for standard exceptions could help ensure that CBP examines shipments as intended. Further, some targeters at targeting units GAO visited were unaware of the guidance on articulable reason waivers and were incorrectly documenting these waivers. As a result, CBP cannot accurately determine the extent to which articulable waivers are being issued and used judiciously per CBP policy. Updating and disseminating guidance in policy could help ensure targeters correctly document such waivers. CBP has efforts in place, such as self-inspections, to provide oversight of its policies on the disposition (whether examined or waived) of high-risk shipments; however, these efforts are not sufficient. For example, the limited sample size of shipments used in self-inspections does not provide CBP with the best estimate of compliance at the national level. In addition, CBP's method for calculating the compliance rate does not accurately reflect compliance because it is not based on the number of shipments sampled. Developing an enhanced methodology for selecting sample shipments, and changing the method for calculating compliance, could improve CBP's estimate of compliance and its ability to identify and correct deficiencies. This is a public version of a sensitive report that GAO issued in November 2014. It does not include details that CBP deemed sensitive security information. What GAO Recommends: GAO recommends, among other things, that CBP define standard exception waiver categories and disseminate policy on documenting articulable reason waivers. Further, CBP should enhance its methodology for selecting shipments for self-inspections and change the way it calculates the compliance rate. The Department of Homeland Security concurred with GAO's recommendations. View GAO-15-294. For more information, contact Jennifer Grover at (202) 512-7141 or groverj@gao.gov. [End of section] Contents: Letter: Background: Less than 1 Percent of Arriving Maritime Shipments from Fiscal Years 2009 through 2013 Were High Risk, but CBP Does Not Have Accurate Data on Their Disposition: CBP Targeting Units Are Not Consistently Applying and Documenting Reasons for Waiving Examinations of High-Risk Shipments: CBP's Mechanisms to Oversee the Implementation of Disposition Policies Could Be Enhanced: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Description of CBP's Layered Security Strategy for Maritime Cargo Shipments: Appendix II: Objectives, Scope, and Methodology: Appendix III: Comments from the Department of Homeland Security: Appendix IV: GAO Contact and Staff Acknowledgments: Related GAO Products: Tables: Table 1: Disposition of Selected High-Risk Shipments Identified as Waived, Fiscal Year 2013: Table 2: Disposition of Selected High-Risk Shipments Identified as Not Examined/Not Waived, Fiscal Year 2013: Table 3: Description of U.S. Customs and Border Protection's (CBP) Core Cargo Security Programs: Figure: Figure 1: Flow Chart Depicting U.S. Customs and Border Protection's (CBP) Targeting Outcomes for High-Risk Shipments: Abbreviations: ATS: Automated Targeting System: CBP: U.S. Customs and Border Protection: CERTS: Cargo Enforcement Reporting and Tracking System: CSI: Container Security Initiative: C-TPAT: Customs-Trade Partnership Against Terrorism: DHS: Department of Homeland Security: GPRA: Government Performance and Results Act: NII: nonintrusive inspection: OFO: Office of Field Operations: SAFE Port Act: Security and Accountability for Every Port Act: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: January 27, 2015: The Honorable Bennie G. Thompson: Ranking Member: Committee on Homeland Security: House of Representatives: Dear Mr. Thompson: The U.S. economy is dependent on the expeditious flow of millions of tons of cargo each day through the global supply chain--the flow of goods from manufacturers to retailers. According to the U.S. Department of Transportation, the majority of U.S. imports arrive by ocean vessel.[Footnote 1] In fiscal year 2013, approximately 12 million cargo shipments arrived at U.S. seaports from foreign ports. Criminal or terrorist attacks using cargo shipments can cause disruptions to the supply chain and can limit global economic growth and productivity.[Footnote 2] Within the federal government, U.S. Customs and Border Protection (CBP), part of the Department of Homeland Security (DHS), is responsible for administering cargo security and reducing the vulnerabilities associated with the supply chain. According to DHS, balancing security concerns with the need to facilitate the free flow of commerce, part of CBP's mission, remains an ongoing challenge.[Footnote 3] CBP has developed a layered security strategy that uses a risk-based approach to focus its limited resources on targeting and examining high-risk cargo shipments that could pose a risk while allowing other cargo shipments to proceed without unduly disrupting commerce arriving in the United States. CBP's layered security strategy is based on programs that include, among other things, analyzing information to identify shipments that may be at high risk of transporting weapons of mass destruction or other contraband, working with foreign governments to examine U.S.-bound high-risk shipments at foreign ports participating in the Container Security Initiative (CSI), and providing benefits to companies that comply with predetermined security measures through the Customs-Trade Partnership Against Terrorism (C-TPAT) program. In October 2006, Congress passed and the President signed the Security and Accountability for Every Port Act (SAFE Port Act) of 2006, establishing a statutory framework for key programs within CBP's layered security strategy that previously had been CBP initiatives not specifically required by law.[Footnote 4] A brief description of the core programs that constitute CBP's layered security strategy is provided in appendix I. An important element of CBP's layered security strategy is obtaining advance cargo information to help identify shipments that are potentially high risk for containing terrorist contraband. CBP's disposition options (i.e., outcomes) for high-risk shipments include examinations or waivers. According to CBP policy, high-risk shipments are to be examined, at a minimum, by radiation detection and nonintrusive inspection (NII) equipment.[Footnote 5] However, the policy also allows CBP to waive examination of a high-risk shipment if it meets certain criteria. Specifically, CBP officials at ports can waive an examination if they determine through research that (1) the shipment falls within a predetermined category of stated exceptions (standard exception), or (2) they can articulate why the shipment should not be considered high risk (articulable reason). For example, a shipment could be identified as high risk because it is associated with a shipper on a terrorist watch list, but through further research, CBP officials determine the shipper is not a true match to the terrorist watch list and, therefore, the shipment should not be considered high risk. In response to your request, we reviewed CBP's disposition of high- risk maritime shipments in recent years. Specifically, this report addresses the following objectives: 1. How many maritime shipments arriving in the United States from fiscal years 2009 through 2013 did CBP determine to be high risk and to what extent does CBP have accurate data on the disposition of each of those high-risk shipments? 2. To what extent is CBP consistently applying standards and documenting reasons for waiving examinations of high-risk shipments? 3. To what extent does CBP ensure that its policies on the disposition of high-risk shipments are being followed? This report is a public version of a prior sensitive report that we provided to you. CBP deemed some of the information in the prior report sensitive security information (SSI), which must be protected from public disclosure.[Footnote 6] Therefore, this report omits sensitive information regarding the specific number of shipments examined and waived, and data reliability issues, among other things. The information provided in this report is more limited in scope, as it excludes such sensitive information, but it addresses the same questions as the sensitive report and the overall methodology used for both reports is the same. To determine the number of maritime shipments arriving in the United States that CBP determined to be high risk, we obtained and analyzed data from CBP on the number of shipments and high-risk shipments, by seaport, during fiscal years 2009 through 2013--the 5 most recent fiscal years for which full-year data were available at the time of our review. To determine the extent to which CBP has accurate data on the disposition of each high-risk shipment, we analyzed CBP's data to determine the number of high-risk shipments examined and waived, as well as those shipments not examined/not waived. We then reviewed the data for obvious errors, discussed how the data were compiled with CBP officials, and determined the accuracy of a subset of both waived and not examined/not waived shipments from CBP's fiscal year 2013 data by further researching the disposition of each selected shipment with the assistance of CBP officials. We determined that the data (1) were not sufficiently reliable at the seaport level because not all shipments were associated with the correct seaport where shipments were unloaded, and (2) as discussed later in this report, contain errors that affect their reliability for calculating the number of high-risk shipments and their disposition. We did determine, though, that the data are sufficiently reliable to illustrate the overall disposition of all high-risk shipments by category--examined, waived, and not examined/not waived. We analyzed data we received from CBP's Fines, Penalties, and Forfeitures Division on "gate outs"--cargo targeted for terrorism or enforcement reasons that is released from CBP custody and departs a port without authorization or examination--for fiscal years 2009 through 2013 in order to determine the frequency of gate out occurrences relative to the overall number of not waived/not examined shipments. We discussed the process for collecting and recording gate out data with CBP officials from the Fines, Penalties, and Forfeitures Division and determined that the gate out data were sufficiently reliable for reporting the number of gate out occurrences. To determine the extent to which CBP is consistently applying standards and documenting reasons for waiving examinations of high- risk shipments, we analyzed CBP's data on high-risk shipment waivers during fiscal year 2013, reviewed CBP policies and guidance for waiving examinations of high-risk shipments, and observed and discussed waiver practices at the Advance Targeting Units (targeting units) we visited.[Footnote 7] We conducted site visits to four targeting units that appeared to issue the greatest percentage of waivers relative to the total high-risk maritime cargo shipments that arrived at their respective seaports in fiscal year 2013.[Footnote 8] Although the results from our visits to these four targeting units are not generalizable to all targeting units, the visits allowed us to understand whether waiver documentation practices are consistent across these targeting units, and how such practices affect the reliability of CBP's disposition data. In order to determine whether targeting units were consistently using uniform criteria when applying standard exception waivers to high-risk shipments, we asked CBP officials at each targeting unit we visited to (1) define certain standard exception waivers listed in CBP's National Maritime Targeting Policy, and (2) comment on their awareness of any existing CBP guidance regarding standard exceptions[Footnote 9]. We compared CBP's practices relative to standard exception waivers against standards in Standards for Internal Control in the Federal Government.[Footnote 10] We also spoke with CBP targeting unit officials about their processes and practices for reviewing, approving, and documenting articulable reason waivers for high-risk shipments. We compared the processes and practices they use with CBP's policy for approving such waivers in the National Maritime Targeting Policy and CBP's guidance on documenting articulable reason waivers as prescribed in a February 2007 memorandum.[Footnote 11] In order to determine the extent to which CBP ensures that its policies on the disposition of high-risk shipments are being followed, we met with CBP officials at both headquarters and targeting units responsible for the management of high-risk maritime shipments and discussed their oversight of CBP's policies. We also met with officials from CBP's Office of Field Operations (OFO) to discuss their design and implementation of the Self-Inspection Program for sea cargo targeting, including the shipment sample size used to assess compliance and method for calculating compliance. In addition, we reviewed CBP reports summarizing the results of self-inspections, including the compliance rate and individual reports submitted by the four targeting units we visited. We then compared CBP's methodology for conducting self-inspections with best practices outlined in guidance on design evaluations.[Footnote 12] Further, we reviewed quarterly reports produced by CBP outlining the disposition of high- risk shipments and discussed how these reports are used to determine whether CBP is meeting a Government Performance and Results Act (GPRA) performance goal with officials from OFO. Finally, we compared CBP's efforts to assess the accuracy of its data used for GPRA with standards in Standards for Internal Control in the Federal Government. [Footnote 13] Additional details regarding our scope and methodology are provided in appendix II. We conducted this performance audit from January 2014 through January 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: Identifying High-Risk Shipments: Through what is referred to as the 24-hour rule, CBP generally requires vessel carriers to electronically transmit cargo manifests to CBP 24 hours before cargo is loaded onto U.S.-bound vessels at foreign ports.[Footnote 14] Through the Importer Security Filing and Additional Carrier Requirements (known as the 10+2 rule), CBP requires importers and vessel carriers to provide data elements for improved identification of cargo shipments that may pose a risk for terrorism.[Footnote 15] Importers are responsible for supplying CBP with 10 shipping data elements--such as country of origin--24 hours prior to loading, while vessel carriers are required to provide 2 data elements--container status messages and stow plans--that are not required by the 24-hour rule.[Footnote 16] The data provided by carriers and importers in compliance with the 24- hour rule and the 10+2 rule are automatically fed into CBP's Automated Targeting System (ATS)--an enforcement and decision support system that compares cargo and conveyance information against intelligence and other law enforcement data. ATS consolidates data from various sources to create a single, comprehensive record for each U.S.-bound shipment. Among other things, ATS uses a set of rules that assess different factors in the data to determine the risk level of a shipment. One set of rules within ATS, referred to collectively as the maritime national security weight set, is programmed to check for information or patterns that could be indicative of suspicious or terrorist activity. Each rule in the set has a specific weight value assigned to it, and for each risk factor that the rules identify, the weight values are added together to calculate an overall risk score for the shipment. ATS assesses and generates risk scores for every cargo shipment as the shipment moves throughout the global supply chain and new data are provided or existing data are revised. CBP classifies the risk scores from the maritime national security weight set as low, medium, or high risk. Shipments with connections to known or suspected terrorists, as well as those that include invalid information, are more likely to be classified as high risk; and shipments from shippers who participate in CBP's C-TPAT program, or "trusted shippers," are more likely to be classified as low risk. Because ATS collects and presents data on shipments, CBP targets shipments--rather than individual containers--for examination. [Footnote 17] Targeting High-Risk Shipments: ATS automatically places high-risk shipments on hold, and CBP officials (targeters) use information in ATS to identify (target) which high-risk shipments should be examined or waived.[Footnote 18] If a shipment is held for examination, a CBP Anti-Terrorism Contraband Enforcement Team (enforcement team) is to conduct the examination, which is to include scanning the cargo with NII equipment, among other things.[Footnote 19] Enforcement team officials are to review the images produced with the NII equipment to detect anomalies or shielding that could indicate the presence of weapons of mass destruction or other contraband. If an anomaly is detected, the shipment is to be transferred to a centralized examination station and the contents of the container are to be removed and physically examined. If contraband is discovered during the physical examination, the shipment is to be seized by CBP; otherwise it is to be released. The enforcement team is responsible for recording the examinations it conducts, as well as the results, in the Cargo Enforcement Reporting and Tracking System (CERTS)--a module within ATS. Examinations of high-risk shipments can be waived if a CBP targeter determines that a high-risk shipment meets a "standard exception" or an "articulable reason."[Footnote 20] CBP policy lists the standard exceptions to mandatory examinations.[Footnote 21] Waivers based on articulable reasons are issued for reasons other than the standard exception categories. If a CBP targeter conducts analysis of available information on a high-risk shipment and determines there is no security risk, he or she is to seek approval from the port director or his/her designee(s) and record the waiver reason in CERTS within ATS. Figure 1 depicts possible targeting outcomes for high-risk shipments. Figure 1: Flow Chart Depicting U.S. Customs and Border Protection's (CBP) Targeting Outcomes for High-Risk Shipments: [Refer to PDF for image: flow chart] Automated Targeting System (ATS) places high-risk shipment on hold: Targeter reviews manifest and shipping data: 3 potential flows: 1. Targeter keeps shipment on hold for examination; Enforcement team scans shipment with nonintrusive inspection equipment: No anomaly identified: Shipment is released from the port; Anomaly is identified: Shipment is physically examined[A]; Enforcement team records examination results in ATS[B]; Shipment is released from the port. 2. Targeter determines shipment meets a standard exception in CBP policy and records the exception in ATS[B]: Shipment is released from the port. 3. Targeter requests a waiver based on an articulable reason; Port director or designee reviews the waiver request and approves the waiver; Targeter records waiver in ATS[B]; Shipment is released from the port. Source: GAO analysis of CBP documents. GAO-15-294. [A] If contraband is discovered during the physical examination, the shipment is to be seized by CBP. [B] Examinations and waivers are recorded in the Cargo Enforcement Reporting and Tracking System (CERTS)--a module within ATS. [End of figure] In addition to obtaining manifest and shipping data (e.g., 10+2 data), CBP requires the importers of goods to file entry documents so CBP can assess and collect duties. Data provided in the entry documents are assessed by ATS and can result in a shipment's risk score previously classified as low or medium risk based on manifest and shipping data to become high risk. Entry documents can also have the opposite effect. For example, entry information can confirm an entity is a C- TPAT member and, therefore, drop the shipment's score below the high- risk threshold. Entry documents can be provided several days after a shipment's arrival in the United States and after a shipment leaves the port.[Footnote 22] Advance Targeting Units: CBP targeters are assigned to targeting units located at or near selected domestic ports, and their targeting efforts are focused on shipments destined for ports within their respective regions. A targeting unit may be responsible for targeting shipments arriving at one port or multiple ports in its region. For example, targeters at the Port of Newark are also responsible for targeting shipments that are bound for ports in New York. CBP targeters at targeting units can review data as soon as carriers and importers submit the required data (in accordance with the 24-hour rule and the 10+2 rule), and the data are available in ATS. Once a shipment is loaded onto a U.S.-bound vessel, CBP targeters continue to review shipment data in ATS because shipment data can be updated with additional or amended information. Targeters use other sources, such as public records, open sources (e.g., Internet search engines), U.S. government systems, and local port knowledge to assess whether the shipment poses a high risk or whether the risk can be mitigated based on research. 100 Percent Scanning and Prior GAO Work: In August 2007, the Implementing Recommendations of the 9/11 Commission Act of 2007 (9/11 Commission Act) was enacted, which required, among other things, that by July 2012, 100 percent of U.S.- bound cargo containers be scanned at foreign ports with both radiation detection and NII equipment before being placed on U.S.-bound vessels. [Footnote 23] In May 2012, the then secretary of homeland security authorized a 2-year extension (until July 2014) of the deadline for implementing the requirement.[Footnote 24] In May 2014, the Secretary of Homeland Security renewed the extension (until July 2016) and stated that "DHS's ability to fully comply with this unfunded mandate of 100 percent scanning, even in [the] long term, is highly improbable, hugely expensive, and in our judgment, not the best use of taxpayer resources to meet this country's port security and homeland security needs." The Secretary also stated that he instructed DHS, including CBP, to do a better job of meeting the underlying objectives of the 100 percent scanning requirement by, in part, refining aspects of CBP's layered security strategy. We have previously reported on the challenges CBP faces in implementing the 100 percent scanning requirement.[Footnote 25] In October 2009, we recommended, among other things, that CBP conduct feasibility and cost-benefit analyses of implementing the 100 percent scanning requirement and provide the results to Congress along with any suggestions of cost-effective alternatives to implementing the 100 percent scanning requirement, as appropriate. CBP partially concurred with the recommendations but did not implement them. We have also reported on the programs that compose CBP's layered security strategy. Specifically, we have reviewed CBP's efforts to collect additional data through the 10+2 rule and utilize these data to identify high- risk shipments;[Footnote 26] examine high-risk shipments before they depart CSI ports;[Footnote 27] and validate security measures taken by C-TPAT members.[Footnote 28] We made several recommendations in these reports, including that CBP establish milestones and time frames for including 10+2 data in its criteria used in the identification of high- risk shipments. In December 2010, CBP provided us with a project plan for integrating the data into its criteria, and in early 2011, CBP implemented the updates to address risk factors present in the 10+2 data. Less than 1 Percent of Arriving Maritime Shipments from Fiscal Years 2009 through 2013 Were High Risk, but CBP Does Not Have Accurate Data on Their Disposition: We determined that less than 1 percent of the maritime shipments arriving in the United States from fiscal years 2009 through 2013 were high risk; however, CBP does not have accurate data on the number and disposition of each high-risk shipment because of various factors. On the basis of our analyses, CBP's data overstate the number of high- risk shipments, including those not examined/not waived. CBP is taking steps to improve its data on the disposition of high-risk shipments. Percentage of High-Risk Shipments Is Very Low and CBP Has Examined the Vast Majority: On the basis of our analyses of CBP data for fiscal years 2009 through 2013, on average each year, approximately 11.6 million maritime shipments arrived in the United States, and less than 1 percent of those were determined by ATS to be high risk based on the maritime national security weight set. CBP, on average, examined the vast majority of these high-risk shipments, with less than 10 percent waived or not examined/not waived.[Footnote 29] CBP Does Not Have Accurate Data on the Disposition of All High-Risk Shipments: The numbers and percentages discussed above represent CBP's data on the number of high-risk shipments and their disposition, but our analyses suggest that CBP does not have accurate data on the disposition of each high-risk shipment because of various factors. [Footnote 30] In particular, CBP's data overstate the number of high- risk shipments, including those not examined/not waived. On the basis of our analysis of selected waived shipments, CBP's data may also overstate the number of waived shipments since not all shipments identified as waived were waived, but we were unable to determine the full extent to which some shipments identified as not waived/not examined were actually waived. CBP officials stated that the data include (1) shipments where the carrier deleted the bill of lading, meaning the shipments ultimately never arrived in the United States (referred to as deleted bills), and (2) shipments for which ATS assigned high risk scores only after entry was filed and the shipment had been released from the port. In further iterations of the data CBP provided us, CBP officials were able to identify the shipments in these two categories and therefore provide us with more accurate data on the number and disposition of high-risk shipments. On the basis of our analyses of CBP's fiscal year 2009 through 2013 data, deleted bills and shipments' risk scores not rising above the high-risk threshold until after entry accounted for 8 percent, on average, of the high-risk shipments identified in CBP's data as waived. However, such shipments were not high-risk shipments requiring review by a targeter and therefore would not have been waived. Further, our analyses of CBP's data showed that deleted bills and risk scores not rising above the high-risk threshold until after entry accounted for 48 percent, on average, of the high-risk shipments identified in CBP's data as not examined/not waived. Therefore, nearly half of the shipments identified as not examined/not waived in CBP's data were, in fact, not high-risk shipments requiring an examination or waiver even though they were identified as such. In addition to deleted bills and shipments with high risk scores only after entry was filed, we also identified other factors contributing to CBP not having accurate disposition data on high-risk shipments. We discussed a nonprobability sample of high-risk shipments with CBP officials at the four targeting units we visited. Specifically, we discussed two sets of shipments--40 high-risk shipments identified in CBP's data as waived, and 40 shipments identified as not examined/not waived.[Footnote 31] We found that CBP did not have accurate disposition data for the 40 high-risk shipments identified as waived since 28 shipments were actually waived. For example, we determined that 3 shipments were examined, but the examinations were not recorded by CBP officials in CERTS within ATS. According to CBP officials at one targeting unit we visited, the 3 shipments were not recorded because of confusion over who was to record the examination. See table 1 for the actual disposition of the 40 high-risk shipments we analyzed that were identified in CBP's data as waived. Table 1: Disposition of Selected High-Risk Shipments Identified as Waived, Fiscal Year 2013: Disposition: Waived; Number of shipments: 28. Disposition: Examined, but not recorded by officials; Number of shipments: 3. Disposition: Not examined/not waived; Number of shipments: 2. Disposition: Not high risk; * Score was below threshold at time of arrival or did not rise above high-risk threshold until after arrival and entry was filed; Number of shipments: 5; * Deleted bill; Number of shipments: 2. Disposition: Total; Number of shipments: 40. Source: GAO analysis of U.S. Customs and Border Protection data and information. GAO-15-294. [End of table] We determined that of the 40 high-risk shipments identified in CBP's data as not examined/not waived, 1 should have been examined, and 1 waived. The remaining 38 shipments were incorrectly identified as not examined/not waived for various reasons, including 5 shipments that were examined or waived and properly recorded, but ATS did not link the records to the shipments. See table 2 for the actual disposition of the 40 high-risk shipments identified in CBP's data as not examined/not waived. Table 2: Disposition of Selected High-Risk Shipments Identified as Not Examined/Not Waived, Fiscal Year 2013: Disposition: Not examined/not waived; Number of shipments: 2. Disposition: Waived or examined, but not recorded properly by officials; Number of shipments: 7. Disposition: Waived or examined, but record not linked to shipment in the Automated Targeting System; Number of shipments: 5. Disposition: Not high risk; * Score was below threshold at time of arrival or did not rise above high-risk threshold until after arrival and entry was filed; Number of shipments: 16; * Deleted bill; Number of shipments: 10. Disposition: Total; Number of shipments: 40. Source: GAO analysis of U.S. Customs and Border Protection data and information. GAO-15-294. [End of table] CBP data on gate out occurrences--cargo targeted for terrorism or enforcement that is released from CBP custody and departs a port without authorization or examination--also call into question the accuracy of CBP's disposition data on high-risk shipments identified as not examined/not waived. CBP requires ports to have a process in place for identifying gate outs and, in 2006, developed a uniform process for ports to report gate outs to CBP's OFO. According to CBP data collected through this process, the number of gate out occurrences is far less than the number of high-risk shipments identified as not examined/not waived, which would equate to a gate out. The number of not examined/not waived high-risk shipments should, in theory, be the same as the number of gate outs. In response to our findings, CBP officials acknowledged that the factors discussed above have contributed to CBP's data not being accurate, and they noted that CBP is taking steps to improve its data on the disposition of high-risk shipments. For example, CBP has already developed a query to identify shipments in its data that are not truly high risk at the time of arrival, including deleted bills and shipments with high risk scores only after arrival and entry is filed. CBP officials added that they are updating the National Maritime Targeting Policy to include the requirement that cargo examinations and waivers be recorded in CERTS within ATS since not all officials are adhering to this requirement and the policy will be finalized after we complete our review. In addition, enhancing certain oversight mechanisms, as discussed later in the report, could help address the inaccuracies in CBP's data. CBP Targeting Units Are Not Consistently Applying and Documenting Reasons for Waiving Examinations of High-Risk Shipments: When determining the disposition of high-risk shipments, CBP's targeting units are inconsistently applying criteria to make some waiver decisions and are also incorrectly documenting the reasons for waivers. On the basis of our review of CBP policy and visits to selected targeting units, we determined that CBP has not established uniform definitions for standard exception waiver categories; some CBP officials were unaware of existing waiver guidance for articulable reason waivers; and some CBP targeters across the targeting units we visited were inconsistently and inaccurately recording waiver reasons in ATS. As a result, CBP cannot accurately determine the extent to which standard exception waivers are used consistently or whether waivers issued for articulable reasons are being used judiciously, as required by policy. Targeting Units Have Varying Definitions of What Constitutes Certain Standard Exceptions: CBP's National Maritime Targeting Policy lists several standard exception waivers, and we found inconsistencies regarding how certain standard exception waiver categories are defined across the four targeting units we visited.[Footnote 32] At these targeting units, we found that CBP targeters consistently review manifest and shipping data, including data provided through the 10+2 rule, to search for evidence that would indicate whether or not a shipment should be waived based on a standard exception. However, the criteria targeters at these targeting units are using to make these determinations are not uniformly established by any central CBP guidance or policy. Instead, they are developed locally by targeting unit officials primarily on the basis of their experience and institutional knowledge of the targeting process. Although CBP's National Maritime Targeting Policy identifies what the standard exception categories are, it does not provide definitions for what specifically constitutes the various standard exception categories. Because of the lack of CBP-wide definitions for the standard exception categories, CBP targeters may be holding some shipments for examinations that should be waived. Alternatively, CBP targeters may also be waiving shipments that should have been examined. Differences in how frequently targeting units receive certain types of shipments may also affect the inconsistent interpretation of standard exceptions, and ultimately influence the variance seen in the proportion of high-risk shipments targeting units waive.[Footnote 33] Defining standard exception waiver categories and disseminating those definitions in policy would better allow targeting units and targeters to consistently apply criteria when making and recording waiver decisions, and could help ensure that CBP is examining shipments as intended. It is key that government agencies implement effective internal controls in order to minimize operational problems and achieve desired program results. According to Standards for Internal Control in the Federal Government, control activities help ensure that management's directives are carried out.[Footnote 34] The control activities should be effective and efficient in accomplishing the agency's objectives. Examples of control activities include establishment and review of performance measures and indicators, accurate and timely recording of transactions and events, and appropriate documentation of transactions and internal controls. CBP officials acknowledged that establishing definitions for standard exceptions could reduce the inconsistent interpretation and documentation of standard exception waivers. CBP Is Unable to Determine if Its Policy Addressing Articulable Reason Waivers Is Being Followed: All four of the targeting units we visited had a process in place for requesting and obtaining approvals for waivers based on articulable reasons in accordance with CBP policy. The four targeting units we visited each employed one of two different but comparable methods for reviewing and approving waivers. The first method involved the use of hard copy forms to request waivers, and the second method involved the use of e-mail requests for making waiver requests. Regardless of which method was used, each of the targeting units we visited requested and obtained waiver approvals in a manner that is consistent with CBP's National Maritime Targeting Policy, which states that the port director or his/her designee(s) is/are responsible for reviewing and approving high-risk shipment waiver requests based on articulable reasons.[Footnote 35] While the targeting units we visited had proper procedures in place for requesting and approving waivers, we found that CBP targeters at those targeting units are not correctly documenting waivers based on articulable reasons in accordance with CBP guidance. CBP issued a memorandum in February 2007 that provides guidance on how targeters are to record waivers in ATS based on articulable reasons. According to that guidance, targeters are to select a specific drop-down menu option in CERTS as the reason for every articulable reason waiver they issue and then targeters are to provide comments in CERTS to support the justification for the waiver. However, during our visits to the targeting units, and on the basis of conversations we had with CBP targeters, we found that targeters' understanding of how to record waivers in CERTS within ATS varied. Our analysis of selected waived shipments (as previously discussed in this report) and discussions with CBP targeters indicate that some targeters are incorrectly recording some waiver reasons in ATS because they are not familiar with the February 2007 CBP memorandum that specifies how articulable reason waivers are to be recorded. According to a senior CBP official, the National Maritime Targeting Policy, which was last disseminated prior to the implementation of CERTS, has not been updated to address the process for recording articulable waivers in CERTS. Neither we nor CBP could easily determine the full extent to which articulable waiver reasons were recorded properly because waivers based on articulable reasons cannot be segregated from waivers issued for standard exceptions. The inconsistent recording of articulable reason waivers in ATS limits CBP's ability to determine whether targeting units are following policy, since CBP's National Maritime Targeting Policy states that waivers based on articulable reasons are to be used "judiciously." In order to evaluate whether targeting units are judiciously making waiver decisions based on articulable reasons, CBP would need accurate records in ATS regarding the basis for why shipments were waived in order to be able to easily distinguish the two types of waivers. In particular, CBP is reliant on targeters' selecting the appropriate waiver reasons from the drop-down menu selections in CERTS in order to be able to accurately analyze waiver data. Although the comments in the remarks section of CERTS justifying the waiver may provide further details regarding the reasons for waivers, it would be difficult for CBP to accurately determine waiver reasons on the basis of large-scale data queries of comments in the remarks section alone. Therefore, it is important for targeters to select the proper drop-down menu options when documenting waiver reasons. According to Standards for Internal Control in the Federal Government, management must continually assess and evaluate its internal controls to ensure that the control activities being used are effective and updated when necessary.[Footnote 36] Updating and disseminating guidance in policy on how to record articulable reason waivers will help ensure that they are correctly recorded. CBP's Mechanisms to Oversee the Implementation of Disposition Policies Could Be Enhanced: CBP has some mechanisms to provide oversight of its policies on the disposition of high-risk shipments, such as biannual self-inspections; however, these are not sufficient to fully identify whether officials are complying with policy on examinations and waivers. Further, CBP could enhance the quality of its reports on the disposition of high- risk shipments. CBP Has Mechanisms to Determine if Its Policies Are Being Followed: CBP's OFO has mechanisms to determine if CBP policies on the disposition of high-risk shipments are being followed. Specifically, OFO monitors compliance with policies on the disposition of high-risk shipments through three efforts: * Self-Inspection Program: OFO requires port directors (or their designees) to complete self-inspection worksheets on cargo targeting every other year to determine whether CBP officials are following policy when it comes to examining high-risk shipments and recording those examinations. In addition to reporting any deficiencies, the inspection reports sent to OFO are to include the corrective actions taken to address the deficiencies identified. * Quarterly performance reports: CBP compiles data from ATS on high- risk shipments to measure CBP's performance in reviewing high-risk shipments in support of GPRA.[Footnote 37] Among other things, the quarterly reports provide data on the number of high-risk shipments that arrived at each U.S. seaport, including the number of shipments reviewed by a targeter and the number waived. * Monitoring gate outs: CBP requires ports to uniformly report all gate out occurrences to OFO as soon as the gate out is discovered. The local port submits a notification worksheet to the Office of Cargo, Conveyance and Security (within OFO) that, in turn, determines if a gate out truly occurred. However, as discussed below, we found weaknesses in some of the mechanisms CBP uses to provide oversight of its policies. CBP Needs to Improve Oversight of Its Policies to Better Identify ATS Data Deficiencies: We found that CBP's efforts are not sufficient to identify when officials are not following policies on high-risk shipments and, subsequently, deficiencies in how data are recorded in ATS, including examinations and waivers. The Self-Inspection Program requires port directors or their designees to analyze selected shipments and complete a worksheet composed of three questions to check compliance with policies on examining high-risk shipments and recording examinations. The three worksheet questions are as follows: 1. Were cargo examination findings input within CERTS and, when appropriate, within other systems? 2. Were all shipments that received an ATS score at or above the national security threshold score placed on hold? 3. Did all shipments that received an ATS score at or above the national security threshold undergo a mandatory examination utilizing, at a minimum, an NII imaging system and screening with radiation detection technology? The guidance provided on how to select shipments to answer the three questions, depending on the question, states that 10 percent of all shipments are to be randomly selected, with a minimum of 10 shipments and no more than 20 shipments, or 20 shipments are to be randomly selected among high-risk shipments. After port directors or their designees submit their completed worksheets, CBP OFO calculates the national compliance rate by dividing the number of worksheets that have at least 1 shipment that did not conform to policy by the total number of self-inspection worksheets submitted to OFO. Given that the sample size is generally the same for all port directors regardless of the number of shipments their ports receive, on the basis of our analysis, the sample does not provide CBP with an efficient estimate of compliance at the national level. For example, by allocating additional samples to ports with more arriving shipments, the national compliance estimate could have a reduced sampling error. Through its Self-Inspection Program, CBP has identified a minimal number of noncompliant shipments requiring corrective action. For example, five corrective actions were taken nationwide based on the 2011 self-inspection reporting cycle, and two corrective actions based on the 2012 cycle. CBP's compliance rate was over 93 percent for both cycles, leading CBP to reduce the frequency of the self-inspections for maritime cargo targeting from every year to every other year, according to CBP OFO officials. However, as discussed previously, we determined that 3 of 40 shipments identified as waived were actually examined, but the examinations were not properly recorded and 6 of 40 shipments identified as not examined/not waived were examined or waived, but not properly recorded. Thus, CBP's compliance estimates may be overstated. CBP could improve its ability to estimate national compliance rates for maritime cargo targeting practices through different methods. For example, CBP could increase the number of shipments sampled by ports with the greatest number of arriving shipments or consider a stratified sample with strata defined in terms of size of ports and other factors that might be related to compliance or risk, such as whether shipments appear in its data as not examined/not waived. By enhancing its methodology for selecting shipment samples, CBP could better identify any deficiencies and take appropriate corrective actions. According to a Director in OFO, CBP is considering changing the shipment sample size to a percentage of high- risk shipments, which would result in port directors of larger ports analyzing a greater number of shipments than the 20 shipments per question they currently sample. However, CBP has not finalized its plans for making this change. Further, CBP's method for calculating the compliance rate does not accurately reflect compliance nationwide because it does not calculate the rate based on the number of shipments sampled. Rather, CBP calculates the rate based on the number of worksheets that contained shipments for which policy was not followed regardless of whether it was 1 shipment or all shipments included in the worksheet. According to guidance on design evaluations, the data should be analyzed in a way that allows valid conclusions to be drawn from the evaluation.[Footnote 38] CBP could improve its ability to estimate national compliance with maritime cargo targeting practices by changing its compliance rate calculation to divide the number of shipments (the unit of measure) that did not conform to policy by the total number of shipments sampled, rather than calculating the rate based on the number of worksheets. In addition to the limited nature of the self-inspections, we also found that CBP does not fully assess the reliability of the data it analyzes on the disposition of high-risk shipments for the quarterly reports it produces in support of GPRA. Through our reliability assessment of CBP's data, we identified high-risk shipments that were not high risk and shipments for which the disposition (e.g., waived) was not recorded accurately (as previously discussed). An independent review team contracted by DHS to verify and validate the completeness and reliability of CBP's performance data used for the GPRA measure on high-risk cargo determined that CBP only addressed significant anomalies in its data contained in the quarterly reports and recommended, in May 2014, that CBP develop a formal quality control process for field data.[Footnote 39] OFO concurred with the recommendation "with reservations," adding that it believed reinforcing existing policies and procedures through the local CBP command structure would address the errors. However, in CBP's data we also found errors not attributable to field data.[Footnote 40] Conclusions: In the absence of being able to ensure that 100 percent of U.S.-bound maritime cargo containers are scanned at foreign ports, as required by statute, the Secretary of Homeland Security has instructed DHS, including CBP, to do a better job of meeting the underlying objectives of the 100 percent scanning requirement by, in part, refining aspects of CBP's layered security strategy. Given that examining and waiving, if appropriate, high-risk shipments are critical aspects of CBP's strategy, it is important for CBP to ensure that these practices are carried out consistently and that results of its targeters' actions regarding the disposition of high-risk cargo shipments are recorded accurately. CBP is starting to take actions to correct errors in its data and revise its National Maritime Targeting Policy, but it needs to take further actions to ensure its policies are consistently being followed and to enhance the reliability of its high-risk shipment data. For example, without defining the standard exceptions in policy, CBP is not able to ensure that all high-risk shipments are being appropriately examined or waived. Further, without updating and disseminating policy on how to record such waivers, CBP will not be able to determine whether its targeting units are using articulable reason waivers judiciously, as called for in policy. Moreover, enhancing the methodology used in CBP's Self-Inspection Program will allow it to better identify instances where policy is not being followed and implement corrective actions. Such actions, in turn, could help CBP better ensure that its policies on the disposition of high-risk shipments are being followed. Recommendations for Executive Action: To help ensure compliance with policies on waiving high-risk shipments, we recommend that the Commissioner of CBP direct OFO to take the following two actions: * develop a definition for each of the standard exception waiver categories and include those definitions in policy to ensure that targeting units are consistently applying those definitions when making and documenting waiver decisions in CERTS, and: * update and disseminate policy to ensure that all targeting units are correctly documenting waivers based on articulable reasons in CERTS. To enhance oversight of the disposition of high-risk shipments-- examinations and waivers--we recommend that the Commissioner of CBP direct OFO to take two actions: * develop an enhanced methodology for selecting shipment samples used for self-inspection to increase the likelihood that any potential deficiencies will be identified so that corrective actions can be taken to reduce errors in the future, and: * develop a better national estimate of compliance with maritime cargo targeting policies by calculating the compliance rate based on individual shipments rather than worksheets. Agency Comments and Our Evaluation: We provided a draft of the sensitive version of this report to DHS for its review and comment. DHS provided technical comments, which have been incorporated into this report, as appropriate. DHS also provided written comments, which are reprinted in appendix III. In its comments, DHS concurred with the report's four recommendations and described actions it has under way or planned to address the recommendations by June 30, 2015. DHS concurred with the first recommendation and stated that CBP plans to develop a definition for each of the standard exception waiver categories. DHS concurred with the second recommendation and stated that CBP will provide guidance on issuing waivers based on articulable reasons in its updated National Cargo Targeting Policy. DHS concurred with the third recommendation and stated that CBP has updated its self- inspection worksheet for the 2015 inspection cycle in response to the recommendation. DHS concurred with the fourth recommendation and stated that CBP will develop the ability to generate reports on noncompliant high-risk shipments and require port directors or their designees to review the reports and take corrective actions based on noncompliant shipments. If implemented as planned, these actions should address the intent of the recommendations to improve CBP's disposition of high-risk shipments. In its comments, DHS also referred to a fifth recommendation related to reviewing port codes. Because DHS deemed the details of this recommendation and its response as sensitive security information, they are not included in this public version of the report. If you or your staff have any questions about this report, please contact me at (202) 512-7141 or GroverJ@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV. Sincerely yours, Signed by: Jennifer Grover: Director: Homeland Security and Justice: [End of section] Appendix I: Description of CBP's Layered Security Strategy for Maritime Cargo Shipments: This appendix describes the core programs related to U.S. Customs and Border Protection's (CBP) strategy for ensuring the security of maritime cargo. CBP has developed this strategy to mitigate the risk of weapons of mass destruction, terrorist-related material, or other contraband being smuggled into the United States. CBP's strategy is based on related programs that attempt to focus resources on high-risk shipments while allowing other cargo shipments to proceed without unduly disrupting the flow of commerce into the United States. The strategy includes obtaining advanced cargo information to identify high-risk shipments, using technology to inspect cargo, and partnering with foreign governments and the trade industry. Table 3 provides a brief description of some of the core programs that compose this security strategy. Table 3: Description of U.S. Customs and Border Protection's (CBP) Core Cargo Security Programs: Obtaining advanced information to identify high-risk cargo: Program and year introduced: Automated Targeting System (ATS), 1999; Description: ATS is an enforcement and decision support system that compares traveler, cargo, and conveyance information against intelligence and other enforcement data by incorporating risk-based targeting scenarios and assessments. ATS assigns a risk score to arriving cargo shipments based on shipping information to help CBP identify and prevent potential terrorists and terrorist weapons from entering the United States. Program and year introduced: 24-hour rule, 2002; Description: CBP generally requires vessel carriers to electronically transmit cargo manifests to CBP's Automated Manifest System 24 hours before U.S.-bound cargo is loaded onto a vessel at a foreign port. The information is used by ATS in its calculation of risk scores. The cargo manifest information is submitted by vessel carriers for all arriving cargo shipments. Program and year introduced: Importer Security Filing and Additional Carrier Requirements (also known as the 10+2 rule), 2009; Description: CBP requires importers and vessel carriers to provide data elements for improved identification of containerized shipments that may pose a risk for terrorism. The importer is responsible for supplying CBP with 10 shipping data elements, such as country of origin, 24 hours prior to loading, while the vessel carrier is required to provide 2 data elements, container status messages and stow plans, not required by the 24-hour rule[A]. Domestic scanning technology deployments: Program and year introduced: Nonintrusive inspection (NII) equipment, 2001; Description: CBP uses NII equipment to actively scan both randomly selected containers and those identified by ATS as high risk. NII uses X-rays or gamma rays to scan a container and create images of the container's contents without opening it. According to CBP, as of August 2014, it had deployed 272 large-scale NII systems to U.S. seaports to scan containers. Program and year introduced: Radiation portal monitors, 2002; Description: CBP's program to scan 100 percent of containers arriving in the United States with radiation detection equipment prior to leaving a domestic port. As of August 2014, the Department of Homeland Security (DHS) had deployed 388 radiation portal monitors at U.S. seaports, through which over 99 percent of all containerized cargo arriving by sea is scanned. Partnerships with foreign governments and the trade industry: Program and year introduced: Container Security Initiative, 2002; Description: CBP places staff at participating foreign ports to work with host country customs officials to target and examine high-risk container cargo for weapons of mass destruction before they are shipped to the United States. CBP officials identify the containers that may pose a risk for terrorism and request that their foreign counterparts examine the contents of the containers. As of July 2014, there were 58 Container Security Initiative ports located in 32 countries. Program and year introduced: Customs-Trade Partnership Against Terrorism, 2001; Description: CBP develops voluntary partnerships with members of the international trade community composed of importers; manufacturers; customs brokers; forwarders; air, sea, and land carriers; and contract logistics providers. Private companies that implement specific security measures and best practices receive facilitated processing, such as a reduced likelihood of security-based examinations of their cargo. Source: GAO summary of information provided by the Department of Homeland Security. GAO-15-294. [A] Container status messages report terminal container movements, such as loading and discharging the vessel, and report the change in the status of containers, such as if they are empty or full. The stow plan contains the position of each cargo container on a vessel. [End of table] [End of section] Appendix II: Objectives, Scope, and Methodology: This report addresses U.S. Customs and Border Protection's (CBP) disposition of high-risk maritime cargo shipments. More specifically, our objectives were to examine (1) the number of maritime shipments arriving in the United States from fiscal years 2009 through 2013 that CBP determined to be high risk and the extent to which CBP has accurate data on the disposition of each of those high-risk shipments, (2) the extent to which CBP is consistently applying standards and documenting reasons for waiving examinations of high-risk shipments, and (3) the extent to which CBP ensures that its policies on the disposition of high-risk shipments are being followed. To address all of these objectives, we reviewed CBP policies regarding the targeting and waiving of high-risk shipments, analyzed CBP data, and spoke with key CBP officials at both headquarters and selected targeting units. To determine the number of maritime shipments arriving in the United States that CBP determined to be high risk, we obtained data from CBP on the number of shipments and high-risk shipments that arrived in the United States by seaport during fiscal years 2009 through 2013--the 5 most recent fiscal years for which full- year data were available at the time of our review.[Footnote 41] To determine the extent to which CBP has accurate data on the disposition of each high-risk shipment, we analyzed CBP's data to determine the number of high-risk shipments examined, and waived--the disposition options we identified in CBP's National Maritime Targeting Policy--as well as those shipments not examined/not waived.[Footnote 42] We excluded foreign cargo remaining on board (FROB) shipments--cargo not discharged in the United States--from the scope of our review to focus on shipments unloaded at United States seaports. To assess the reliability of the data, we reviewed the data for obvious errors and discussed our observations with CBP officials who compiled the data. We also discussed with CBP officials how the data are entered and maintained and interviewed officials who enter the data in CBP's Automated Targeting System (ATS). We also selected a nonprobability sample of shipments from CBP's fiscal year 2013 data to determine the accuracy of the disposition data. Specifically, from the list of waived shipments in CBP's fiscal year 2013 shipment data, we selected a nonprobability sample of 40 waived shipments that represented a variety of waiver reasons recorded in ATS. We also selected a second nonprobability sample of 40 not examined/not waived shipments from the same set of data, to include shipments CBP identified as deleted bills and shipments with high-risk scores only after entry was filed. These shipment samples were associated with the four Advance Targeting Units (targeting units) we visited (see below for site visit selection criteria). We discussed the accuracy of both sample sets with targeting unit officials. We determined that CBP's data likely overstate the number of high-risk shipments, including those not examined/not waived, but the data are sufficiently reliable to illustrate the overall disposition of all high-risk shipments by category--examined, waived, and not examined/not waived--since a small percentage of shipments were waived and not examined/not waived relative to the number examined. In addition to analyzing CBP's disposition data, we collected and analyzed gate out data for fiscal years 2009 through 2013 from CBP's Fines, Penalties, and Forfeitures Division in order to determine the frequency of gate out occurrences relative to the overall number of not waived/not examined shipments. To assess the reliability of the data, we reviewed gate out case files and assessed whether the information contained in the files matched with fiscal years 2009 through 2013 gate out data we received from CBP. We also discussed the process for collecting and recording gate out data with CBP officials from the Fines, Penalties, and Forfeitures Division. We determined that the gate out data were sufficiently reliable for reporting the number of gate out occurrences. To determine the extent to which CBP is consistently applying standards and documenting reasons for waiving examinations of high- risk shipments, we analyzed CBP's data on high-risk shipment waivers recorded during fiscal year 2013, reviewed CBP policies and guidance on waiving examinations of high-risk shipments, and observed and discussed waiver practices at the targeting units we visited.[Footnote 43] We conducted site visits to four targeting units that appeared to issue the greatest percentage of waivers relative to the total high- risk maritime cargo shipments that arrived in their respective seaports in fiscal year 2013.[Footnote 44] These targeting units also represent a variety of geographical locations within the United States, as they are situated on the Gulf, East, and West Coasts. Additionally, they encompass seaports of varying sizes based on arriving shipments, ranging from approximately 146,000 shipments to 4.4 million shipments. We reviewed selected samples of waived shipments in CBP's data with officials at each targeting unit in order to gain an understanding of how targeters selected and documented waiver reasons, including standard exceptions for each of the shipments. Although the results from our visits to these four targeting units are not generalizable to all targeting units across the United States, the visits allowed us to understand whether waiver documentation practices are consistent across the targeting units we visited, and how such practices affect the reliability of CBP's disposition data. We asked CBP officials at each targeting unit we visited to define certain standard exception waivers listed in CBP's National Maritime Targeting Policy in order to determine whether targeting units were consistently using uniform criteria when applying standard exception waivers to high-risk shipments. Additionally, we asked targeters to comment on their awareness of any existing CBP policy or central guidance regarding the definition of the various standard exceptions. We compared CBP's practices relative to standard exception waivers against standards in Standards for Internal Control in the Federal Government, which state that control activities should be effective and efficient in accomplishing the agency's objectives.[Footnote 45] We spoke with CBP officials at each of the four targeting units we visited about their processes for reviewing and approving articulable reason waivers for high-risk shipments and assessed their compliance with CBP policy on approving waivers documented in the National Maritime Targeting Policy. At each of the four targeting units, we also met with CBP targeters to discuss and observe their practices for recording examination waivers based on articulable reasons in ATS. We then compared those practices with CBP guidance on how to record articulable reasons waivers, as prescribed in a February 2007 CBP memorandum.[Footnote 46] In order to determine the extent to which CBP ensures that its policies on the disposition of high-risk shipments are being followed, we met with CBP officials at both headquarters and targeting units responsible for the management of high-risk maritime shipments and discussed their oversight of CBP's policies. We met with officials from CBP's Office of Field Operations (OFO) to discuss their design and implementation of the Self-Inspection Program for maritime cargo targeting, including the shipment sample size used to assess compliance and method for calculating compliance. We reviewed CBP reports summarizing the results of self-inspections, including the compliance rate, as well as individual reports submitted by the four targeting units we visited. We then compared CBP's methodology for conducting self-inspections with best practices outlined in guidance on design evaluations.[Footnote 47] Further, we reviewed quarterly reports produced by CBP outlining the disposition of high-risk shipments and discussed with officials from OFO how these reports are used to determine whether CBP is meeting a Government Performance and Results Act (GPRA) performance goal. We compared CBP's efforts to assess the accuracy of its data used in support of GPRA with standards in Standards for Internal Control in the Federal Government.[Footnote 48] We also met with officials from CBP's Fines, Penalties, and Forfeitures Division to discuss CBP's procedures for identifying and reporting gate outs. We conducted this performance audit from January 2014 through January 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix III: Comments from the Department of Homeland Security: U.S. Department of Homeland Security: Washington, DC 20528: January 9, 2015: Jennifer Grover: Director, Homeland Security and Justice: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Re: Draft Report GAO-15-294, "Supply Chain Security: CBP Needs to Enhance Its Guidance and Oversight of High-Risk Maritime Cargo Shipments" Dear Ms. Grover: Thank you for the opportunity to review and comment on this draft report. The U.S. Department of Homeland Security (DHS) appreciates the U.S. Government Accountability Office's (GAO's) work in planning and conducting its review and issuing this report. The Department appreciates GAO's acknowledgment of U.S. Customs and Border Protection's (CBP's) layered security strategy which uses a risk-based approach to focus limited resources on the targeting and examination of cargo. DHS also notes GAO's recognition of CBP's efforts to enhance the reliability of it high-risk shipment data. CBP is committed to enhancing supply chain security using a positive, proactive approach to global trade, and working with others to build a more secure and more efficient worldwide trade environment. The draft report contained four recommendations with which the Department concurs. Specifically, GAO recommended that the Commissioner of CBP direct OFO [Office of Field Operations] to: Recommendation 1: Develop a definition for each of the standard exception waiver categories and include those definitions in policy to ensure that targeting units are consistently applying those definitions when making and documenting waiver decisions in CERTS (Cargo Enforcement Reporting and Tracking System). Response: Concur. CBP's OFO is drafting an updated, comprehensive National Cargo Targeting Policy. The revisions to the policy will include definitions for each of the standard exception waiver categories. Estimated Completion Date (ECD): April 30, 2015. Recommendation 2: Update and disseminate policy to ensure that all targeting units are correctly documenting waivers based on articulable reasons in CERTS. Response: Concur. OFO is drafting an updated, comprehensive National Cargo Targeting Policy to ensure that all targeting units are correctly documenting waivers based on articulable reasons in CERTS. ECD: Apri1 30, 2015. Recommendation 3: Develop an enhanced methodology for selecting shipment samples used for self-inspection to increase the likelihood that any potential deficiencies will be identified so that corrective actions can be taken to reduce errors in the future. Response: Concur. OFO has already updated the Commercial Enforcement Targeting Sea Cargo Self-Inspection Program (SIP) worksheet and believes it has satisfied the intent of this recommendation. CBP is working with GAO to close this recommendation, as appropriate. Recommendation 4: Develop a better national estimate of compliance with maritime cargo targeting policies by calculating the compliance rate based on individual shipments rather than worksheets. Response: Concur. OFO, in conjunction with CBP's Office of Information and Technology, will develop system enhancements to CERTS to enable CERTS to generate reports to identify high-risk shipments which are not in compliance with the National Maritime Targeting Policy. OFO is currently revising its policy to reflect that the Port Director or his/her designee will be accountable for intermittently reviewing the CERTS reports to identify non-compliance and will be responsible for taking corrective actions in order to conform with the National Maritime Targeting Policy. ECD: June 30, 2015. GAO also made a recommendation for CBP to conduct a comprehensive review of existing port code tables. The details of GAO's recommendation and DHS's response are Sensitive Security Information and not included in the public version of the report. Again, thank you for the opportunity to review and comment on this draft report. Technical comments were previously provided under separate cover. Please feel free to contact me if you have any questions. We look forward to working with you in the future. Sincerely, Signed by: Jim H. Crumpacker, CIA, CFE: Director: Departmental GAO-OIG Liaison Office: [End of section] Appendix IV: GAO Contact and Staff Acknowledgments: GAO Contact: Jennifer Grover, Director, (202) 512-7141 or GroverJ@gao.gov. Staff Acknowledgments: In addition to the contact named above, Stephen Caldwell (Director), Christopher Conrad (Assistant Director), Lisa Canini, and Daniel McKenna made key contributions to this report. Also contributing to this report were Frances Cook, Eric Hauswirth, Tracey King, Stanley Kostyla, Thomas Lombardi, Ruben Montes de Oca, Jessica Orr, and Mark Ramage. [End of section] Related GAO Products: Maritime Critical Infrastructure Protection: DHS Needs to Better Address Port Cybersecurity. [hyperlink, http://www.gao.gov/products/GAO-14-459]. Washington, D.C.: June 5, 2014. Maritime Security: Progress and Challenges with Selected Port Security Programs. [hyperlink, http://www.gao.gov/products/GAO-14-636T]. Washington, D.C.: June 4, 2014. Maritime Security: Progress and Challenges in Key DHS Programs to Secure the Maritime Borders. [hyperlink, http://www.gao.gov/products/GAO-14-196T]. Washington, D.C.: November 19, 2013. Supply Chain Security: DHS Could Improve Cargo Security by Periodically Assessing Risks from Foreign Ports. [hyperlink, http://www.gao.gov/products/GAO-13-764]. Washington, D.C.: September 16, 2013. Supply Chain Security: CBP Needs to Conduct Regular Assessments of Its Cargo Targeting System. [hyperlink, http://www.gao.gov/products/GAO-13-9]. Washington, D.C.: October 25, 2012. Maritime Security: Progress and Challenges 10 Years after the Maritime Transportation Security Act. [hyperlink, http://www.gao.gov/products/GAO-12-1009T]. Washington, D.C.: September 11, 2012. Supply Chain Security: Container Security Programs Have Matured, but Uncertainty Persists over the Future of 100 Percent Scanning. [hyperlink, http://www.gao.gov/products/GAO-12-422T]. Washington, D.C.: February 7, 2012. Supply Chain Security: CBP Has Made Progress in Assisting the Trade Industry in Implementing the New Importer Security Filing Requirements, but Some Challenges Remain. [hyperlink, http://www.gao.gov/products/GAO-10-841]. Washington, D.C.: September 10, 2010. Supply Chain Security: Feasibility and Cost-Benefit Analysis Would Assist DHS and Congress in Assessing and Implementing the Requirement to Scan 100 Percent of U.S.-Bound Containers. [hyperlink, http://www.gao.gov/products/GAO-10-12]. Washington, D.C.: October 30, 2009. Supply Chain Security: Challenges to Scanning 100 Percent of U.S.- Bound Cargo Containers. [hyperlink, http://www.gao.gov/products/GAO-08-533T]. Washington, D.C.: June 12, 2008. Supply Chain Security: U.S. Customs and Border Protection Has Enhanced Its Partnership with Import Trade Sectors, but Challenges Remain in Verifying Security Practices. [hyperlink, http://www.gao.gov/products/GAO-08-240]. Washington, D.C.: April 25, 2008. Supply Chain Security: Examinations of High-Risk Cargo at Foreign Seaports Have Increased, but Improved Data Collection and Performance Measures Are Needed. [hyperlink, http://www.gao.gov/products/GAO-08-187]. Washington, D.C.: January 25, 2008. [End of section] Footnotes: [1] U.S. Department of Transportation, Research and Innovative Technology Administration, Bureau of Transportation Statistics, America's Container Ports: Linking Markets at Home and Abroad (Washington, D.C.: January 2011). [2] The White House, National Strategy for Global Supply Chain Security (Washington, D.C.: January 2012). [3] In addition to its priority mission of keeping terrorists and their weapons out of the United States, CBP is also responsible for securing the border, facilitating international trade and travel, collecting duties, and enforcing numerous U.S. laws and regulations pertaining to immigration and illicit drugs, among other things. [4] Pub. L. No. 109-347, 120 Stat. 1884 (2006). [5] Radiation detection equipment detects radiation being emitted from a container. Through an NII scan, CBP can identify anomalies in a container's image that could, among other things, indicate the presence of material to shield weapons of mass destruction. NII uses X- rays or gamma rays to scan a container and create images of the container's contents without having to open it. [6] See 49 C.F.R. pt. 1520. [7] CBP officials are assigned to targeting units located at or near select domestic ports, and their efforts at these targeting units are focused on shipments destined for ports within their respective regions. [8] Because of issues relating to the accuracy of CBP's data that we uncovered during our site visits and therefore subsequent to developing our site visit methodology, the targeting units we visited may not actually represent the four targeting units that issued the greatest percentage of waivers relative to their total high-risk maritime cargo shipments in fiscal year 2013. [9] U.S. Customs and Border Protection, National Maritime Targeting Policy, CBP Directive No. 3290-007B (Washington, D.C.: Dec. 28, 2007). [10] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: Nov. 1, 1999). [11] U.S. Customs and Border Protection, Authorization to Waive ATS Threshold Exams in All Transportation Modes (Washington, D.C.: Feb. 26, 2007). [12] GAO, Designing Evaluations: 2012 Revision, [hyperlink, http://www.gao.gov/products/GAO-12-208G] (Washington, D.C.: January 2012). This report addresses the reasons to systematically plan evaluations, which include (1) enhancing their quality, credibility, and usefulness, and (2) using time and resources effectively. It is based on GAO studies and policy documents and program evaluation literature. [13] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [14] 19 C.F.R. § 4.7(b). Cargo manifests are prepared by the ocean carrier and are composed of bills of lading for each shipment loaded onto a vessel to describe the contents of the shipments. Bills of lading are documents issued by a carrier describing the goods, the details of the intended voyage, and the conditions of transportation. [15] Importer Security Filing and Additional Carrier Requirements, 73 Fed. Reg. 71,730 (Nov. 25, 2008) (codified at 19 C.F.R. pts. 4, 12, 18, 101, 103, 113, 122, 123, 141, 143, 149, 178, & 192). [16] Container status messages report terminal container movements, such as loading and discharging the vessel, and report the change in the status of containers, such as if they are empty or full. Container status messages also report conveyance movements, such as vessel arrivals and departures. A vessel stow plan includes information such as the vessel operator, voyage number, the stow position of each container, hazardous material code (if applicable), and the port of discharge. [17] For containerized shipments, a single shipment may consist of one or more containers, or multiple shipments may be consolidated into a single container for transport. The 12 million shipments that arrived in the United States in fiscal year 2013 included 12.6 million containers. [18] Targeters can also use their discretion and place low-and medium- risk shipments on hold for examination. [19] In locations where no enforcement team exists, port management assigns examinations to CBP officials within the port. [20] Although targeters overseas at CSI ports can recommend examination waivers, for the purposes of this report, we are reviewing actions taken domestically since only domestic targeters can request waivers. [21] Examples of standard exceptions are not included because CBP considers them to be sensitive security information. [22] According to a senior CBP official, if the entry documents are not available until well after a shipment's arrival (e.g., after a shipment moves in bond to its final port of entry), and the score then equals or exceeds the high-risk threshold, the shipment cannot be released from the warehouse (or bonded facility) until CBP has properly mitigated the risk and changed the entry status to "released." [23] Pub. L. No. 110-53, § 1701(a), 121 Stat. 266, 489-90 (amending 6 U.S.C. § 982(b)). [24] The 9/11 Commission Act scanning provision includes possible extensions for containers loaded at a port or ports for which DHS certifies that at least two out of a list of specific conditions exist. Among others, these conditions include the following: (1) adequate scanning equipment is not available or cannot be integrated with existing systems, (2) a port does not have the physical characteristics to install the equipment, or (3) use of the equipment will significantly affect trade capacity and the flow of cargo. See 6 U.S.C. § 982(b)(4). [25] GAO, Supply Chain Security: Container Security Programs Have Matured, but Uncertainty Persists over the Future of 100 Percent Scanning, [hyperlink, http://www.gao.gov/products/GAO-12-422T] (Washington, D.C.: Feb. 7, 2012), and Supply Chain Security: Feasibility and Cost-Benefit Analysis Would Assist DHS and Congress in Assessing and Implementing the Requirement to Scan 100 Percent of U.S.- Bound Containers, [hyperlink, http://www.gao.gov/products/GAO-10-12] (Washington, D.C.: Oct. 30, 2009). [26] GAO, Supply Chain Security: CBP Needs to Conduct Regular Assessments of Its Cargo Targeting System, [hyperlink, http://www.gao.gov/products/GAO-13-9] (Washington, D.C.: Oct. 25, 2012), and Supply Chain Security: CBP Has Made Progress in Assisting the Trade Industry in Implementing the New Importer Security Filing Requirements, but Some Challenges Remain, [hyperlink, http://www.gao.gov/products/GAO-10-841] (Washington, D.C.: Sept. 10, 2010). [27] GAO, Supply Chain Security: DHS Could Improve Cargo Security by Periodically Assessing Risks from Foreign Ports, [hyperlink, http://www.gao.gov/products/GAO-13-764] (Washington, D.C.: Sept. 16, 2013); Supply Chain Security: Examinations of High-Risk Cargo at Foreign Seaports Have Increased, but Improved Data Collection and Performance Measures Are Needed, [hyperlink, http://www.gao.gov/products/GAO-08-187] (Washington, D.C.: Jan. 25, 2008); and Container Security: A Flexible Staffing Model and Minimum Equipment Requirements Would Improve Overseas Targeting and Inspection Efforts, [hyperlink, http://www.gao.gov/products/GAO-05-557] (Washington, D.C.: Apr. 26, 2005). [28] GAO, Supply Chain Security: U.S. Customs and Border Protection Has Enhanced Its Partnership with Import Trade Sectors, but Challenges Remain in Verifying Security Practices, [hyperlink, http://www.gao.gov/products/GAO-08-240] (Washington, D.C.: Apr. 25, 2008), and Cargo Security: Partnership Program Grants Importers Reduced Scrutiny with Limited Assurance of Improved Security, [hyperlink, http://www.gao.gov/products/GAO-05-404] (Washington, D.C.: Mar. 11, 2005). [29] According to CBP data, 99 percent of all high-risk shipments are reviewed by a targeter, meaning the targeter reviewed the shipments' records in ATS to identify potential threats before the shipments were loaded onto a vessel at a foreign port. [30] We determined that CBP's data on the overall disposition of high- risk shipments--specifically those examined, waived, and not examined/ not waived--may not reflect the actual disposition of the high-risk shipments since some discrepancies were found among waived and not examined/not waived high-risk shipments that represent a small percentage of high-risk shipments. [31] Further details on the shipments we selected can be found in app. II. [32] Examples of standard exception definitions are not included because CBP considers them to be sensitive security information. [33] Examples of the impact of inconsistent definitions are not included because CBP considers them to be sensitive security information. [34] GAO/AIMD-00-21.3.1. [35] Although the policy to review and approve waiver requests does not apply to standard exception waivers, we found that standard exceptions were subject to the same review and approval process as articulable reason waivers at all four targeting units we visited. [36] GAO/AIMD-00-21.3.1. [37] The Government Performance and Results Act of 1993 provides the statutory framework for performance management in the federal government. Congress updated GPRA with the GPRA Modernization Act of 2010, which enhances the requirements for agencies to consult with Congress when establishing or adjusting government-wide and agency goals. [38] [hyperlink, http://www.gao.gov/products/GAO-12-208G]. [39] Energetics, Independent Verification and Validation of Performance Measure Data: FY 2014 Review and Report of Findings (May 2014). [40] Details on the data errors and their impact on operations are not included because CBP considers them to be sensitive security information. [41] For the purpose of our review, a high-risk shipment is a shipment CBP's Automated Targeting System (ATS) identified as high risk based on the maritime national security weight set. [42] U.S. Customs and Border Protection, National Maritime Targeting Policy, CBP Directive No. 3290-007B (Washington, D.C.: Dec. 28, 2007). [43] We reviewed both types of waivers identified in the National Maritime Targeting Policy--standard exceptions and articulable reasons. Articulable waivers are requested based on discretion, whereas standard exceptions are specified in policy and not intended to be subject to interpretation. [44] Because of issues relating to the accuracy of CBP's data that we uncovered during our site visits and therefore subsequent to developing our site visit methodology, it is possible that the targeting units we visited may not actually represent the four targeting units that issued the greatest percentage of waivers relative to their total high-risk maritime cargo shipments in fiscal year 2013. [45] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [46] U.S. Customs and Border Protection, Authorization to Waive ATS Threshold Exams in All Transportation Modes (Washington, D.C.: Feb. 26, 2007). [47] [hyperlink, http://www.gao.gov/products/GAO-12-208G]. [48] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]