According to the Transportation Security Administration, the threat of explosives in air cargo coming into the United States is significant. TSA requires air carriers to x-ray or screen the cargo by other means before it enters the country.
In 2020, TSA began field testing imaging technology (similar to CT medical imaging) to screen small parcels for explosives. We found TSA didn't follow key design and evaluation practices during testing. As a result, TSA can't ensure that the imaging technology meets detection requirements. We recommended TSA perform additional testing or analysis on the technology before TSA approves it for use.
What GAO Found
The Department of Homeland Security's (DHS) Transportation Security Administration (TSA) and U.S. Customs and Border Protection (CBP) address U.S.-bound air cargo security through separate programs and have taken steps to measure their effectiveness. For example, TSA conducts an inspection program to help ensure that air carriers comply with specific cargo-related security requirements, such as requirements related to cargo acceptance, control and custody, and screening procedures.
Air Cargo Pallet and Air Cargo Loaded onto an Aircraft
From January 2020 through April 2021, TSA conducted a field assessment on the use of a computed tomography (CT)-based explosives detection system to screen air cargo as part of its ongoing process to qualify the system for use by air carriers. This type of system produces images of parcels that are examined by computer for signs of explosives. However, TSA's assessment did not fully meet three of five key design and evaluation practices. While the assessment identified goals and established metrics, TSA did not incorporate other key practices, such as collecting all necessary data about the system's ability to detect threats (probability of detection) in the field, consistent with TSA's standards. Since TSA officials cannot use live explosives in the field to measure the probability of detection, they relied on image quality testing, using a manufacturer's test kit to compare system performance in the field with earlier tests performed in a laboratory with live explosives.
However, TSA did not validate that the test kit was an acceptable alternative test method for determining the CT system's probability of detection in the field. TSA did not (1) independently validate that the test kit captures all ways system performance could degrade or (2) collect any of the underlying quantitative data from the test kit. TSA officials told GAO they did not validate the test kit because its performance was certified during laboratory testing at DHS's Transportation Security Laboratory; however, officials from the Transportation Security Laboratory told GAO they do not certify the performance of test kits. Without a suitable alternative testing approach to determine the probability of detection, TSA will not have all relevant data to assess whether the CT system meets TSA's detection standard requirements in the field and should be qualified for use by air carriers.
Why GAO Did This Study
According to DHS—which is responsible for ensuring the security of air cargo transported to the United States—the threat from explosives in air cargo remains significant.
The TSA Modernization Act includes a provision for GAO to review DHS's processes for securing U.S.-bound air cargo and efforts to use CT technology for air cargo screening. This report addresses, among other things, how DHS secures inbound air cargo, and the extent to which TSA's field assessment of a CT screening system included key practices for design and evaluation.
GAO reviewed TSA and CBP air cargo security procedures and documents and analyzed a random sample of air cargo shipment data from calendar year 2019. GAO also interviewed TSA and CBP headquarters and National Targeting Center officials, and interviewed TSA field and air carrier officials regarding operations with two foreign airports, selected based on TSA risk data and the amount of air cargo transported from these airports to the United States. This is a public version of a sensitive report that GAO issued in May 2021. Information that DHS deemed sensitive was omitted.
In the May 2021 report, GAO made four recommendations, including that TSA ensure it collects all necessary data for field assessments of explosives detection systems for air cargo screening prior to qualifying the systems for use by air carriers. DHS agreed with all four recommendations and said TSA and CBP are taking or planning actions to address them.
Recommendations for Executive Action
|Department of Homeland Security||The Secretary of DHS should ensure that the Administrator of TSA and the Commissioner of CBP establish a documented process to ensure that relevant officials from both agencies are aware of and have access to applicable data to inform their inbound air cargo risk assessment efforts. (Recommendation 1)|
|Department of Homeland Security||The Administrator of TSA, prior to designating the explosives detection system for air cargo screening currently under evaluation as "qualified" on the air cargo screening technology list, should, to the extent practicable, verify through additional data collection or analysis that the system's probability of detection in the field matches the performance measured in laboratory testing. (Recommendation 2)|
|Department of Homeland Security||The Administrator of TSA should ensure that necessary data are collected during field assessments to independently verify that the probability of detection of explosives detection systems for air cargo screening in the field matches the performance measured in laboratory testing, prior to designating systems as "qualified" on the air cargo screening technology list. TSA could provide this verification either through live explosives testing or, when operational considerations limit TSA's ability to use live threat materials, TSA should use an independently validated, fully documented alternative testing strategy. (Recommendation 3)|
|Department of Homeland Security||The Administrator of TSA should ensure statistical techniques are used to analyze data from TSA field assessments, including data from the current field assessment, of explosives detection systems for air cargo screening, prior to designating systems as "qualified" on the air cargo screening technology list. This statistical analysis should include the following elements: (1) calculating error values for each quantitative measurement, (2) identifying all necessary performance thresholds, and (3) comparing the measured values and errors against each threshold to determine the statistical confidence of the results. (Recommendation 4)|