Fast Facts

TSA tests its screening technologies before installing them at airports to ensure that they are detecting certain dangerous items as intended.

But screening technologies can become less effective over time, and we found that TSA does not continue to fully test them once they are installed. Some airport equipment that detects trace explosives or tests bottled liquids wasn’t performing as intended when the Department of Homeland Security evaluated it in 2015 and 2016.

We recommended that TSA ensure that its screening technologies continue to meet requirements after they are installed at airports.

Airport screening equipment

Airport screening equipment

Skip to Highlights
Highlights

What GAO Found

The Department of Homeland Security's (DHS) Transportation Security Administration (TSA) operationalizes, or puts into effect, detection standards for its screening technologies by acquiring and deploying new technologies, which can take years. Detection standards specify the prohibited items (e.g., guns, explosives) that technologies are to detect, the minimum rate of detection, and the maximum rate at which technologies incorrectly flag an item. TSA operationalizes standards by adapting them as detection requirements, working with manufacturers to develop and test new technologies (software or hardware), and acquiring and deploying technologies to airports. For the standards GAO reviewed, this process took 2 to 7 years, based on manufacturers' technical abilities and other factors.

TSA's deployment decisions are generally based on logistical factors and it is unclear how risk is considered when determining where and in what order technologies are deployed because TSA did not document its decisions. TSA considers risks across the civil aviation system when making acquisition decisions. However, TSA did not document the extent risk played a role in deployment, and could not fully explain how risk analyses contributed to those decisions. Moving forward, increased transparency about TSA's decisions would better ensure that deployment of technologies matches potential risks.

Technology performance can degrade over time; however, TSA does not ensure that technologies continue to meet detection requirements after deployment to airports. TSA certifies technologies to ensure they meet requirements before deployment, and screeners are to regularly calibrate deployed technologies to demonstrate they are minimally operational. However, neither process ensures that technologies continue to meet requirements after deployment. In 2015 and 2016, DHS tested a sample of deployed explosives trace detection and bottled liquid scanner units and found that some no longer met detection requirements. Developing and implementing a process to ensure technologies continue to meet detection requirements after deployment would help ensure that TSA screening procedures are effective and enable TSA to take corrective action if needed.

Transportation Security Administration's (TSA) Process for Acquiring Screening Technologies to Meet Detection Standards

Transportation Security Administration's (TSA) Process for Acquiring Screening Technologies to Meet Detection Standards

Why GAO Did This Study

TSA is responsible for overseeing security operations at roughly 440 TSA-regulated airports as part of its mission to protect the nation's civil aviation system. TSA uses technologies to screen passengers and their bags for prohibited items.

The TSA Modernization Act includes a provision for GAO to review TSA's deployment of screening technologies, and GAO was asked to review the detection standards of these screening technologies. This report addresses, among other things, (1) how TSA operationalizes detection standards, (2) the extent to which TSA considered risk when making deployment decisions, and (3) the extent to which TSA ensures technologies continue to meet detection requirements after deployment.

GAO reviewed DHS and TSA procedures and documents, including detection standards; visited DHS and TSA testing facilities; observed the use of screening technologies at seven airports, selected for varying geographic locations and other factors; and interviewed DHS and TSA headquarters and field officials.

Skip to Recommendations

Recommendations

GAO is making five recommendations, including that TSA document analysis of risk in deploying technologies, and implement a process to ensure technologies continue to meet detection requirements after deployment. DHS agreed with all five recommendations and said TSA either has taken or will take actions to address them.

Recommendations for Executive Action

Agency Affected Recommendation Status
Transportation Security Administration 1. The TSA Administrator should update TSA guidance for developing and approving screening technology explosives detection standards to reflect designated procedures, the roles and responsibilities of stakeholders, and changes in the agency's organizational structure. (Recommendation 1)
Closed - Implemented
In December 2019, we reported on the Transportation Security Administration's (TSA) processes for developing detection standards--which identify the characteristics of prohibited items, such as explosives, that passenger and checked baggage screening technologies are to detect--among other issues. During the course of our review, we found that TSA had not updated its 2015 guidance for developing new detection standards to reflect key changes in its procedures. Specifically, we found that, as of August 2019, the guidance did not accurately reflect (1) designated procedures for developing detection standards, (2) the roles and responsibilities of key stakeholders, and (3) TSA's organizational structure. For example, the 2015 guidance calls for an annual assessment of emerging threats, which a senior TSA official told us TSA no longer conducts because relevant emerging threats are now occurring more frequently and intelligence information is processed on an ongoing basis. Consequently, we recommended that TSA update the 2015 guidance for developing and approving screening technology explosives detection standards to reflect designated procedures, the roles and responsibilities of stakeholders, and changes in the agency's organizational structure. In November 2019, TSA provided the Requirements Engineering Integrated Process Manual (RE IPM), which presents designated procedures for developing detection standards and specifies the roles and responsibilities of key stakeholders, with references to offices in TSA's organizational structure. Updating its guidance for developing detection standards should provide TSA with better assurance that detection standards are developed in accordance with established policies and practices. As a result, this recommendation is closed as implemented.
Transportation Security Administration 2. The TSA Administrator should require and ensure that TSA officials document key decisions, including testing and analysis decisions, used to support the development and consideration of new screening technology explosives detection standards. (Recommendation 2)
Open
In December 2019, we reported on the Transportation Security Administration's (TSA) development of detection standards for passenger and checked baggage screening technologies. During the course of our review of TSA's steps to develop detection standards from fiscal years 2014 through 2018, we found that TSA and DHS Science and Technology Directorate (S&T) did not document all key decisions--those that could potentially affect outcomes--regarding the testing and analyses (characterization) of explosive threat materials and the development of explosives detection standards. Specifically, we found that in five of the seven sets of testing and analyses of explosive materials--referred to as material threat assessments--we reviewed, TSA and S&T did not consistently document key steps in the testing and analysis of materials, such as how selected samples were prepared for testing. For example, one S&T material threat assessment we reviewed did not document the method used to synthesize (chemically produce) material samples used for testing. Consequently, we recommended that TSA should require and ensure that TSA officials document key decisions, including testing and analysis decisions, used to support the development and consideration of new screening technology explosives detection standards. In November 2019, TSA provided the Requirements Engineering Integrated Process Manual (RE IPM), which it stated provides the process for documenting key decisions, including testing and analysis decisions, used in the development of new screening technology explosives detection standards. We are reviewing the RE IPM and the extent to which it includes the process for documenting key decisions in the development of new screening technology explosives detection standards and ensures those decisions are documented.
Transportation Security Administration 3. The TSA Administrator should require and ensure that TSA officials document their assessments of risk and the rationale—including the assumptions, methodology, and uncertainty considered—behind decisions to deploy screening technologies. (Recommendation 3)
Open
In December 2019, we reported that TSA officials did not have documentation or analysis to support their assumption that every airport is a possible entry point into the aviation system for a terrorist, and that they do not consider there to be a significant difference in vulnerability among airports when deploying screening technologies, with the exception of a handful of the highest-risk airports. We found that TSA's process for how it would change its deployment plans to specific airports based on risk lacks transparency. For example, TSA officials said that as part of the acquisition process for screening technologies, they have ongoing discussions with stakeholders about deployment strategies, including security and intelligence officials who would inform them of any relevant risk information. Officials said these discussions are generally informal and not documented; officials could not provide an example of when risk information had directly influenced technology deployment in the recent past. Consequently, we recommended that TSA should require and ensure that agency officials document their assessments of risk and the rationale behind decisions to deploy screening technologies. In May 2020, TSA said it had instituted an improved process for documenting elements contributing to deployment decisions across stakeholders--including assumptions, risk, applicability, and logistical factors--and that these decisions are documented in the associated capability's deployment plan, an example of which--the August 2018 Computed Tomography (CT) deployment plan--TSA provided to us in July 2019. TSA said it will continue to use this approach in deploying all future technologies. We agree that the CT deployment plan is an example of how TSA might document its rationale for deploying a screening technology across airports, but the plan was developed prior to TSA's updated process and was the only deployment plan developed to date. To fully address this recommendation, TSA should provide documentation for its updated process and an example of a screening technology deployment plan developed under this revised process.
Transportation Security Administration 4. The TSA Administrator should develop a process to ensure that screening technologies continue to meet detection requirements after deployment to commercial airports. (Recommendation 4)
Open
In December 2019, we reported that TSA practices do not ensure that screening technologies continue to meet detection requirements after they have been deployed to airports. TSA officials stated that the agency uses certification to confirm that technologies meet detection requirements before they are deployed to airports, and calibration to confirm that technologies are at least minimally operational while in use at airports. We found that while certification and calibration serve important purposes in the acquisition and operation of screening technologies, they have not ensured that TSA screening technologies continue to meet detection requirements after they have been deployed. Consequently, we recommended that TSA develop a process to ensure that screening technologies continue to meet detection requirements after deployment to commercial airports. In May 2020, TSA stated it had completed and approved agency policy on the development of Post Implementation Reviews (PIR) for all screening technologies; these PIRs are to explain how TSA will assess screening technology performance following deployment, including detection over time. Because TSA cannot use live explosives in sufficient quantities to test the detection capability of screening technologies, and simulants have limitations, TSA said it intends to typically measure performance of system components within the detection chain instead of directly measuring detecting requirements. Each technology system would require its own PIR--or roadmap for reviewing component performance of the detection chain--because each one has unique logistics data and detection chain. In May 2020, TSA provided the Post Implementation and Periodic Review Policy (APM-20-031). We are reviewing this policy and the extent to which it identifies a process to ensure that screening technologies continue to meet detection requirements after deployment to airports.
Transportation Security Administration 5. The TSA Administrator should implement the process it develops to ensure that screening technologies continue to meet detection requirements after deployment to commercial airports. (Recommendation 5)
Open
In December 2019, we reported that TSA practices do not ensure that screening technologies continue to meet detection requirements after they have been deployed to airports. TSA officials stated that the agency uses certification to confirm that technologies meet detection requirements before they are deployed to airports, and calibration to confirm that technologies are at least minimally operational while in use at airports. We found that while certification and calibration serve important purposes in the acquisition and operation of screening technologies, they have not ensured that TSA screening technologies continue to meet detection requirements after they have been deployed. Consequently, we recommended that TSA implement a process to ensure that screening technologies continue to meet detection requirements after deployment to commercial airports. In May 2020, TSA said it had completed and approved agency policy on the development of Post Implementation Reviews (PIR) for all screening technologies; these PIRs are to explain how TSA will assess screening technology performance following deployment, including detection over time. Because TSA cannot use live explosives in sufficient quantities to test the detection capability of screening technologies, and simulants have limitations, TSA said it intends to measure the performance of system components within the detection chain instead of directly measuring detecting requirements. Each technology system would require its own PIR--or roadmap for reviewing component performance of the detection chain--because each one has unique logistics data and detection chain. In May 2020, TSA provided the Post Implementation and Periodic Review Policy (APM-20-031) and stated that it was on schedule to develop a PIR for at least one screening technology by December 31, 2020. To fully address this recommendation, TSA should provide at least one completed PIR for review, and the results of the review.

Full Report

GAO Contacts