Aviation Security:

TSA Improved Covert Testing but Needs to Conduct More Risk-Informed Tests and Address Vulnerabilities

GAO-19-374: Published: Apr 4, 2019. Publicly Released: Apr 4, 2019.

Additional Materials:

Contact:

William Russell
(202) 512-8777
RussellW@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

To test security screening at U.S. airports, TSA regularly tries to sneak guns and simulated bombs through checkpoints or in checked baggage. TSA changed its testing practices to better identify and address screening vulnerabilities.

We observed 26 covert tests and reviewed the test program and how results are used. We found that TSA’s ability to run covert tests has improved, but a new process intended to address vulnerabilities found in testing hasn't fully worked.

We made 9 recommendations, including that TSA establish timeframes for addressing the vulnerabilities it discovers.

 

TSA screening passengers at the checkpoint.

TSA screening passengers at the checkpoint.

Additional Materials:

Contact:

William Russell
(202) 512-8777
RussellW@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Two offices within the Transportation Security Administration (TSA) conduct covert tests at U.S. airports—Inspection and Security Operations. The Department of Homeland Security requires that agencies use risk information to make decisions, and TSA issues annual risk assessments of threats that its program offices should consult when making risk-based decisions, such as what covert tests to conduct. Of the two TSA offices that conduct covert tests, Inspection officials used TSA's risk assessment to guide their efforts. However, Security Operations officials relied largely on their professional judgment in making decisions about what scenarios to consider for covert testing. By not using a risk-informed approach, TSA has limited assurance that Security Operations is targeting the most likely threats.

Both Inspection and Security Operations have implemented processes to ensure that their covert tests produce quality results. However, GAO found that only Inspection has established a new process that has resulted in quality test results. Specifically, for the two reports Inspection completed for testing conducted in fiscal years 2016 and 2017 using its new process, GAO found that the results were generally consistent with quality analysis and reporting practices. On the other hand, Security Operations has not been able to ensure the quality of its covert test results, and GAO identified a number of factors that could be compromising the quality of these results. Unless TSA assesses the current practices used at airports to conduct tests, and identifies the factors that may be impacting the quality of covert testing conducted by TSA officials at airports, it will have limited assurance about the reliability of the test results it is using to address vulnerabilities.

In 2015, TSA established the Security Vulnerability Management Process to leverage agency-wide resources to address systemic vulnerabilities; however, this process has not yet resolved any identified security vulnerabilities. Since 2015, Inspection officials submitted nine security vulnerabilities identified through covert tests for mitigation, and as of September 2018, none had been formally resolved through this process. GAO found that in some cases, it took TSA officials overseeing the process up to 7 months to assign an office responsible to begin mitigation efforts. In part, this is because TSA has not established time frames and milestones for this process or established procedures to ensure milestones are met, in accordance with best practices for program management. Without doing so, TSA cannot ensure efficient and effective progress in addressing security vulnerabilities.

This is a public version of a classified report that GAO issued in January 2019. Information that TSA deemed classified or sensitive security information, such as the results of TSA's covert testing and details about TSA's screening procedures, have been omitted.

Why GAO Did This Study

TSA uses covert testing to identify potential vulnerabilities in checkpoint and checked baggage screening systems at U.S. airports. In 2015, TSA identified deficiencies in its covert testing process, and in 2017, the Department of Homeland Security Office of Inspector General's covert testing identified deficiencies in screener performance. Since these findings, TSA has taken steps intended to improve its covert test processes and to use test results to better address vulnerabilities.

GAO was asked to review TSA's covert test programs, including how the results are used to address vulnerabilities. This report analyzes the extent to which (1) TSA covert tests are risk-informed, (2) TSA covert tests for fiscal years 2016 through March 2018 produced quality information, and (3) TSA uses covert test results to address any identified security vulnerabilities.

GAO observed 26 TSA covert tests, reviewed TSA guidance, analyzed test data for fiscal years 2016, 2017, and through March 2018, and interviewed TSA officials.

What GAO Recommends

GAO is making nine recommendations, including that TSA use a risk-informed approach for selecting covert test scenarios, take steps to improve the quality of airport covert test results, and establish time frames and milestones for the key steps in its vulnerability management process. TSA concurred with all nine GAO recommendations.

For more information, contact William Russell at (202) 512-8777 or RussellW@gao.gov.

 

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: In April 2019, we reported on the Transportation Security Administration's (TSA) Covert Test program. We reported that managers for TSA's covert test program are required to document their rationale for making risk-informed program decisions, and that fully documented risk-informed decisions address how threats, vulnerabilities, and consequences were considered. We found that the two offices within TSA that conduct covert testing, Inspection and Security Operations, were not fully documenting their respective risk-informed rationales for choosing test scenarios--i.e., the screening activities (e.g., pat downs, bag searches) and threat items (e.g., guns, explosive devices) involved in tests. In June 2019, TSA provided us documentation to show that its selection of scenarios for Security Operations' covert tests would be based on an intelligence and data-driven methodology that utilizes information from multiple TSA risk assessments to prioritize domestic aviation security system vulnerabilities. Further, in September 2019, TSA provided a memorandum concerning Inspection's risk-informed rationale for selecting scenarios that included a discussion of how the agency considers threat, vulnerability, and consequence when selecting scenarios. Fully documenting their risk-informed rationales will help TSA program managers ensure their scenario selection decisions are appropriately accounting for risk, as called for by DHS and TSA guidance. As a result, this recommendation is closed as implemented.

    Recommendation: The Administrator of TSA should document its rationale for key decisions related to its risk-informed approach for selecting covert test scenarios, for both the Security Operations' and the Inspection's testing process. (Recommendation 1)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  2. Status: Closed - Implemented

    Comments: In April 2019, we reported on the Transportation Security Administration's (TSA) Covert test program. We found that TSA officials relied largely on their professional judgment in making decisions about what scenarios to use for covert testing, but by not using a risk-informed approach, TSA had limited assurance that it was targeting the most likely threats. We recommended that TSA incorporate a more risk-informed approach into its process for selecting the scenarios that TSA officials at airports use for covert tests. In April 2019, TSA developed and implemented a new, risk-informed process that will better align its selection of scenarios for covert tests at U.S. airports with TSA risk assessments. For example, the number of test scenarios selected for the program will correspond with an assessment of the likelihood of such scenarios taking place. Given these actions, TSA is better positioned to ensure that its testing resources are targeting the most likely threats to checkpoint and checked baggage screening. As a result, this recommendation is closed as implemented.

    Recommendation: The Administrator of TSA should incorporate a more risk-informed approach into Security Operations' process for selecting the covert test scenarios that are used for tests conducted by TSA officials at airports. (Recommendation 2)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  3. Status: Open

    Comments: TSA concurred with this recommendation and said it would take steps to implement it. When we confirm what actions the TSA has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Administrator of TSA should assess the current covert testing process used by TSA officials at airports—including factors that may affect the covertness and consistency of the tests—to identify opportunities to improve the quality of test data, and make changes as appropriate. (Recommendation 3)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  4. Status: Open

    Comments: TSA concurred with this recommendation and said it would take steps to implement it. When we confirm what actions the TSA has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Administrator of TSA should assess Security Operations guidance for applying root causes for test failures, and identify opportunities to clarify how they should be applied. (Recommendation 4)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  5. Status: Open

    Comments: TSA concurred with this recommendation and said it would take steps to implement it. When we confirm what actions the TSA has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Administrator of TSA should document the methodology for using the results of covert testing conducted by headquarters staff as a quality assurance process for covert testing conducted by TSA officials at airports. (Recommendation 5)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  6. Status: Closed - Implemented

    Comments: In April 2019, we reported on the Transportation Security Administration's (TSA) Covert test program. We found that TSA established the Security Vulnerability Management Process in 2015 to address any system-wide vulnerabilities it identifies through covert testing; however, this process has not yet resolved any security vulnerabilities identified through covert testing. In part, this is because TSA has not established time frames and milestones for this process. Without doing so, TSA cannot ensure efficient and effective progress in addressing security vulnerabilities that could result in potentially serious consequences for the traveling public. We recommended that TSA establish timeframes and milestones for the process that are appropriate for the level of effort required to mitigate identified vulnerabilities. In June 2019, TSA provided us with a revised version of the Security Vulnerability Management Process charter that established milestones and timeframes for completing key aspects of the process. For example, the revised charter requires that within 60 days of a vulnerability being submitted, the executive committee overseeing the process must identify a vulnerability owner (the entity within TSA that will lead efforts to address the vulnerability) and establish mitigation action plans, among other things. Given these actions, TSA is better positioned to make efficient and effective progress addressing security vulnerabilities identified through covert testing. As a result, this recommendation is closed as implemented.

    Recommendation: The Administrator of TSA should establish timeframes and milestones for key steps in its Security Vulnerability Management Process that are appropriate for the level of effort required to mitigate identified vulnerabilities. (Recommendation 6)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  7. Status: Closed - Implemented

    Comments: In April 2019, we reported on the Transportation Security Administration's (TSA) Covert test program. We found that TSA established the Security Vulnerability Management Process in 2015 to address any system-wide vulnerabilities it identifies through covert testing; however, this process has not yet resolved any security vulnerabilities identified through covert testing. In part, this is because TSA has not established a process for monitoring agency progress against established timeframes and milestones for addressing vulnerabilities. Without doing so, TSA cannot ensure efficient and effective progress in addressing security vulnerabilities that could result in potentially serious consequences to the traveling public. We recommended that TSA revise existing guidance for the Security Vulnerability Management Process to establish procedures for monitoring vulnerability owners' progress against timeframes and milestones for vulnerability mitigation, including a defined process for escalating cases when milestones are not met. In September 2019, TSA revised and approved its charter for the Security Vulnerability Management Process. The revised charter establishes new procedures for monitoring; for example, to enhance accountability for monitoring, the revised charter requires that an executive-level sponsor be assigned to each vulnerability being addressed. Furthermore, according to the revised charter, TSA will review the status of each vulnerability on a quarterly basis and escalate any mitigation efforts that do not meet established milestones to an executive body composed of TSA leadership, including the TSA Administrator and Deputy Administrator. Given these actions, TSA is better positioned to make efficient and effective progress addressing security vulnerabilities identified through covert testing. As a result, this recommendation is closed as implemented.

    Recommendation: The Administrator of TSA should revise existing guidance for the Security Vulnerability Management Process to establish procedures for monitoring vulnerability owners' progress against timeframes and milestones for vulnerability mitigation, including a defined process for escalating cases when milestones are not met. (Recommendation 7)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  8. Status: Open

    Comments: TSA concurred with this recommendation and said it would take steps to implement it. When we confirm what actions the TSA has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Administrator of TSA should develop processes for conducting and reporting to relevant stakeholders a comprehensive analysis of covert test results collected by TSA headquarters officials and TSA officials at airports to identify vulnerabilities in screener performance and common root causes contributing to screener test passes and failures. (Recommendation 8)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  9. Status: Open

    Comments: TSA concurred with this recommendation and said it would take steps to implement it. When we confirm what actions the TSA has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Administrator of TSA should develop a standard process for systematically documenting and disseminating to airport Federal Security Directors beneficial practices for conducting covert tests and using test results. (Recommendation 9)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

 

Explore the full database of GAO's Open Recommendations »

Oct 29, 2019

Oct 22, 2019

Oct 17, 2019

Oct 3, 2019

Sep 17, 2019

Sep 10, 2019

Aug 26, 2019

Aug 22, 2019

Jul 31, 2019

Looking for more? Browse all our products here