Transportation Security:

TSA Has Developed a Risk-Based Covert Testing Program, but Could Better Mitigate Aviation Security Vulnerabilities Identified Through Covert Tests

GAO-08-958: Published: Aug 8, 2008. Publicly Released: Aug 14, 2008.

Additional Materials:

Contact:

Stephen M. Lord
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Transportation Security Administration (TSA) uses undercover, or covert, testing to approximate techniques that terrorists may use to identify vulnerabilities in and measure the performance of airport security systems. During these tests, undercover inspectors attempt to pass threat objects through passenger and baggage screening systems, and access secure airport areas. In response to a congressional request, GAO examined (1) TSA's strategy for conducting covert testing of the transportation system and the extent to which the agency has designed and implemented its covert tests to achieve identified goals; and (2) the results of TSA's national aviation covert tests conducted from September 2002 to June 2007, and the extent to which TSA uses the results of these tests to mitigate security vulnerabilities. To conduct this work, GAO analyzed covert testing documents and data and interviewed TSA and transportation industry officials.

TSA has designed and implemented risk-based national and local covert testing programs to achieve its goals of identifying vulnerabilities in and measuring the performance the aviation security system, and has begun to determine the extent to which covert testing will be used in non-aviation modes of transportation. TSA's Office of Inspection (OI) used information on terrorist threats to design and implement its national covert tests and determine at which airports to conduct tests based on the likelihood of a terrorist attack. However, OI did not systematically record the causes of test failures or practices that resulted in higher pass rates for tests. Without systematically recording reasons for test failures, such as failures caused by screening equipment not working properly, as well as reasons for test passes, TSA is limited in its ability to mitigate identified vulnerabilities. OI officials stated that identifying a single cause for a test failure is difficult since failures can be caused by multiple factors. TSA recently redesigned its local covert testing program to more effectively measure the performance of passenger and baggage screening systems and identify vulnerabilities. However, it is too early to determine whether the program will meet its goals since it was only recently implemented and TSA is still analyzing the results of initial tests. While TSA has a well established covert testing program in commercial aviation, the agency does not regularly conduct covert tests in non-aviation modes of transportation. Furthermore, select domestic and foreign transportation organizations and DHS components use covert testing to identify security vulnerabilities in non-aviation settings. However, TSA lacks a systematic process for coordinating with these organizations. TSA covert tests conducted from September 2002 to June 2007 have identified vulnerabilities in the commercial aviation system at airports of all sizes, and the agency could more fully use the results of tests to mitigate identified vulnerabilities. While the specific results of these tests and the vulnerabilities they identified are classified, covert test failures can be caused by multiple factors, including screening equipment that does not detect a threat item, Transportation Security Officers (TSOs), formerly known as screeners, not properly following TSA procedures when screening passengers, or TSA screening procedures that do not provide sufficient detail to enable TSOs to identify the threat item. TSA's Administrator and senior officials are routinely briefed on covert test results and are provided with test reports that contain recommendations to address identified vulnerabilities. However, TSA lacks a systematic process to ensure that OI's recommendations are considered and that the rationale for implementing or not implementing OI's recommendations is documented. Without such a process, TSA is limited in its ability to use covert test results to strengthen aviation security. TSA officials stated that opportunities exist to improve the agency's processes in this area. In May 2008, GAO issued a classified report on TSA's covert testing program. That report contained information that was deemed either classified or sensitive. This version of the report summarizes our overall findings and recommendations while omitting classified or sensitive security information.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: According to the Transportation Security Administration (TSA), the Office of Inspections (OI) has developed an Access database for the Office of Security Operations (OSO), to track and respond to OI's recommendations. In addition to current recommendations, OSO plans to enter future OI recommendations into the database. In addition, OSO issued an Operations Directive establishing a process for addressing OI recommendations. These actions, as described by TSA, address the intent of our recommendation

    Recommendation: To help ensure that the results of covert tests are more fully used to mitigate vulnerabilities identified in the transportation security system, the Assistant Secretary of Homeland Security for TSA should develop a systematic process to ensure that the Office of Security Operations (OSO) considers all recommendations made by OI in a timely manner as a result of covert tests, and document its rationale for either taking or not taking action to address these recommendations.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

  2. Status: Closed - Implemented

    Comments: In October and November 2010, Transportation Security Administration (TSA) officials provided a response on what actions the agency has taken to address this recommendation. According to TSA, the agency has addressed this recommendation through the efforts of its International Working Group on Land Transportation Security (IWGLTS). Through the IWGLTS, TSA has a process in place that encourages the sharing of information about any international covert testing program that may be in place with various foreign partners. The results so far have indicated there is little activity in this area. In addition, Transportation Security Inspectors were also queried regarding any knowledge of active domestic covert testing programs, and the results were the same. Similarly, TSA collects smart security practices from the domestic transit/passenger rail agencies through the use of our Baseline Assessment for Security Enhancement security assessment program. The TSA Transportation Surface Inspectors report any smart security practice in their final reports of the assessments they have conducted of the largest 100 transit agencies. To date, there have been no smart practices identified in the covert testing category. As a result, this recommendation is closed as implemented.

    Recommendation: To help ensure that the results of covert tests are more fully used to mitigate vulnerabilities identified in the transportation security system, the Assistant Secretary of Homeland Security for TSA should, as TSA explores the use of covert testing in non-aviation modes of transportation, develop a process to systematically coordinate with domestic and foreign transportation organizations that already conduct these tests to learn from their experiences.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

  3. Status: Closed - Implemented

    Comments: In October and November 2010, Transportation Security Administration Officials (TSA) officials provided a response on what actions the agency has taken to address this recommendation. The Office of Inspection (OI) is now collecting reasons for success and inputting that information directly into the database in addition to the practice of putting such information into the remarks field of the database. In addition to its periodic reports on covert testing, OI is also preparing briefings on significant best practices for the Assistant Administrator for Security Operations and his staff. OI will also continue the practice of briefing the affected Federal Security Director and staff at each covert testing out brief on areas of superior performance as OI has done since the inception of covert testing. In addition, TSA's Office of Security Operations (OSO) has established a process for collecting, analyzing, and disseminating recommendations for improving screening performance. Screening performance data is collected through the Aviation Screening Assessment Program (ASAP) through covert testing conducted locally and entered into a national database. At the completion of each cycle, test results are analyzed to uncover points of failure, identify their root causes, and determine recommendations to improve aviation security. The findings, root causes and recommendations are then shared with the Assistant Administrator of OSO for review and approval. Once recommendations are approved, the information is shared with various TSA offices for implementation. Four full cycles of ASAP data (6 months each) have already been completed. Twenty nine recommendations have been approved by OSO senior leadership. Some of these recommendations have either been implemented or are planned to be implemented. As a result, this recommendation is closed as implemented.

    Recommendation: To help ensure that the results of covert tests are more fully used to mitigate vulnerabilities identified in the transportation security system, the Assistant Secretary of Homeland Security for TSA should develop a process for collecting, analyzing, and disseminating information on practices in place at those airports that perform well during national and local covert tests in order to assist TSA managers in improving the effectiveness of checkpoint screening operations.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

  4. Status: Closed - Implemented

    Comments: In October and November 2010, Transportation Security Administration officials provided a response on what actions the agency has taken to address this recommendation. The Office of Inspection (OI) has modified its database to include a drop-down field to record reasons for success or failure. Further, the data collection forms used by OI inspection teams have been modified to require the Team Leader to assess and indicate the primary reason for success or failure. OI analysts then input this information into the database and include this information in their analysis as part of briefing and report generation. As a result, this recommendation is closed as implemented.

    Recommendation: To help ensure that the results of covert tests are more fully used to mitigate vulnerabilities identified in the transportation security system, the Assistant Secretary of Homeland Security for TSA should require OI inspectors to document the specific causes of all national covert testing failures--including documenting failures related to TSOs, screening procedures, and equipment--in the covert testing database to help TSA better identify areas for improvement, such as additional TSO training or revisions to screening procedures.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

  5. Status: Closed - Implemented

    Comments: According to the Transportation Security Administration (TSA), the Office of Security Operations (OSO) was reorganized in December 2008, and responsibility for addressing Office on Inspections (OI), Aviation Assessment Screening Program (ASAPP, and other recommendations will be addressed using an Integrated Process Team (IPT) approach. The IPT is known as CROSSING-Continuous Reinforcement of Operational Security Skills with Intelligently Networked Guidance. Within the CROSSING framework, recommendations derived from covert testing will continue to be gathered and evaluated (e.g., study, pilot, experiment). CROSSING IPT members will analyze and evaluate resulting data to determine whether a recommendation effectively mitigates vulnerabilities identified during covert testing. The first study, (step 1), focused on evaluating the effectiveness of the Improvised Explosive (IED) drills, a nationally implemented program originating from an OI recommendation. The results from this study have been analyzed and the final report of recommendations was routed to TSA leadership in July/August 2008. Where appropriate, OSO will continue to use this process to initiate studies to determine the success of actions implemented to address security gaps identified by OI covert testing. The IED drill study final report was briefed to OSO, and associated recommendations were provided to Operations and Technical Training (OTT) for implementation. OTT is in the process of expanding the scope if the IED drills from carry-on baggage only to all procedures and technology at the checkpoint to include IEDs on person and pat-down as has been identified by OI. OTT completed the update to the Operations Directive, OD-400-50-1-12, IED Checkpoint Drills. These actions, when taken together, address the intent of our recommendation.

    Recommendation: To help ensure that the results of covert tests are more fully used to mitigate vulnerabilities identified in the transportation security system, the Assistant Secretary of Homeland Security for TSA should require OSO to develop a process for evaluating whether the action taken to implement OI's recommendations mitigated the vulnerability identified during covert tests, such as using follow-up national or local covert tests to determine if these actions were effective.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security: Transportation Security Administration

 

Explore the full database of GAO's Open Recommendations »

Dec 17, 2014

Nov 6, 2014

Oct 14, 2014

Sep 30, 2014

Sep 24, 2014

Sep 18, 2014

Sep 17, 2014

Sep 10, 2014

Looking for more? Browse all our products here