Advanced Imaging Technology:

TSA Needs Additional Information before Procuring Next-Generation Systems

GAO-14-357: Published: Mar 31, 2014. Publicly Released: Apr 30, 2014.

Additional Materials:

Contact:

Stephen M. Lord
(202) 512-4379
lords@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Department of Homeland Security's (DHS) Transportation Security Administration (TSA) does not collect or analyze available information that could be used to enhance the effectiveness of the advanced imaging technology (AIT) with automated target recognition (ATR) system. Specifically, TSA does not collect or analyze available data on drills using improvised explosive devices (IED) at the checkpoint that could provide insight into how well screening officers (SO) resolve anomalies, including objects that could pose a threat to an aircraft, identified by AIT systems, because it does not enforce compliance with its operational directive. TSA's operational directive requires personnel at airports to conduct drills to assess SO compliance with TSA's screening standard operating procedures and to train SOs to better resolve anomalies identified by AIT-ATR systems. GAO found that TSA personnel at about half of airports with AIT systems did not report any IED checkpoint drill results on those systems from March 2011 through February 2013. According to TSA, it does not ensure compliance with the directive at every airport because it is unclear which office should oversee enforcing the directive. Without data on IED checkpoint drills, TSA lacks insight into how well SOs resolve anomalies detected by AIT systems, information that could be used to help strengthen existing screening processes. Potential weaknesses in the screening process could be caused by TSA not clarifying which office is responsible for overseeing TSA's operational directive, directing that office to ensure enforcement of the directive in conducting these drills, and analyzing the data. Further, when determining AIT-ATR system effectiveness, TSA uses laboratory test results that do not reflect the combined performance of the technology, the personnel who operate it, and the process that governs AIT-related security operations. TSA officials agreed that it is important to analyze performance by including an evaluation of the technology, operators, and processes and stated that TSA is planning to assess the performance of all layers of security. By not measuring system effectiveness based on the performance of the technology and SOs who operate the technology or taking into account current processes and deployment strategies, DHS and TSA are not ensuring that future procurements meet mission needs.

TSA completed the installation of ATR software upgrades intended to address privacy concerns for all deployed AIT systems; however, it has not met proposed milestones for enhancing capabilities as documented in its AIT roadmap—a document that contains milestones for achieving enhanced capabilities to meet the agency's mission needs. For example, TSA began operational test and evaluation for Tier II upgrades 17 months after the expected start date. Moreover, TSA did not use available scientific research or information from experts from the national laboratories or vendors on the technological challenges that it faces in developing requirements and milestones, because, according to TSA, it relied on time frames proposed by vendors. Thus, TSA cannot ensure that its roadmap reflects the true capabilities of the next generation of AIT systems by using scientific evidence and information from DHS's Science and Technology Directorate, the national laboratories, and vendors to develop a realistic schedule with achievable milestones that outlines the technological advancements, estimated time, and resources needed to achieve enhanced capabilities as outlined in TSA's roadmap.

Why GAO Did This Study

TSA accelerated the deployment of AIT systems, or full-body scanners, in response to the December 25, 2009, attempted terrorist attack on Northwest Airlines Flight 253. Pursuant to the Federal Aviation Administration Modernization and Reform Act of 2012, TSA was mandated to ensure that AIT systems were equipped with ATR software, which displays generic outlines of passengers rather than actual images, by June 1, 2013. All deployed AIT systems were equipped with ATR software by the deadline. GAO was asked to evaluate TSA's AIT-ATR systems' effectiveness. This report addresses the extent to which (1) TSA collects and analyzes available information that could be used to enhance the effectiveness of the AIT-ATR system and (2) TSA has made progress toward enhancing AIT capabilities to detect concealed explosives and other threat items, and any challenges that remain. GAO analyzed testing results conducted by the Transportation Security Laboratory and TSA personnel at airports and interviewed DHS and TSA officials. This is a public version of a classified report that GAO issued in December 2013. Information DHS and TSA deemed classified or sensitive has been omitted, including information and recommendations related to improving AIT capabilities.

What GAO Recommends

GAO recommends that TSA, among other things, clarify which office should oversee its operational directive, better measure system effectiveness, and develop a realistic schedule before procuring future generations. TSA concurred with GAO's recommendations.

For more information, contact Stephen M. Lord at (202) 512-4379 or lords@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: In response to this recommendation, the Transportation Security Administration (TSA) conducted a review of programs that contribute to assessing screening performance and determined that the Improvised Explosive Device Checkpoint Drills training program did not result in meaningful data that could be analyzed to identify potential weaknesses. Further, TSA determined that the program is resource intensive and no longer fulfills an existing need within TSA. In May 2014, TSA provided GAO with a memo cancelling the checkpoint drills operational directive, which was approved by the TSA Administrator on May 20, 2014. Based on these actions, we are closing this recommendation as not implemented.

    Recommendation: To help ensure that TSA improves SO performance on AIT-ATR systems and uses resources effectively, the Administrator of the Transportation Security Administration should clarify which office is responsible for overseeing TSA's IED screening checkpoint drills operational directive, direct the office to ensure enforcement of the directive in conducting these drills, and analyze the data to identify any potential weaknesses in the screening process.

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  2. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To help ensure that TSA improves SO performance on AIT-ATR systems and uses resources effectively, the Administrator of the Transportation Security Administration should establish protocols that facilitate the capturing of operational data on secondary screening of passengers at the checkpoint to determine the extent to which AIT-ATR system false alarm rates affect operational costs once AIT-ATR systems are networked together.

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  3. Status: Closed - Implemented

    Comments: TSA provided documentation showing that, while conducting operational testing of the AIT-2 system, the agency considered screening officer performance and measured AIT-2 system effectiveness based on both the performance of the AIT-2 technology and the screening officers who operate it. Based on these actions, we are closing this recommendation as implemented.

    Recommendation: To help ensure that TSA invests in screening technology that meets mission needs, the Administrator of the Transportation Security Administration should measure system effectiveness based on the performance of the AIT-2 technology and screening officers who operate the technology, while taking into account current processes and deployment strategies before procuring AIT-2 systems.

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  4. Status: Closed - Implemented

    Comments: In response to this recommendation, TSA sought input from the DHS Science & Technology Directorate, U.S. government national laboratories, industry, and academia to develop a technology maturation roadmap for Advanced Imaging Technology (AIT). This technology roadmap, which was prepared by The Johns Hopkins University Applied Physics Laboratory on behalf of TSA, is designated for official use only. TSA subsequently produced a significantly scaled down public version of the technology roadmap, which it published on the Federal Business Opportunities website in February 2015. Because TSA's public roadmap does not contain all elements outlined in the original recommendation (technological advancements, estimated time, and resources needed to achieve TSA's Tier IV end state), we are closing this recommendation as implemented based on the technology roadmap produced by Johns Hopkins University.

    Recommendation: To help ensure that TSA invests in screening technology that meets mission needs, the Administrator of the Transportation Security Administration should use scientific evidence and information from DHS's Science and Technology Directorate, and the national laboratories, as well as information and data provided by vendors to develop a realistic schedule with achievable milestones that outlines the technological advancements, estimated time, and resources needed to achieve TSA's Tier IV end state before procuring AIT-2 systems.

    Agency Affected: Department of Homeland Security: Transportation Security Administration

 

Explore the full database of GAO's Open Recommendations »

May 26, 2016

May 24, 2016

May 17, 2016

Apr 25, 2016

Apr 19, 2016

Apr 15, 2016

Apr 14, 2016

Apr 12, 2016

Mar 31, 2016

Looking for more? Browse all our products here