Key Issues > Duplication & Cost Savings > GAO's Action Tracker > TSA's Behavior-Based Screening (2011-77)
justice icon, source: Comstock

Homeland Security/Law Enforcement: TSA's Behavior-Based Screening (2011-77)

Validation of the Transportation Security Administration’s behavior-based screening program is needed to justify future funding.

Action:

The Department of Homeland Security (DHS) should use an independent panel of experts to assess the methodology of its initial validation study of the Transportation Security Administration (TSA) behavior detection program to provide DHS with additional assurance regarding whether the study's methodology is sufficiently comprehensive to validate the Screening of Passengers by Observation Techniques (SPOT) program.

Progress:

GAO is not assessing this action separately as it was consolidated into action 2.

Implementing Entity:

Action:

The Department of Homeland Security (DHS) could conduct additional research to provide additional information on the extent to which the Screening of Passengers by Observation Techniques (SPOT) program can be effectively implemented in airports and to help determine the need for periodic refresher training.

This action was revised to consolidate it with action 1 cited in GAO's March 2011 report. Specifically, GAO suggests that DHS use an independent panel of experts to assess the methodology of its initial validation study of the Transportation Security Administration (TSA) behavior detection program, and conduct additional research to provide further information and assurance on the extent to which the SPOT program can be effectively implemented in airports, and to help determine the need for periodic refresher training.

Progress:

As a result of GAO’s November 2013 report, GAO is no longer assessing this action. DHS has taken steps to validate the SPOT program, to conduct additional research on the extent to which SPOT can be effectively implemented in airports, and to conduct research to identify potential gaps in training, as GAO suggested in March 2011. However, in November 2013, GAO found that DHS has not developed scientifically validated evidence establishing whether behavior detection techniques could be used to detect aviation security threats and recommended that DHS direct TSA to limit future funding support for the agency’s behavior detection activities until TSA can provide scientifically validated evidence that demonstrates that behavioral indicators can be used to identify passengers who may pose a threat to aviation security. GAO updated its May 2010 and March 2011 assessments of the SPOT program in November 2013. Specifically, GAO found that the data used in DHS’s April 2011 validation study were unreliable and that the study results were inconclusive because of methodological limitations in the study’s design and data collection. GAO reported on the results of over 400 studies conducted by the scientific community related to the ability of humans to detect deception using behavioral indicators. The studies indicated that humans’ ability to detect deception is the same as or slightly better than chance. GAO recommended in the November 2013 report that DHS direct TSA to limit future funding support for the agency’s behavior detection activities until TSA can provide scientifically validated evidence that demonstrates that behavioral indicators can be used to identify passengers who may pose a threat to aviation security. DHS did not concur with the recommendation and in March 2014, TSA stated that it was conducting another round of research to substantiate the science behind the behavioral indicators. Given the findings from this work, GAO will no longer assess TSA’s progress in conducting additional research on the extent to which SPOT can be effectively implemented in airports and in conducting research to identify potential gaps in training, but will instead assess TSA’s progress in implementing GAO’s November 2013 recommendation, which was added to GAO’s Action Tracker in April 2014 (see action 5).

Implementing Entity:

Department of Homeland Security

Action:

Congress may wish to consider limiting program funding pending receipt of an independent assessment of TSA's SPOT program. Specifically, Congress could consider freezing appropriation levels for the SPOT program at the 2010 level until the validation effort is complete.

Progress:

Program funds were frozen at fiscal year 2010 levels for fiscal year 2011. The conference report accompanying the consolidated appropriations act for fiscal year 2012 stated that funding was included for 145 additional behavior detection officers.[1] This increase is less than half of TSA’s fiscal year 2012 request for 350 full-time behavior detection officers. The conference report also directed TSA to brief congressional committees no later than 90 days after the enactment of the act on its plans and actions to implement recommendations from the DHS validation study and GAO’s May 2010 report.[2]

 

[1]H.R. Rep. No. 112-331, at 971 (2011) (Conf. Rep.).

[2] See GAO, Aviation Security: Efforts to Validate TSA’s Passenger Screening Behavior Detection Program Underway, but Opportunities Exist to Strengthen Validation and Address Operational Challenges, GAO-10-763 (Washington, D.C.: May 20, 2010).



Implementing Entity:

Congress

Action:

Upon completion of the validation effort, Congress may also wish to consider the study's results— including the Screening of Passengers by Observation Techniques (SPOT) program's effectiveness in using behavior-based screening techniques to detect terrorists in the aviation environment—in making future funding decisions regarding the program.

Progress:

As a result of GAO’s November 2013 report, GAO is no longer assessing this action. As of March 2014, the Transportation Security Administration (TSA) had not completed work needed to fully validate the science for using behavior detection techniques for counterterrorism purposes in an airport environment, and legislation had not been enacted into law that takes into consideration the results of the April 2011 SPOT program validation study, as GAO suggested in March 2011.

DHS’s April 2011 validation study on the SPOT program noted that the assessment was an initial validation step and was not designed to fully validate whether behavior detection can be used to reliably identify individuals who may pose a security risk in an airport environment. Further, GAO updated its May 2010 and March 2011 assessments of the SPOT program in a November 2013 report. Specifically, GAO found that the data used in DHS’s April 2011 validation study were unreliable and that the study results were inconclusive because of methodological limitations in the study’s design and data collection. GAO reported on the results of over 400 studies conducted by the scientific community related to the ability of humans to detect deception using behavioral indicators. The studies indicated that humans’ ability to detect deception is the same as or slightly better than chance (54 percent). GAO included a matter for congressional consideration in the November 2013 report that stated that Congress should consider the findings in the report regarding the absence of scientifically validated evidence for using behavioral indicators to identify aviation security threats when assessing the potential benefits of behavior detection activities relative to their costs when making future funding decisions related to aviation security.1In March 2014, TSA stated that it was conducting another round of research to substantiate the science behind the behavioral indicators. Given the findings from GAO’s November 2013 report, GAO is no longer tracking the status of this action, but will instead assess the November 2013 matter for congressional consideration, which was added to GAO’s Action Tracker in April 2014 (see action 6). Until TSA provides scientifically validated evidence demonstrating that behavioral indicators can be used to identify passengers who may pose a threat to aviation security, Congress is unable to assess the potential benefits of behavior detection activities relative to their costs in its future funding decisions. Additional information on these efforts could help Congress make future funding decisions regarding these activities.

[1] See also H.R. Rep. No. 113-91, at 55 (May 29, 2013) (accompanying H.R. 2217, 113th Cong. (1st Sess. 2013)) (explaining that outstanding questions remain over the value of TSA’s Behavior Detection Officer (BDO) program, which had not been sufficiently validated and for which few measures had been developed to provide its intrinsic value to the aviation security environment and directing that TSA brief the committee no later than 90 days after the date of enactment on the program’s impact on aviation security, the metrics utilized to measure this impact, and steps taken to develop a robust risk-based strategy for deploying BDOs, and recommending that TSA implement standardization testing on an annual basis at those airports where the SPOT program is established), and S. Rep. No. 113-77, at 61 (July 18, 2013) (accompanying S. 2217, 113th Cong. (1st. Sess. 2012)) (directing that TSA brief the committee no later than 90 days after the date of enactment on the progress made to implement the recommendations in the June 2013 DHS Office of Inspector General report). Pursuant to the explanatory statement accompanying Division F of the Consolidated Appropriations Act, 2014 (Public Law 113-76), enacted on January 17, 2014, TSA is directed to provide the briefings to both committees within 90 days of the act’s enactment.

Implementing Entity:

Congress

Action:

The Secretary of Homeland Security should direct the Transportation Security Administration (TSA) Administrator to limit future funding support for the agency's behavior detection activities until TSA can provide scientifically validated evidence that demonstrates that behavioral indicators can be used to identify passengers who may pose a threat to aviation security.

Progress:

Between 2013 and 2018, the Department of Homeland Security (DHS) and TSA reduced funding for behavior detection activities, ended the stand-alone behavior detection program, and eliminated the behavior detection officer position. In its fiscal year 2016 and 2017 budget requests, DHS requested funding to support a reduced number of behavior detection officers resulting in a savings of about $72 million. Although DHS did not concur with GAO’s recommendation, TSA officials stated that the recommendation was one of several factors in DHS's decision to support a reduction in the number of behavior detection officer positions. In fiscal year 2017, TSA eliminated the standalone behavior detection officer position and began integrating the former behavior detection officers into its transportation security officer screening workforce, resulting in a transfer of approximately 2,660 screeners and $196 million in funding to support increased passenger volume at TSA's checkpoints, according to TSA officials. These efforts continued through fiscal year 2018.
 
TSA also revised its list of behavior indicators, reducing the number of indicators from 94 to 36, and hired a contractor to search available literature for sources supporting its revised list of indicators. In 2017, TSA provided GAO with 178 sources to demonstrate that its indicators could be used to identify passengers who pose a threat to aviation security. In July 2017, GAO reported that it reviewed the sources and found that 175 of 178 of the sources did not provide valid evidence—that is, original research that met generally accepted research standards—for specific behavioral indicators in TSA's revised list and that the remaining 3 sources could be used as valid evidence to support 8 of the 36 indicators. While TSA should continue to limit funding for the agency's behavior detection activities until TSA can provide valid evidence for the indicators, actions taken by TSA over the past 5 years, which have limited funding for behavior detections activities, meet the intent of this recommendation.

Implementing Entity:

Department of Homeland Security

Action:

Congress should consider the findings in GAO’s November 2013 report regarding the absence of scientifically validated evidence for using behavioral indicators to identify aviation security threats when assessing the potential benefits of behavior detection activities relative to their cost when making future funding decisions related to aviation security.

Progress:

The Department of Homeland Security Appropriations Act, 2015, enacted in March 2015, imposed a funding restriction on the Transportation Security Administration (TSA) based in part on the findings in GAO’s November 2013 report.[1] Specifically, the act provided that $25 million of TSA’s appropriation shall be withheld from obligation for “Headquarters Administration” until TSA submits to the Appropriations Committees of the Senate and House of Representatives a report providing evidence demonstrating that behavioral indicators can be used to identify passengers who may pose a threat to aviation security and the plans that will be put into place to collect additional performance data.[2] In response, in August 2015, TSA submitted a report to Congress that discussed the scientific evidence it had gathered and used as a basis to revise the behavior indicators and a new behavior detection protocol it had developed. The report also discusses test strategies TSA had planned if the decision was made to deploy the protocols nationwide.[3] These tests included a pilot test of the new protocols that was underway at that time and two efforts that were under development—an operational test of the effectiveness of behavior detection and a study to examine potential disparity issues to ensure that the protocols do not systematically target individuals based on demographic, ethnic, or religious garb demographics. With regard to the act’s provision requiring TSA to outline its plans to collect performance data, TSA stated in the August 2015 report that following the operational test, it would analyze the test data collected and establish required thresholds for determining behavior detection effectiveness, including the rates at which Behavior Detection Officers accurately assess behavioral indicators and refer individuals for additional screening and the frequency with which these referrals lead to high-risk outcomes.


[1] See Pub. L. No. 114-4, 129 Stat. 39, 46 (2015). See also H.R. Rep. No. 113-481, at 64, 73 (June 19, 2014) (explaining that funding would be withheld to help ensure that security-related funding is directed to programs that have demonstrated their effectiveness, and providing further that questions remain over the value of TSA’s Behavior Detection Officer program, which has not been sufficiently validated and for which few measures have been developed to prove its intrinsic value to the aviation security environment.)

[2] TSA’s ability to obligate the $25 million is further contingent upon the submission of a report addressing the procurement of next-generation Advanced Imaging Technology systems. See Pub. L. No. 114-4, 129 Stat. at 46.

[3] Department of Homeland Security, Transportation Security Administration, Fiscal Year 2015 Report to Congress: Scientific Substantiation of Behavioral Indicators (Aug. 17, 2015). TSA provided a copy of this report to GAO on October 2, 2015.

Implementing Entity:

Congress
  • portrait of
    • Bill Russell
    • Director, Homeland Security and Justice
    • russellw@gao.gov
    • (202) 512-6360