GAO-11-318SP: Homeland security/Law enforcement: Validation of TSA's behavior-based screening program is needed to justify funding or expansion

Homeland security/Law enforcement

Validation of TSA's behavior-based screening program is needed to justify funding or expansion

Why Area Is Important

 

The terrorist attacks of September 11, 2001, highlighted the need to improve security within the nation's civil aviation system to deter persons seeking to repeat similar attacks on the nation's critical infrastructure. To enhance aviation security, the Department of Homeland Security's (DHS) Transportation Security Administration (TSA) began testing in October 2003 of its Screening of Passengers by Observation Techniques (SPOT) program to identify persons who may pose a risk to aviation security. In fiscal year 2010, about 3,000 Behavior Detection Officers were deployed to 161 airports at an annual cost of over $200 million. As highlighted in GAO's May 2010 report, TSA did not validate the science supporting the program or determine if behavior detection techniques could be successfully used across the aviation system to detect threats before deploying the SPOT program.

What GAO Found

 

TSA has implemented and now seeks to expand a behavior screening program, which has not yet been validated. A validation study is underway, but questions exist regarding whether the study's methodology is sufficiently comprehensive to validate the SPOT program. The results of an independent assessment are needed to determine whether current validation efforts are sufficiently comprehensive to validate the program.

After operationally testing behavioral detection screening started in October 2003, TSA created separate Behavior Detection Officer positions as part of the SPOT program beginning in fiscal year 2007. TSA designed SPOT to provide these officers with a means of identifying persons who may pose a potential security risk at TSA-regulated airports by focusing on behaviors and appearances that deviate from an established baseline, and that may be indicative of stress, fear, or deception. Behavior Detection Officers have been selectively deployed to 161 of the 462 TSA-regulated airports in the United States. The conference report accompanying the fiscal year 2010 DHS appropriations act provided that $211.9 million was for the SPOT program.[1] The administration has requested $232 million for SPOT for fiscal year 2011, a $20.2 million (9.5 percent) increase over the current funding level. This increase would support a workforce increase from about 3,000 to 3,350 Behavior Detection Officers.

As discussed in GAO's May 2010 report, TSA deployed SPOT nationwide before first determining whether there was a scientifically valid basis for using behavior and appearance indicators as a means for reliably identifying passengers who may pose a risk to the U.S. aviation system. According to TSA, SPOT was deployed before a scientific validation of the program was completed in response to the need to address potential threats, but was based upon scientific research available at the time regarding human behaviors. TSA officials also stated that no other large-scale U.S. or international screening program incorporating behavior- and appearance-based indicators has ever been rigorously scientifically validated.

However, a 2008 report issued by the National Research Council of the National Academy of Sciences noted that an information-based program, such as a behavior detection program, should first determine if a scientific foundation exists and use scientifically valid criteria to evaluate its effectiveness before going forward. The report added that programs should have a sound experimental basis and that the documentation on the program's effectiveness should be reviewed by an independent entity capable of evaluating the supporting scientific evidence.[2] Thus, and as recommended in GAO's May 2010 report, an independent panel of experts could help DHS develop a comprehensive methodology to determine if the SPOT program is based on valid scientific principles that can be effectively applied in an airport environment for counterterrorism purposes. Specifically, GAO's May 2010 report recommended that the Secretary of Homeland Security convene an independent panel of experts to review the methodology of a validation study on the SPOT program being conducted by DHS's Science and Technology Directorate to determine whether the study's methodology is sufficiently comprehensive to validate the SPOT program. GAO recommended that this assessment include appropriate input from other federal agencies with expertise in behavior detection and relevant subject matter experts. DHS concurred and stated that its current validation study includes an independent review of the program that will include input from other federal agencies and relevant experts. According to DHS, this independent review is expected to be completed in February 2011.



[1]H.R. Rep. 111-298, at 77 (2009) (Conf. Rep.).

[2]A study performed by the JASON program office raised similar concerns. The JASON program office is an independent scientific advisory group that provides consulting services to the U.S. government on matters of defense science and technology.

Actions Needed

 

As discussed in GAO's May 2010 report, DHS has contracted with the American Institutes for Research to conduct its validation study. However, DHS's response to GAO's report did not describe how the review currently planned is designed to determine whether the study's methodology is sufficiently comprehensive to validate the SPOT program. As GAO noted in its report, research on other issues, such as determining the number of individuals needed to observe a given number of passengers moving at a given rate per day in an airport environment or the duration that such observation can be conducted by Behavior Detection Officers before observation fatigue affects effectiveness, could provide additional information on the extent to which SPOT can be effectively implemented in airports. Additional research could also help determine the need for periodic refresher training since research has not yet determined whether behavior detection is easily forgotten or can be potentially degraded with time or lack of use. Because such questions exist, using an independent panel of experts to assess the methodology of the study could provide DHS with additional assurance regarding whether the study's methodology is sufficiently comprehensive to validate the SPOT program. DHS stated that the ongoing independent review is being conducted by an independent panel of experts that includes a broad range of operational agencies and academia and will include, among other things, recommended additional studies that should be undertaken to more fully validate the science underlying the SPOT screening process. Moreover, DHS stated that its current effort to validate the science underlying SPOT includes three years of operational SPOT referral data and preliminary results indicate that it is supportive of SPOT. However, in May 2010, GAO reported weaknesses in TSA's process for maintaining operational data from the SPOT program database. Because of these data-related issues, GAO reported that meaningful analyses could not be conducted to determine if there is an association between certain behaviors and the likelihood that a person displaying certain behaviors would be referred to a law enforcement officer or whether any behavior or combination of behaviors could be used to distinguish deceptive from nondeceptive individuals.

Congress may wish to consider limiting program funding pending receipt of an independent assessment of TSA's SPOT program. GAO identified potential budget savings of about $20 million per year if funding were frozen at current levels until validation efforts are complete. Specifically, in the near term, Congress could consider freezing appropriation levels for the SPOT program at the 2010 level until the validation effort is completed. Assuming that TSA is planning to expand the program at a similar rate each year, this action could result in possible savings of about $20 million per year, since TSA is seeking about a $20 million increase for SPOT in fiscal year 2011. Upon completion of the validation effort, Congress may also wish to consider the study's results—including the program's effectiveness in using behavior-based screening techniques to detect terrorists in the aviation environment—in making future funding decisions regarding the program.

Framework for Analysis

 

The information contained in this analysis is based on the related GAO product listed under the "Related GAO Products" tab.

Area Contact

 

For additional information about this area, contact Steve Lord at (202) 512-4379 or lords@gao.gov.