Secure Flight: TSA Should Take Additional Steps to Determine Program Effectiveness
Highlights
What GAO Found
Since 2009, Secure Flight has changed from a program that identifies passengers as high risk solely by matching them against the No Fly and Selectee Lists to one that assigns passengers a risk category: high risk, low risk, or unknown risk. In 2010, following the December 2009 attempted attack of a U.S.-bound flight, which exposed gaps in how agencies used watchlists to screen individuals, the Transportation Security Administration (TSA) began using risk-based criteria to identify additional high-risk passengers who may not be in the Terrorist Screening Database (TSDB), but who should be designated as selectees for enhanced screening. Further, in 2011, TSA began screening against additional identities in the TSDB that are not already included on the No Fly or Selectee Lists. In addition, as part of TSA PreüTM, a 2011 program through which TSA designates passengers as low risk for expedited screening, TSA began screening against several new lists of preapproved low-risk travelers. TSA also began conducting TSA Pre✓™ risk assessments, an activity distinct from matching against lists that uses the Secure Flight system to assign passengers scores based upon travel-related data, for the purpose of identifying them as low risk for a specific flight.
TSA has processes in place to implement Secure Flight screening determinations at airport checkpoints, but could take steps to enhance these processes. TSA information from May 2012 through February 2014 indicates that screening personnel have made errors in implementing Secure Flight determinations at the checkpoint. However, TSA does not have a process for systematically evaluating the root causes of these screening errors. GAO's interviews with TSA officials at airports yielded examples of root causes TSA could identify and address. Evaluating the root causes of screening errors, and then implementing corrective measures, in accordance with federal internal control standards, to address those causes could allow TSA to strengthen security screening at airports.
Since 2009, Secure Flight has established program goals that reflect new program functions to identify additional types of high-risk and also low-risk passengers; however, current program performance measures do not allow Secure Flight to fully assess its progress toward achieving all of its goals. For example, Secure Flight does not have measures to assess the extent of system matching errors. Establishing additional performance measures that adequately indicate progress toward goals would allow Secure Flight to more fully assess the extent to which it is meeting program goals. Furthermore, TSA lacks timely and reliable information on all known cases of Secure Flight system matching errors. More systematic documentation of the number and causes of these cases, in accordance with federal internal control standards, would help TSA ensure Secure Flight is functioning as intended.
This is a public version of a sensitive report that GAO issued in July 2014. Information that the Department of Homeland Security (DHS) and the Department of Justice deemed sensitive has been removed.
Why GAO Did This Study
In 2009, DHS's TSA began using Secure Flight to screen passengers against high-risk lists. These lists, subsets of the TSDB—the U.S. government's consolidated list of known and suspected terrorists—included the No Fly List, to identify those who should be prohibited from boarding flights, and the Selectee List, to identify those who should receive enhanced screening at airport checkpoints.
GAO was asked to assess the current status of the program. This report examines (1) changes to the Secure Flight program since 2009, (2) TSA's efforts to ensure that Secure Flight's screening determinations for passengers are implemented at airport checkpoints, and (3) the extent to which program performance measures assess progress toward goals. GAO analyzed TSA data and documents—including checkpoint data from 2012 through 2014 and Secure Flight performance measures—and interviewed relevant DHS officials.
Recommendations
GAO recommends that TSA develop a process to regularly evaluate the root causes of screening errors at security checkpoints and implement measures to address these causes. GAO also recommends that TSA develop measures to address all aspects of performance related to program goals and develop a mechanism to systematically document the number and causes of Secure Flight system matching errors. DHS concurred with GAO's recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Transportation Security Administration | To further improve the implementation of Secure Flight risk determinations at the screening checkpoint, the Transportation Security Administration's Administrator should develop a process for regularly evaluating the root causes of screening errors across airports so that corrective measures can be identified. |
In July 2014, we reported on the Transportation Security Administration's (TSA) Secure Flight program, which was created to identify passengers who may pose a security risk before they board aircraft. During the course of our review, we found that TSA had processes in place to implement Secure Flight screening determinations at airport checkpoints, but could take steps to enhance these processes. Specifically, TSA data show that screening personnel made errors implementing Secure Flight screening determinations at airport checkpoints, but we found TSA does not have a process to systematically evaluate the root causes of these errors. Further our interviews with TSA officials at airports yielded examples of root causes that TSA could identify and address. Therefore, we recommended that TSA develop a process for regularly evaluating the root causes of screening errors involving Secure Flight determinations across airports to identify corrective measures. In August 2014, TSA provided us a description of a new process it had developed to analyze information collected on the root causes of screening errors across all national airports to identify corrective action effectiveness and communicate the results to TSA leadership. In September 2015, TSA provided us multiple examples, dating from June through early September 2015, of the results of this analysis that are being included in regular reporting to senior TSA leadership. TSA's actions should provide it with a process for regularly evaluating the root causes of errors related to Secure Flight screening determinations at the checkpoint. As a result, this recommendation is closed as implemented.
|
| Transportation Security Administration | To address the root causes of screening errors at the checkpoint, thereby strengthening checkpoint operations, the Transportation Security Administration's Administrator should implement the corrective measures TSA identifies through a root cause evaluation process. |
In July 2014, we reported on the Transportation Security Administration's (TSA) Secure Flight program, which was created to identify passengers who may pose a security risk before they board aircraft. During the course of our review, we found that TSA had processes in place to implement Secure Flight screening determinations at airport checkpoints, but could take steps to enhance these processes. Specifically, TSA data show that screening personnel made errors implementing Secure Flight screening determinations at airport checkpoints, but we found TSA does not have a process to systematically evaluate the root causes of these errors. Further our interviews with TSA officials at airports yielded examples of root causes that TSA could identify and address. Therefore, we recommended that TSA develop a process for regularly evaluating the root causes of screening errors involving Secure Flight determinations across airports to identify corrective measures. We also recommended that TSA implement the corrective measures identified through this process. In August 2014, TSA developed a new process to analyze information collected on the root causes of screening errors across all national airports. In June of 2016, TSA provided us the results of its root cause analysis of screening errors involving Secure Flight determinations from August 2015 through May 2016 as well as documentation of agency initiatives implemented during the period to address these causes. For example, TSA?s root cause analysis found that lack of attentiveness at the checkpoint led to screening errors. In response, TSA issued new standard operating procedures and provided additional information to TSA personnel at airports to help screeners focus on Secure Flight screening determinations, among other activities. As a result of these and other activities, TSA documentation reflected a decrease in number of errors in implementing Secure Flight screening determinations from August 2015 through May 2016. TSA's actions show that it has been implementing corrective measures based on its process for regularly evaluating the root causes of errors related to Secure Flight screening determinations at the checkpoint. As a result, the recommendation that TSA implement corrective measures identified through its root cause analysis process is closed as implemented.
|
| Transportation Security Administration | To assess the progress of the Secure Flight program toward achieving its goals, the Transportation Security Administration's Administrator should develop additional measures to address key performance aspects related to each program goal, and ensure these measures clearly identify the activities necessary to achieve progress toward the goal. |
In July 2014, we reported on the Transportation Security Administration's (TSA) Secure Flight program, which was created to identify commercial airline passengers who may pose a security risk before they board aircraft by matching passenger data with data from the Terrorist Screening Database (TSDB)--the government's consolidated list of known and suspected terrorists. We found that Secure Flight did not have program performance measures that would allow the program to fully assess progress toward achieving all of its goals. For example, Secure Flight had goals related to accurately matching passengers to lists derived from the TSDB (e.g., the No Fly and Selectee lists), but Secure Flight did not have measures to assess the extent to which the Secure Flight System was missing passengers who were actual matches to these lists. We recommended that TSA establish additional performance measures that adequately indicate progress toward Secure Flight program goals. In Spring 2018, a TSA contractor completed work on a data set for testing the Secure Flight system. According to TSA documents, the data set allows Secure Flight to determine how changes to the Secure Flight system will affect the number passengers accurately matched to No Fly, Selectee, PreCheck and any other lists used for screening, including the extent to which individuals on lists may be missed. TSA officials reported plans to use the data set to assess the accuracy of system matching before any future technical changes to the Secure Flight System, and to conduct bi-annual tests of the data set against the current version of the TSDB to identify and resolve any discrepancies. In addition, in July 2014 we found that TSA lacked measures to fully address two other Secure Flight goals--one related to incorporating risk-based security capabilities into the Secure Flight system, and another related to protecting passengers' privacy by safeguarding their personal information. To identify progress toward its goal of incorporating risk-based security capabilities into Secure Flight, TSA uses a document to track its enhancements to Secure Flight's risk-based capabilities, by year. For example, the document shows that program managers have adjusted the Secure Flight system to automatically designate new categories of travelers for expedited screening. To address its privacy goal, TSA developed a mechanism to log requests from external entities for Secure Flight passenger data that are handled according to privacy requirements. With these additional efforts and measures, TSA should have more complete information to assess progress toward achieving all program goals related to the Secure Flight system.
|
| Transportation Security Administration | To provide Secure Flight program managers with timely and reliable information on cases in which TSA learns of Secure Flight system matching errors, the Transportation Security Administration's Administrator should develop a mechanism to systematically document the number and causes of such cases, for the purpose of improving program performance. |
In July 2014, we reported on the Transportation Security Administration's (TSA) Secure Flight program, which was created to identify commercial airline passengers who may pose a security risk before they board aircraft by matching passenger data with data from the Terrorist Screening Database (TSDB)--the government's consolidated list of known and suspected terrorists. During the course of our review, we found that TSA did not have timely and reliable information on matching errors of the Secure Flight system (i.e., the computerized matching and manual reviews TSA conducts to identify matches of passenger and TSDB data). Specifically, it took TSA several months to identify a list of system matching errors, and, upon review, we found the list to be incomplete. Data on matching errors of the Secure Flight system are reviewed by Secure Flight's Match Review Board to determine if any actions should be taken to prevent similar errors from reoccurring. Without timely and reliable information on system matching errors, TSA is not in the best position to determine all potential causes of system matching errors and to identify and implement sufficient corrective actions. Consequently, we recommended that TSA develop a mechanism to systematically document the number and causes of Secure Flight system matching errors for the purposes of improving program performance. In November 2015, TSA provided us a copy of its new tracking tool to document all cases of system matching errors. To demonstrate use of the tool, TSA also provided details on incidents occurring from January through October 2015, including a description of the reason for each incident. TSA also provided documentation from October and November 2015 meetings of its Match Review Board that included analysis of specific incidents documented by the tracking tool and proposed measures to address their respective causes. Given these actions, TSA should have more timely and reliable information on Secure Flight system matching errors for its deliberations on improving overall system performance. As a result, this recommendation is closed as implemented.
|