Aviation Security:

TSA Has Completed Key Activities Associated with Implementing Secure Flight, but Additional Actions Are Needed to Mitigate Risks

GAO-09-292: Published: May 13, 2009. Publicly Released: May 13, 2009.

Additional Materials:

Contact:

Stephen M. Lord
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

To enhance aviation security, the Department of Homeland Security's (DHS) Transportation Security Administration (TSA) developed a program--known as Secure Flight--to assume from air carriers the function of matching passenger information against terrorist watch-list records. In accordance with a mandate in the Department of Homeland Security Appropriations Act, 2008, GAO's objective was to assess the extent to which TSA met the requirements of 10 statutory conditions related to the development of the Secure Flight program. GAO is required to review the program until all 10 conditions are met. In September 2008, DHS certified that it had satisfied all 10 conditions. To address this objective, GAO (1) identified key activities related to each of the 10 conditions; (2) identified federal guidance and best practices that are relevant to successfully meeting each condition; (3) analyzed whether TSA had demonstrated, through program documentation and oral explanation, that the guidance was followed and best practices were met; and (4) assessed the risks associated with not fully following applicable guidance and meeting best practices.

As of April 2009, TSA had generally achieved 9 of the 10 statutory conditions related to the development of the Secure Flight program and had conditionally achieved 1 condition (TSA had defined plans, but had not completed all activities for this condition). Also, TSA's actions completed and those planned have reduced the risks associated with implementing the program. Although DHS asserted that TSA had satisfied all 10 conditions in September 2008, GAO completed its initial assessment in January 2009 and found that TSA had not demonstrated Secure Flight's operational readiness and that the agency had generally not achieved 5 of the 10 statutory conditions. Consistent with the statutory mandate, GAO continued to review the program and, in March 2009, provided a draft of this report to DHS for comment. In the draft report, GAO noted that TSA had made significant progress and had generally achieved 6 statutory conditions, conditionally achieved 3 conditions, and had generally not achieved 1 condition. After receiving the draft report, TSA took additional actions and provided GAO with documentation to demonstrate progress related to 4 conditions. Thus, GAO revised its assessment in this report. Related to the condition that addresses the efficacy and accuracy of search tools, TSA had not yet developed plans to periodically assess the performance of the Secure Flight system's name-matching capabilities, which would help ensure that the system is working as intended. GAO will continue to review the Secure Flight program until all 10 conditions are generally achieved.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: Section 522(a) of the Department of Homeland Security (DHS) Appropriations Act, 2005, set forth 10 conditions related to the development and implementation of Secure Flight that the program must successfully meet in order to become operational. Under this act, GAO was required to review the Secure Flight program until it determined that all 10 conditions had been successfully met. Statutory Condition 3 required the Transportation Security Administration (TSA) to demonstrate the efficacy and accuracy of the search tools used as part of Secure Flight and perform stress testing on the Secure Flight system. In May 2009, we reported that TSA had generally achieved the part of Condition 3 that requires TSA to demonstrate the efficacy and accuracy of the search tools used as part of Secure Flight. At that time, TSA reported that it was considering conducting periodic reviews of the Secure Flight system's matching capabilities and results (i.e., false positives and false negatives) to determine whether the system was performing as intended, but had not yet made final decisions regarding whether to conduct such reviews. GAO recommended that DHS take action to periodically assess the performance of the Secure Flight system's name-matching capabilities and results. In response, TSA developed performance measures to report and monitor Secure Flight's name matching capabilities. According to TSA, Secure Flight leadership reviews the daily reports, which reflect quality, match rate, false positive rates, and other metrics. Reviews include analysis, discussion with program leadership, and identification of process and data quality improvements to increase efficiency and reduce possible false positive matches to the watchlist. In addition, DHS established a multi-departmental Match Review Board Working Group and a Match Review Board to review the performance measures and watchlist matching system behaviors and recommend system changes to improve system performance. The working group meets every 2 weeks and is co-chaired by the Secure Flight Director of Vetting Analysis and the Deputy Director for Vetting. The Match Review Board is chaired by the Transportation Threat Assessment and Credentialing Office General Manager for Operations and co-chaired by the Deputy Assistant Administrator, Office of Intelligence, and consists of management-level individuals who have decision-making authority concerning procedural and system change recommendations. The board meets monthly, or as required, to review working group findings and to make system change recommendations. Based on these actions, we consider this recommendation closed as implemented.

    Recommendation: To mitigate future risks of performance shortfalls and strengthen management of the Secure Flight program moving forward, the Secretary of Homeland Security should direct the Assistant Secretary for the Transportation Security Administration to periodically assess the performance of the Secure Flight system's matching capabilities and results to determine whether the system is accurately matching watch-listed individuals while minimizing the number of false positives--consistent with the goals of the program; document how this assessment will be conducted and how its results will be measured; and use these results to determine whether the system settings should be modified.

    Agency Affected: Department of Homeland Security

  2. Status: Closed - Not Implemented

    Comments: Section 522(a) of the Department of Homeland Security (DHS) Appropriations Act, 2005, set forth 10 conditions related to the development and implementation of Secure Flight that the program must successfully meet in order to become operational. Under this act, GAO was required to review the Secure Flight program until it determined that all 10 conditions had been successfully met. Statutory Condition 9 required the Transportation Security Administration (TSA)--pursuant to the requirements of section 44903(i)(2)(A)[sic] of title 49, United States Code--to modify Secure Flight with respect to intrastate transportation to accommodate states with unique air transportation needs and passengers who might otherwise regularly trigger primary selectee status. In May 2009, we reported that TSA had generally achieved this condition. According to TSA, the agency modified Computer Assisted Passenger Prescreening System (CAPPS) rules to address air carriers operating in states with unique transportation needs and passengers who might otherwise regularly trigger primary selectee status. However, our review found that TSA lacked data on the effect of its modifications on air carrier selectee rates. To ensure that passenger prescreening is working as intended, we recommended that TSA conduct outreach to air carriers--particularly carriers in states with unique transportation needs--to determine whether modifications to the CAPPS rules and the related security amendment had achieved their intended effect. In August 2011, TSA reported that the recommendation was no longer valid because of changes to the security amendment. Therefore, we consider this recommendation closed as not implemented based on TSA's follow-up response.

    Recommendation: To ensure that passenger prescreening is working as intended, the Secretary of Homeland Security should direct the Assistant Secretary for the Transportation Security Administration to conduct outreach to air carriers--particularly carriers in states with unique transportation needs--to determine whether modifications to the CAPPS rules and related security amendment have achieved their intended effect.

    Agency Affected: Department of Homeland Security

 

Explore the full database of GAO's Open Recommendations »

Jun 10, 2015

Jun 2, 2015

May 13, 2015

Apr 30, 2015

Apr 22, 2015

Apr 6, 2015

Looking for more? Browse all our products here