TSA Acquisitions:

Further Actions Needed to Improve Efficiency of Screening Technology Test and Evaluation

GAO-16-117: Published: Dec 17, 2015. Publicly Released: Dec 17, 2015.

Additional Materials:

Contact:

Michele Mackin
(202) 512-4841
mackinm@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Transportation Security Administration's (TSA) test and evaluation process has enabled TSA and Department of Homeland Security (DHS) officials to identify passenger and baggage screening technologies that will meet mission needs, but technology failures during testing have contributed to inefficiencies in the acquisition process. Consistent with departmental guidance and acquisition best practices, TSA's test and evaluation process provides information regarding the ability of technologies to meet mission needs before agency officials decide whether to begin full production, saving the agency from investing in potentially expensive yet ineffective equipment. From June 2010 to July 2015, half of the 22 systems that TSA tested successfully completed qualification and operational testing. TSA procured all but 1 of the 11 successful systems. Technologies that entered the test and evaluation process and were immature required significant modifications and retesting.

Number of Passenger and Baggage Screening Systems (Including Upgrades) Completing TSA Test and Evaluation Phases from June 2010 to July 2015

Number of Passenger and Baggage Screening Systems (Including Upgrades) Completing TSA Test and Evaluation Phases from June 2010 to July 2015

TSA has taken steps to improve its test and evaluation process by helping ensure technologies are mature before entering testing, but it is too soon to tell whether these actions will address all of the factors that contribute to acquisition inefficiencies. A key action TSA is taking involves developing a third party testing strategy, through which a third party will help ensure systems are mature prior to entering TSA's test and evaluation process. TSA plans to implement its approach in 2016, but it has yet to finalize key aspects of the strategy. For example, TSA has not identified whether there are a sufficient number of eligible third party testers or established a mechanism to oversee that testing. Without a finalized strategy, TSA risks unintended consequences, such as increasing acquisition costs. Further, TSA has not conducted or documented a comprehensive assessment of testing data and thus may be missing opportunities to identify additional areas for improvements to its acquisition process. An assessment of this data, such as costs incurred, could help TSA guide future reforms to the test and evaluation process to help ensure they address factors contributing to any acquisition inefficiencies.

Why GAO Did This Study

TSA, within the Department of Homeland Security, is responsible for securing the nation's civil aviation system while facilitating the movement of passengers and commerce at approximately 440 airports nationwide. TSA tests passenger and baggage screening technologies developed by industry to ensure they support TSA missions. In reviews from 2010 to 2014, GAO found that TSA encountered challenges in acquiring and deploying technologies. The Transportation Security Acquisition Reform Act contained a provision for GAO to assess TSA's test and evaluation activities for security-related technologies. This report assesses the extent to which (1) TSA's test and evaluation process helps it meet mission needs through the acquisition of passenger and baggage screening technologies, and (2) TSA has taken steps to improve the test and evaluation process.

GAO reviewed DHS and TSA acquisition and test policies, analyzed testing and acquisition documentation for technologies tested in the past five years, observed the testing process at DHS and TSA facilities, and spoke with DHS, TSA, and industry officials.

What GAO Recommends

TSA should finalize its third party testing strategy before implementation and conduct and document a comprehensive assessment of testing data to identify key factors contributing to any acquisition inefficiencies. DHS concurred with GAO's recommendations.

For more information, contact Michele Mackin at (202) 512-4841 or mackinm@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: In its response to this recommendation, the Department of Homeland Security concurred and identified initial planned actions to implement the finalize third party strategy. Subsequently, in the Spring of 2016, the TSA Office of Security Capabilities finalized its third party tester application and approval process; established quality conformance standards for potential third party testers; and gathered and considered industry feedback on potential third party test strategy consequences among other actions. Collectively, TSA established and published program requirements and procedures for the third party test strategy. In late 2016, TSA formally delayed its planned implementation of the third-party testing program by a calendar year to now be completed by December 31, 2017. TSA cited a need to conduct additional assessments, coordination challenges, and larger TSA security equipment related initiatives as the reasons for the delay. As part of its regular recommendation status reporting to GAO, TSA in the Spring 2017, noted that it is on track to meet the intent of the recommendation by the later revised date. TSA noted it had recently updated its qualification process by which qualified product lists will be populated and has already incorporated various aspects of third party testing for legacy security equipment qualification.

    Recommendation: To help ensure that actions taken to improve the test and evaluation process address identified challenges, the Administrator of TSA should finalize all aspects of the third party testing strategy before implementing further third party testing requirements for vendors to enter testing.

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  2. Status: Closed - Implemented

    Comments: In its response to this recommendation, the Department of Homeland Security concurred and established a plan to complete such an assessment by June 30, 2016. In February 2016, TSA updated us on the status of this recommendation and noted it had completed its test and evaluation master tracking tool which would be essential to performing the comprehensive assessment. TSA also notified us that it planned to analyze the tracker's outputs and interview key stakeholders to further understand all testing and evaluation challenges. The tracker included information on technology testing activities since June 2010, which is consistent with our report review. In late July 2016, provided us with the results of the comprehensive assessment of available testing data it completed. The assessment incorporated analyses of requirements definition challenges, schedule delays, and cost impacts of technology failures. Based on this analysis, TSA identified key issues driving technology acquisition delays and cost overruns. These problems included system immaturity as well as internal agency coordination problems among other items. Based on the assessment's findings, TSA is continuing to implement key initiatives aimed at driving more efficient, technology acquisition outcomes.

    Recommendation: To help ensure that the reforms TSA has underway are informed by existing information, the Administrator of TSA should conduct and document a comprehensive assessment of testing data available to date, such as timeframes for completing testing, costs incurred, and testing delays across all technology areas to identify key factors contributing to any acquisition inefficiencies and potential areas for reform.

    Agency Affected: Department of Homeland Security: Transportation Security Administration

 

Explore the full database of GAO's Open Recommendations »

Oct 5, 2017

Sep 20, 2017

Sep 7, 2017

Aug 28, 2017

Aug 3, 2017

Jul 27, 2017

Jul 26, 2017

Jul 20, 2017

May 31, 2017

Looking for more? Browse all our products here