US-VISIT Pilot Evaluations Offer Limited Understanding of Air Exit Options
GAO-10-860, Aug 10, 2010
The Department of Homeland Security's (DHS) U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT) program is to control and monitor the entry and exit of foreign visitors by storing and processing biometric and biographic information. The entry capability has operated since 2006; an exit capability is not yet implemented. In September 2008, the Consolidated Security, Disaster Assistance, and Continuing Appropriations Act, 2009, directed DHS to pilot air exit scenarios with the U.S. Customs and Border Protection (CBP) and airlines, and to provide a report to congressional committees. DHS conducted CBP and Transportation Security Administration (TSA) pilots and issued its evaluation report in October 2009. Pursuant to the act, GAO reviewed the evaluation report to determine the extent to which (1) the report addressed statutory conditions and legislative directions; (2) the report aligned with the scope and approach in the pilot evaluation plan; (3) the pilots were conducted in accordance with the evaluation plan; and (4) the evaluation plan satisfied relevant guidance. To do so, GAO compared the report to statutory conditions, the evaluation plan, and relevant guidance.
The evaluation report partially addressed statutory conditions and legislative directions and expectations. Specifically, the report addressed the statutory condition for CBP to collect biometric information on exiting foreign nationals and four legislative directions and expectations for conducting the pilots. However, DHS was unable to address the statutory condition for an airline scenario because no airline was willing to participate. Also, the report did not meet a legislative expectation for gathering information on the security of information collected from visitors subject to US-VISIT. DHS officials told us that DHS did not view the expectation in the House report as a requirement. Moreover, they said that security requirements were tested prior to the pilots and there were no reported security incidents. However, DHS did not supply documentation that demonstrated the operational verification of pilot security requirements. The evaluation report generally aligned with the scope and approach in the evaluation plan. Specifically, the objectives and operational conditions described in the evaluation report were generally consistent with the evaluation plan. However, the report did not fully align with the evaluation plan because certain metrics, observations, and costs (e.g., percentage of system downtime or inoperability, costs for requirements analysis) were not reported as planned. Also, the reported scope and approach of the pilots included limitations not defined in the plan, such as suspending exit screening at departure gates to avoid flight delays. Such divergence was due, in part, to a desire to minimize the pilot's impact on the airports, airlines, and travelers. The pilots were not conducted in accordance with the evaluation plan, in that they did not meet the plan's stated purpose of operationally evaluating the air exit requirements. More specifically, about 30 percent of the requirements were not operationally tested, either as part of the pilots or as part of another exit project. Rather, they were tested, for example, prior to commencement of pilot operations or as part of another exit project that has yet to complete operational testing. DHS officials considered such testing of requirements to be sufficient. The evaluation plan did not satisfy relevant guidance, such as defining standards for gauging the pilots' performance, defining a comprehensive methodology for selecting airports and flights, and planning data analysis to ensure that the results of the evaluation support air exit decision making. The evaluation plan diverged from such guidelines, in part, because DHS viewed reporting on how the pilot results would be used to be outside the scope of its report. Collectively, the above limitations in scope, approach, and reporting restrict the pilots' ability to inform a decision for a long-term air exit solution and point to the need for DHS to leverage compensating sources of information on air exit's operational impacts in making air exit solution decisions. GAO recommends that the Secretary of Homeland Security identify additional sources of information beyond the pilots to inform a strategic air exit solution decision. DHS agreed with the recommendation.
- Closed - implemented
- Closed - not implemented
Recommendation for Executive Action
Recommendation: To the extent that the limitations in the Air Exit Pilots are not addressed through other information sources, the Secretary of Homeland Security should direct the Under Secretary for National Protection and Programs to have the US-VISIT Program Director identify additional sources for the operational impacts of air exit not addressed in the pilots' evaluation and to incorporate these sources into its air exit decision making and planning.
Agency Affected: Department of Homeland Security
Comments: In May 2012, the Department of Homeland Security (DHS) reported internally on the results of the DHS Science and Technology Directorate's analysis of previous US-VISIT air exit pilot programs and assessment of available biometric technologies. The report also discussed DHS's plans to implement several recommendations that have the potential to result in additional information sources to assess the operational impacts of a biometric air exit capability on stakeholders (e.g. airports, airlines, and travelers). These additional information sources may include a new biometric air exit concept of operations, developmental scenario testing to validate the concept of operations, operational tests to validate performance, and development of an evaluation framework for biometric air exit. However, the report also cited significant remaining questions, such as those regarding the current performance of the biographic air exit process, the comparative value of adding a biometric capability to the existing air exit process, and the overall value and cost of a biometric air exit capability. As of April 2013, officials from DHS's Office of Policy and the Science and Technology Directorate stated that they are taking steps to implement the recommendations, including developmental scenario-based testing, as well as conducting outreach and soliciting information from airlines and airports regarding their operations. We will continue to evaluate the department's progress implementing our recommendation.