The Department of Homeland Security is required by law to report annually on 43 specific measures of border security effectiveness.
DHS reported on 35 of those metrics in its 2017 report and generally used data and methods that should produce reliable results.
However the report disclosed some, but not all, of the data's limitations. For example, the report didn't indicate that different locations may have recorded some border-related data differently. Complete information about the data would provide context for Congress and the public to fully and appropriately understand the report.
We recommended ways to help DHS improve report quality.
Average wait time at land ports of entry is one of the 43 metrics.
Photo of vehicles lined up at a border crossing.
What GAO Found
The Department of Homeland Security (DHS) reported on 35 of 43 metrics called for by the National Defense Authorization Act (NDAA) for Fiscal Year 2017 (see figure); it generally used quality information, but did not identify some data limitations. GAO found that about half of the 35 metrics generally included elements as called for by the NDAA, while 17 metrics differed, such as in scope or calculation. For example, DHS only provided information on the southwest border for some metrics, such as the estimate of undetected unlawful border crossers for which a methodology for estimating unlawful crossings for the northern border had not yet been completed. DHS components responsible for collecting the metric data generally have processes in place to ensure the reliability of the data and the quality of the information provided. DHS also identified and disclosed limitations for some, but not all, of the data elements and metrics used. For example, GAO found that DHS did not disclose limitations on data related to apprehensions of individuals that were assisted by unmanned aerial systems. By developing and implementing a process to systematically review the reliability of the data and comprehensively identify and communicate limitations, DHS would improve the quality of the information provided.
Number of Metrics Included in the Department of Homeland Security's (DHS) Fiscal Year 2017 Border Security Metrics Report
DHS used a statistical model to estimate three metrics on unlawful border entries but did not validate some assumptions the model employs through sensitivity analyses and provide measures of statistical uncertainty in accordance with standards for federal agencies. For example, DHS's model assumes that 100 percent of families unlawfully crossing the border will be apprehended, but DHS did not provide information on the extent to which the assumption affected its metrics. DHS also did not provide information on the level of statistical uncertainty for the metrics, such as margins of error. Providing such information would allow Congress and the public to better understand the potential limitations and accuracy of these metrics of unlawful entry. Additionally, DHS's statistical model, which is based on Mexican adults not seeking asylum, represents a small and declining share of those apprehended at the border and DHS is developing a new model to account for current border conditions.
Why GAO Did This Study
According to DHS, the United States has approximately 6,000 miles of land borders, 95,000 miles of coastline, and more than 300 ports of entry where travelers and cargo are inspected and processed for entry. Securing U.S. border areas is a key part of DHS's mission, and the department's ability to measure its border security efforts is essential for it to manage its responsibilities effectively and efficiently.
The NDAA for Fiscal Year 2017 requires DHS to report annually on 43 border security metrics. DHS issued its first report in May 2018. The Act also includes a provision for GAO, within 270 days of receipt and biennially for the following 10 years, to review and report on the data and methodology contained in DHS's report. This report assesses the extent to which DHS: (1) reported metrics as outlined in the NDAA using quality information; and (2) validated assumptions and conveyed statistical uncertainty for unlawful entry metrics, among other objectives. GAO assessed the methodology and data in DHS's report, analyzed DHS's use of statistical models, and interviewed officials from DHS offices and components involved in developing the metrics.
GAO is making four recommendations, including that DHS develop and implement a process to systematically review the reliability of metric data, identify and communicate limitations of the metrics, and include the results of sensitivity analyses and measures of statistical uncertainty for metrics derived from statistical models. DHS concurred with the recommendations.
Recommendations for Executive Action
|Department of Homeland Security||1. The Secretary of Homeland Security should develop and implement a process to systematically review the reliability of the data used in its <i>Border Security Metrics Report</i> and comprehensively identify any limitations with the data and methodologies that underlie its metrics. (Recommendation 1)|
|Department of Homeland Security||2. The Secretary of Homeland Security should ensure the communication of the limitations of the metrics identified through the systematic review in the department's annual <i>Border Security Metrics Report</i>. (Recommendation 2)|
|Department of Homeland Security||3. The Under Secretary for the Office of Strategy, Policy, and Plans should include the results of sensitivity analyses to key assumptions in its statistical models of unlawful entry estimates in its annual <i>Border Security Metrics Report</i>. (Recommendation 3)|
|Department of Homeland Security||4. The Under Secretary for the Office of Strategy, Policy, and Plans should include measures of statistical uncertainty for all metrics based on estimates derived from statistical models in its annual <i>Border Security Metrics Report</i>. (Recommendation 4)|