Border Security:

DHS Should Improve the Quality of Unlawful Border Entry Information and Other Metric Reporting

GAO-19-305: Published: Mar 21, 2019. Publicly Released: Mar 21, 2019.

Additional Materials:

Contact:

Rebecca Gambler
(202) 512-8777
gamblerr@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The Department of Homeland Security is required by law to report annually on 43 specific measures of border security effectiveness.

DHS reported on 35 of those metrics in its 2017 report and generally used data and methods that should produce reliable results.

However the report disclosed some, but not all, of the data's limitations. For example, the report didn't indicate that different locations may have recorded some border-related data differently. Complete information about the data would provide context for Congress and the public to fully and appropriately understand the report.

We recommended ways to help DHS improve report quality.

Average wait time at land ports of entry is one of the 43 metrics.

Photo of vehicles lined up at a border crossing.

Photo of vehicles lined up at a border crossing.

Additional Materials:

Contact:

Rebecca Gambler
(202) 512-8777
gamblerr@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Department of Homeland Security (DHS) reported on 35 of 43 metrics called for by the National Defense Authorization Act (NDAA) for Fiscal Year 2017 (see figure); it generally used quality information, but did not identify some data limitations. GAO found that about half of the 35 metrics generally included elements as called for by the NDAA, while 17 metrics differed, such as in scope or calculation. For example, DHS only provided information on the southwest border for some metrics, such as the estimate of undetected unlawful border crossers for which a methodology for estimating unlawful crossings for the northern border had not yet been completed. DHS components responsible for collecting the metric data generally have processes in place to ensure the reliability of the data and the quality of the information provided. DHS also identified and disclosed limitations for some, but not all, of the data elements and metrics used. For example, GAO found that DHS did not disclose limitations on data related to apprehensions of individuals that were assisted by unmanned aerial systems. By developing and implementing a process to systematically review the reliability of the data and comprehensively identify and communicate limitations, DHS would improve the quality of the information provided.

Number of Metrics Included in the Department of Homeland Security's (DHS) Fiscal Year 2017 Border Security Metrics Report

HL_5 - 102724

DHS used a statistical model to estimate three metrics on unlawful border entries but did not validate some assumptions the model employs through sensitivity analyses and provide measures of statistical uncertainty in accordance with standards for federal agencies. For example, DHS's model assumes that 100 percent of families unlawfully crossing the border will be apprehended, but DHS did not provide information on the extent to which the assumption affected its metrics. DHS also did not provide information on the level of statistical uncertainty for the metrics, such as margins of error. Providing such information would allow Congress and the public to better understand the potential limitations and accuracy of these metrics of unlawful entry. Additionally, DHS's statistical model, which is based on Mexican adults not seeking asylum, represents a small and declining share of those apprehended at the border and DHS is developing a new model to account for current border conditions.

Why GAO Did This Study

According to DHS, the United States has approximately 6,000 miles of land borders, 95,000 miles of coastline, and more than 300 ports of entry where travelers and cargo are inspected and processed for entry. Securing U.S. border areas is a key part of DHS's mission, and the department's ability to measure its border security efforts is essential for it to manage its responsibilities effectively and efficiently.

The NDAA for Fiscal Year 2017 requires DHS to report annually on 43 border security metrics. DHS issued its first report in May 2018. The Act also includes a provision for GAO, within 270 days of receipt and biennially for the following 10 years, to review and report on the data and methodology contained in DHS's report. This report assesses the extent to which DHS: (1) reported metrics as outlined in the NDAA using quality information; and (2) validated assumptions and conveyed statistical uncertainty for unlawful entry metrics, among other objectives. GAO assessed the methodology and data in DHS's report, analyzed DHS's use of statistical models, and interviewed officials from DHS offices and components involved in developing the metrics.

What GAO Recommends

GAO is making four recommendations, including that DHS develop and implement a process to systematically review the reliability of metric data, identify and communicate limitations of the metrics, and include the results of sensitivity analyses and measures of statistical uncertainty for metrics derived from statistical models. DHS concurred with the recommendations.

For more information, contact Rebecca Gambler at or gamblerr@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: DHS concurred with the recommendation and in a May 2020 update stated that it plans to address this recommendation in its FY2019 Border Security Metrics Report scheduled to be issued in the summer of 2020. We will continue to monitor DHS's ongoing efforts to do so.

    Recommendation: The Secretary of Homeland Security should develop and implement a process to systematically review the reliability of the data used in its Border Security Metrics Report and comprehensively identify any limitations with the data and methodologies that underlie its metrics. (Recommendation 1)

    Agency Affected: Department of Homeland Security

  2. Status: Open

    Comments: In its comment letter, DHS concurred with the recommendation and requested that we consider it closed as implemented because the department already detailed some of the limitations in its fiscal year 2017 report, and plans to continue to identify known limitations and the progress made to mitigate previously identified limitations in future reports. As discussed in the report, we agree that DHS identified and disclosed limitations for some metrics in its fiscal year 2017 Border Security Metrics Report; however, we identified at least one additional limitation for 21 of the 35 metrics on which DHS reported that DHS did not disclose or about which it could have been more transparent. To address the intent of this recommendation, once DHS has implemented a process to systematically review the reliability of the data used in its report and comprehensively identified related limitations, it should disclose those limitations in its annual Border Security Metrics Report. In a May 2020 update, DHS stated that it plans to address this recommendation in its FY2019 Border Security Metrics Report scheduled to be issued in the summer of 2020. We will continue to monitor DHS's ongoing efforts to do so.

    Recommendation: The Secretary of Homeland Security should ensure the communication of the limitations of the metrics identified through the systematic review in the department's annual Border Security Metrics Report. (Recommendation 2)

    Agency Affected: Department of Homeland Security

  3. Status: Open

    Comments: DHS concurred with the recommendation and in a May 2020 update stated that it plans to address this recommendation in its FY2019 Border Security Metrics Report scheduled to be issued in the summer of 2020. We will continue to monitor DHS's ongoing efforts to do so.

    Recommendation: The Under Secretary for the Office of Strategy, Policy, and Plans should include the results of sensitivity analyses to key assumptions in its statistical models of unlawful entry estimates in its annual Border Security Metrics Report. (Recommendation 3)

    Agency Affected: Department of Homeland Security

  4. Status: Open

    Comments: DHS concurred with the recommendation and in a May 2020 update stated that it plans to address this recommendation in its FY2019 Border Security Metrics Report scheduled to be issued in the summer of 2020. We will continue to monitor DHS's ongoing efforts to do so.

    Recommendation: The Under Secretary for the Office of Strategy, Policy, and Plans should include measures of statistical uncertainty for all metrics based on estimates derived from statistical models in its annual Border Security Metrics Report. (Recommendation 4)

    Agency Affected: Department of Homeland Security

 

Explore the full database of GAO's Open Recommendations »

Feb 26, 2021

Feb 25, 2021

Feb 23, 2021

Feb 19, 2021

Feb 12, 2021

Feb 3, 2021

Feb 2, 2021

Jan 28, 2021

Jan 25, 2021

Jan 21, 2021

Looking for more? Browse all our products here