Arizona Border Surveillance Technology:

More Information on Plans and Costs Is Needed before Proceeding

GAO-12-22: Published: Nov 4, 2011. Publicly Released: Nov 4, 2011.

Additional Materials:

Contact:

Rebecca S. Gambler
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

In recent years, nearly half of all annual apprehensions of illegal aliens along the entire Southwest border with Mexico have occurred along the Arizona border. Keeping illegal flows of people and drugs under control remains a top priority for the Department of Homeland Security's (DHS) U.S. Customs and Border Protection (CBP). In 2005, the Secure Border Initiative Network (SBInet) was conceived as a surveillance technology to create a "virtual fence" along the border. After spending nearly $1 billion, DHS deployed SBInet systems along 53 miles of Arizona's border that represent the highest risk for illegal entry. In January 2011, in response to concerns regarding SBInet's performance, cost, and schedule, DHS cancelled future procurements. CBP developed the Arizona Border Surveillance Technology Plan (Plan) for the remainder of the Arizona border. Funding for this Plan for fiscal year 2012 is $242 million. GAO was requested to assess the extent to which CBP (1) has the information needed to support and implement the Plan and (2) estimated life-cycle costs for future investments in accordance with best practices. GAO analyzed Plan documents and cost estimates, compared those estimates with best practices, and interviewed CBP officials.

CBP does not have the information needed to fully support and implement its Arizona Border Surveillance Technology Plan in accordance with DHS and Office of Management and Budget (OMB) guidance. In developing the Plan, CBP conducted an analysis of alternatives and outreach to potential vendors. However, CBP has not documented the analysis justifying the specific types, quantities, and deployment locations of border surveillance technologies proposed in the Plan. Best practices for developing and managing costs indicate that a business case analysis should be rigorous enough that independent parties can review it and clearly understand why a particular alternative was chosen to support mission requirements. Without documentation of the analysis, there is no way to verify the process CBP followed, identify how the underlying analyses were used, assess the validity of the decisions made, or justify the funding requested for the Plan. CBP officials also have not yet defined the mission benefits expected from implementing the new Plan. GAO has previously reported that a solid business case providing an understanding of the potential return of large investments can be helpful to decision makers for determining whether continued investment is warranted after deployment. Defining the expected benefit could help improve CBP's ability to assess the effectiveness of the Plan as it is implemented. CBP does not intend to assess and address operational issues regarding the effectiveness and suitability of SBInet, steps that could provide CBP with information to help make decisions regarding alternatives for implementing the Plan. OMB guidance suggests that a post-implementation review occur when a system has been in operation for 6 months or immediately following investment termination. Such a review could help CBP make the most effective use of existing SBInet systems that, in connection with the Plan, could build a comprehensive and integrated approach for surveillance technology along the entire Arizona border. CBP's 10-year life-cycle cost estimate for the Plan of $1.5 billion is based on a rough order of magnitude analysis, and agency officials were unable to determine a level of confidence in their estimate as best practices suggest. Specifically, GAO's review of the estimate concluded that the estimate reflected substantial features of best practices, being both comprehensive and accurate, but it did not sufficiently meet other characteristics of a high-quality cost estimate, such as credibility, because it did not identify a level of confidence or quantify the impact of risks. GAO and OMB guidance emphasize that reliable cost estimates are important for program approval and continued receipt of annual funding. In addition, because CBP was unable to determine a level of confidence in its estimate, it will be difficult for CBP to determine what levels of contingency funding may be needed to cover risks associated with implementing new technologies along the remaining Arizona border. Thus, it will be difficult for CBP to provide reasonable assurance that its cost estimate is reliable and that its budget request for fiscal year 2012 and beyond is realistic and sufficient. A robust cost estimate--one that includes a level of confidence and quantifies the impact of risk--would help ensure that CBP's future technology deployments have sufficient funding levels related to the relative risks. GAO recommends that CBP document the analysis justifying the technologies proposed in the Plan, determine its mission benefits, conduct a post-implementation review of SBInet and determine a more robust life-cycle cost estimate for the Plan. DHS concurred with the recommendations.

Recommendations for Executive Action

  1. Status: Closed - Not Implemented

    Comments: DHS concurred with the recommendation but, subsequently, CBP officials stated that it is not planning further analyses or additional documentation given that officials consider their analyses to be sufficiently documented in the Arizona Border Surveillance Technology Plan (the Plan). As of March 2014, CBP had awarded 5 out of 7 technology contracts. For the remaining two technologies, CBP plans to award the contracts in July 2014 and in March 2015. As CBP has moved forward in awarding contracts for the Plan's technology programs and does not plan to conduct further analyses, we have closed this recommendation as not implemented.

    Recommendation: To increase the likelihood of successful implementation of the Arizona Border Surveillance Technology Plan and maximize the effectiveness of technology already deployed, the Commissioner of CBP should take the following step in planning the agency's new technology approach: ensure that the underlying analyses of the Plan are documented in accordance with DHS guidance and internal controls standards.

    Agency Affected: Department of Homeland Security: United States Customs and Border Protection

  2. Status: Open

    Comments: As GAO recommended in November 2011, CBP has identified mission benefits to be derived from implementing the Arizona Border Surveillance Technology Plan (Plan). In April 2013, CBP issued its Multi-Year Investment and Management Plan for Border Security Fencing, Infrastructure, and Technology for Fiscal Years 2014 through 2017, which identifies mission benefits to be achieved by all surveillance technologies (e.g., cameras or sensors) to be deployed under the Plan. According to CBP, the majority of these technologies will provide the mission benefits of improved situational awareness and agent safety. Furthermore, according to CBP's Multi-Year Investment and Management Plan for Border Security Fencing, Infrastructure, and Technology for Fiscal Years 2014 - 2017, the technologies deployed or planned for deployment as part of the Plan are intended to help enhance the ability of Border Patrol agents to detect, identify, deter, and respond to threats along the border. CBP's identification of mission benefits will help position CBP to assess its progress in implementing the Plan and the effectiveness of the Plan's technologies in achieving their intended goals. CBP has made some progress in identifying key attributes for metrics to assess implementation of the Arizona Border Surveillance Technology Plan (Plan), as GAO recommended in November 2011, but it has not yet fully identified and applied attributes for metrics for all technologies under the Plan. Since August 2010, CBP has operated multiple technology systems under the Secure Border Initiative Network (SBInet), which preceded the Plan and is a combination of surveillance technologies aimed at creating a "virtual fence" along the southwest border. Specifically, CBP has operated two surveillance systems under SBInet's initial deployment in high-priority regions of the Arizona border. In October 2012, CBP officials stated that these operations identified examples of key attributes for metrics that can be useful in assessing the implementation for technologies. For example, according to CBP, to help measure whether illegal activity has decreased, examples of key attributes include decreases in arrests, complaints by citizens and ranchers, and destruction of public and private lands and property. In November 2014, CBP identified a set of potential key attributes for performance metrics for all technologies to be deployed under the Plan. While CBP has yet to apply these measures, it established a timeline for developing performance measures for each technology. CBP initially expected baselines for each performance measure to be developed by the end of fiscal year 2015. However, in October 2015, CBP officials stated that CBP had modified its time frame for developing baselines and that additional time would be needed to implement and apply key attributes for metrics. In March 2016, CBP officials stated that CBP had planned to use the baseline data to establish a tool by the end of fiscal year 2016. In addition, CBP officials stated these performance measures would profile levels of situational awareness in various areas of the border. In September 2016, CBP provided GAO a case study that assessed technology assist data along with other measures such as field-based assessments of capability gaps, to determine the contributions of surveillance technologies to its mission. While this is a start to developing and applying performance metrics, the case study was limited to one border location and the analysis was limited to select technologies. Until CBP completes its efforts to fully develop and apply key attributes for performance metrics for all technologies to be deployed under the Plan, it will likely not be able to fully assess its progress in implementing the Plan and determine when mission benefits have been fully realized.

    Recommendation: To increase the likelihood of successful implementation of the Arizona Border Surveillance Technology Plan and maximize the effectiveness of technology already deployed, the Commissioner of CBP should take the following step in planning the agency's new technology approach: determine the mission benefits to be derived from implementation of the plan and develop and apply key attributes for metrics to assess program implementation.

    Agency Affected: Department of Homeland Security: United States Customs and Border Protection

  3. Status: Closed - Implemented

    Comments: In January 2013, CBP released the results of its Post Implementation Review (PIR) and operational assessment for SBInet Block I. The PIR resulted in recommendations to improve the operational effectiveness of the SBInet systems. Additionally, the PIR included information on CBP's efforts to address to recommendations made by the Army Test and Evaluation Command (ATEC). The PIR noted that CBP had yet to address three of the nine recommendations in the ATEC report. In July 2015, CBP stated that the ATEC findings/observations were reviewed but superseded by the PIR, which was determined to be more current and applicable to the SBInet Block I deployment plan. Accordingly, CBP decided that no action would be taken on the ATEC findings/observations. In addition, regarding CBP's efforts to address the recommendations outlined in the PIR, in July 2015, CBP provided GAO with specific details on the agency's efforts to address the recommendations and actionable items in the PIR. CBP provided documentation of its assessment and review of each recommendation in the PIR as well as the associated cost and benefits of each recommendation. Specifically, in September 2013, USBP and OTIA conducted an evaluation to determine the merits of the six PIR recommendations and 12 actionable items. CBP concurred with four actionable items and non-concurred with eight items. In July 2015, CBP officials told us that no further action will be taken to address the PIR recommendations and provided documentation of its assessment of the costs and benefits of addressing the recommendations in PIR. Based on GAO's review of the documentation provided, CBP's actions meet the intent of this recommendation.

    Recommendation: To increase the likelihood of successful implementation of the Arizona Border Surveillance Technology Plan and maximize the effectiveness of technology already deployed, the Commissioner of CBP should take the following step in planning the agency's new technology approach: conduct a post-implementation review and operational assessment of SBInet, including consideration of the Army Test and Evaluation Command's (ATEC) test results, and assess the costs and benefits of addressing the issues identified to help ensure the security of the 53 miles already covered by SBInet and enhance security on the Arizona border.

    Agency Affected: Department of Homeland Security: United States Customs and Border Protection

  4. Status: Open

    Comments: As GAO recommended in 2011, CBP provided cost estimates for the IFT and RVSS programs, the two highest cost programs in the Arizona Border Surveillance Technology Plan (Plan), in February and March 2012, respectively, and updated the cost estimate for the Plan in June 2013. However, these estimates do not fully meet cost-estimating best practices. In November 2011, GAO reported that the Plan's original cost estimate met some, but not all, cost-estimating best practices. Specifically, CBP had not conducted a sensitivity analysis and a risk and uncertainty analysis to determine a level of confidence in the original estimate, nor did CBP compare the original cost estimate with an independent estimate. For the cost estimate that CBP provided for the IFT in February 2012 and RVSS in March 2012, CBP partially documented the data used in the cost model for the IFT's LCCE (but needs to provide additional data and document management approval) and fully documented the cost model for the RVSS' LCCE. Developing a well-documented cost estimate is a best practice. CBP also conducted a sensitivity analysis and risk and uncertainty analysis to determine the level of confidence in both LCCEs so that contingency funding could be established relative to quantified risk. In addition, CBP's June 2013, CBP revised the cost estimate for the Plan does not fully address our concerns. For example, the IFT and RVSS compose over 90 percent of the Plan's cost in the June 2013 cost estimate; however, CBP has not independently verified its cost estimates for these two programs with independent cost estimates and reconciled any differences. Such action would help CBP better ensure the reliability of each system's cost estimate. Furthermore, the remainder of the June 2013 cost estimate is not fully documented for the Plan's other five programs, consistent with best practices. For example, the estimates for the other five programs are not fully documented because they are provided as summary program costs without detailed descriptions, such as including back-up documentation for labor hours. In November 2015, CBP had yet to update its LCCEs for two of its highest cost programs under the plan. In May 2016, CBP officials stated that the DHS's Cost Analysis Division had started piloting DHS's Independent Cost Estimate capability on the RVSS program in fiscal year 2016. According to CBP officials, this pilot test within CBP is an opportunity to assist DHS in developing its Independent Cost Estimate capability and that CBP selected the RVSS program for the pilot because the RVSS program is at a point in its planning and execution process where it can benefit most from having an independent cost estimate performed as these technologies are being deployed along the southwest border, beyond Arizona. As of October 2016, CBP officials stated that the RVSS schedule and analysis, as well as the results of the independent program cost estimate pilot is expected to be completed at the end of fiscal year 2017. CBP officials stated that they will provide information on the final reconciliation of the independent cost estimate and the RVSS program cost estimate once the pilot has been completed. Further, CBP officials have yet to detail similar plans for the IFT program. As updated life-cycle cost estimates have yet to be completed and independent cost estimates have not been conducted, GAO cannot determine the extent to which the agency is following best practices when updating the life-cycle cost estimates. Moreover, to fully address our recommendation, a LCCE for the Plan that fully addresses best practices is needed to ensure that the estimate is comprehensive, accurate, well-documented, and credible to help the agency and Congress fully understand the impacts of integrating the Plan's various programs.

    Recommendation: To increase the reliability of CBP's Cost Estimate for the Arizona Border Surveillance Technology Plan, the Commissioner of CBP should update its cost estimate for the Plan using best practices, so that the estimate is comprehensive, accurate, well-documented, and credible. Specifically, the CBP's Office of Technology Innovation and Acquisition (OTIA) program office should (1) fully document data used in the cost model; (2) conduct a sensitivity analysis and risk and uncertainty analysis to determine a level of confidence in the estimate so that contingency funding can be established relative to quantified risk; and (3) independently verify the new life-cycle cost estimate with an independent cost estimate and reconcile any differences.

    Agency Affected: Department of Homeland Security: United States Customs and Border Protection

 

Explore the full database of GAO's Open Recommendations »

Jul 20, 2017

Jul 18, 2017

Jun 29, 2017

Jun 7, 2017

Jun 1, 2017

May 18, 2017

May 3, 2017

May 1, 2017

Apr 13, 2017

Looking for more? Browse all our products here