Secure Border Initiative:
Testing and Problem Resolution Challenges Put Delivery of Technology Program at Risk
GAO-10-511T: Published: Mar 18, 2010. Publicly Released: Mar 18, 2010.
This testimony is based on our report "Secure Border Initiative: DHS Needs to Address Testing and Performance Limitations That Place Key Technology Program at Risk." In September 2008, we reported to Congress that important aspects of SBInet were ambiguous and in a continuous state of flux, making it unclear and uncertain what technology capabilities were to be delivered when. In addition, the program did not have an approved integrated master schedule to guide the program's execution, and key milestones continued to slip. This schedule-related risk was exacerbated by the continuous change in and the absence of a clear definition of the approach used to define, develop, acquire, test, and deploy SBInet. Furthermore, different levels of SBInet requirements were not properly aligned, and all requirements had not been properly defined and validated. Also, the program office had not tested the individual system components to be deployed to initial locations, even though the contractor had initiated integration testing of these components with other system components and subsystems, and its test management strategy did not contain, among other things, a clear definition of testing roles and responsibilities; or sufficient detail to effectively guide planning for specific test events, such as milestones and metrics. Accordingly, we made recommendations to address these weaknesses which DHS largely agreed to implement. In light of SBInet's important mission, high cost, and risks, you asked us to conduct a series of four SBInet reviews. This statement and report being released today provide the results for the first of these reviews. Specifically, they address (1) the extent to which SBInet testing has been effectively managed, including identifying the types of tests performed and whether they were well planned and executed; (2) what the results of testing show; and (3) what processes are being used to test and incorporate maturing technologies into SBInet.
SBInet testing has not been adequately managed, as illustrated by poorly defined test plans and numerous and extensive last-minute changes to test procedures. Further, testing that has been performed identified a growing number of system performance and quality problems--a trend that is not indicative of a maturing system that is ready for deployment anytime soon. Further, while some of these problems have been significant, the collective magnitude of the problems is not clear because they have not been prioritized, user reactions to the system continue to raise concerns, and key test events remain to be conducted. Collectively, these limitations increase the risk that the system will ultimately not perform as expected and will take longer and cost more than necessary to implement. For DHS to increase its chances of delivering a version of SBInet for operational use, we are recommending that DHS improve the planning and execution of future test events and the resolution and disclosure of system problems. DHS agreed with our recommendations.