Summary
The Department of Homeland Security's (DHS) Domestic Nuclear Detection Office (DNDO) is testing new advanced spectroscopic portal (ASP) radiation detection monitors. DNDO expects ASPs to reduce both the risk of missed threats and the rate of innocent alarms, which DNDO considers to be key limitations of radiation detection equipment currently used by Customs and Border Protection (CBP) at U.S. ports of entry. Congress has required that the Secretary of DHS certify that ASPs provide a significant increase in operational effectiveness before obligating funds for full-scale procurement. GAO was asked to review (1) the degree to which DHS's criteria for a significant increase in operational effectiveness address the limitations of existing radiation detection equipment, (2) the rigor of ASP testing and preliminary test results, and (3) the ASP test schedule. GAO reviewed the DHS criteria, analyzed test plans, and interviewed DHS officials.
The DHS criteria for a significant increase in operational effectiveness require a minimal improvement in the detection of threats and a large reduction in innocent alarms. Specifically, the criteria require a marginal improvement in the detection of certain weapons-usable nuclear materials, considered to be a key limitation of current-generation portal monitors. The criteria require improved performance over the current detection threshold, which for certain nuclear materials is based on the equipment's limited sensitivity to anything more than lightly shielded materials, but do not specify a level of shielding that smugglers could realistically use. In addition, DNDO has not completed efforts to improve current-generation portal monitors' performance. As a result, the criteria do not take the current equipment's full potential into account. With regard to innocent alarms, the other key limitation of current equipment, meeting the criteria could result in hundreds fewer innocent alarms per day, thereby reducing CBP's workload and delays to commerce. DHS increased the rigor of ASP testing in comparison with previous tests. For example, DNDO mitigated the potential for bias in performance testing (a concern GAO raised about prior testing) by stipulating that there would be no ASP contractor involvement in test execution. Such improvements added credibility to the test results. However, the testing still had limitations, such as a limited set of scenarios used in performance testing to conceal test objects from detection. Moreover, the preliminary results are mixed. The results show that the new portal monitors have a limited ability to detect certain nuclear materials at anything more than light shielding levels: ASPs performed better than current-generation portal monitors in detection of such materials concealed by light shielding approximating the threat guidance for setting detection thresholds, but differences in sensitivity were less notable when shielding was slightly below or above that level. Testing also uncovered multiple problems in ASPs meeting the requirements for successful integration into operations at ports of entry. CBP officials anticipate that, if ASPs are certified, new problems will appear during the first few years of deployment in the field. While DNDO's schedule underestimated the time needed for ASP testing, test delays have allowed more time for review and analysis of results. DNDO's original schedule anticipated completion in September 2008. Problems uncovered during testing of ASPs' readiness to be integrated into operations at U.S. ports of entry caused the greatest delays to this schedule. DHS's most recent schedule anticipated a decision on ASP certification as early as May 2009, but DHS recently suspended field validation due to ASP performance problems and has not updated its schedule for testing and certification. In any case, DNDO does not plan to complete computer simulations that could provide additional insight into ASP capabilities and limitations prior to certification even though delays have allowed more time to conduct the simulations. DNDO officials believe the other tests are sufficient for ASPs to demonstrate a significant increase in operational effectiveness.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director:
Team:
Phone:
Eugene E. Aloise
Government Accountability Office: Natural Resources and Environment
(202) 512-6870
Recommendations for Executive Action
Recommendation: The Secretary of Homeland Security should direct the Director of DNDO to assess whether ASPs meet the criteria for a significant increase in operational effectiveness based on a valid comparison with PVTs' full performance potential, including the potential to further develop PVTs' use of energy windowing to provide greater sensitivity to threats to ensure a sound basis for a decision on ASP certification. Such a comparison could also be factored into an updated cost-benefit analysis to determine whether it would be more cost-effective to continue to use PVTs or deploy ASPs for primary screening at particular ports of entry.
Agency Affected: Department of Homeland Security
Status: In process
Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
Recommendation: The Secretary of Homeland Security should direct the Director of DNDO to revise the schedule for ASP testing and certification to allow sufficient time for review and analysis of results from the final phases of testing and completion of all tests, including injection studies to ensure a sound basis for a decision on ASP certification.
Agency Affected: Department of Homeland Security
Status: In process
Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
Recommendation: If ASPs are certified, the Secretary of Homeland Security should direct the Director of DNDO to develop an initial deployment plan that allows CBP to uncover and resolve any additional problems not identified through testing before proceeding to full-scale deployment--for example, by initially deploying ASPs at a limited number of ports of entry
Agency Affected: Department of Homeland Security
Status: In process
Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
