Why Area Is Important

The federal government allocates billions of dollars for researching, developing, and testing technologies and other countermeasures to address chemical, biological, radiological, nuclear, and other threats facing the nation. The Department of Homeland Security's (DHS) Science and Technology Directorate (S&T) conducts research and development efforts to improve homeland security by, among other things, providing its federal, state, local, tribal, and territorial emergency responder customers with technology to help them achieve their missions. DHS's Domestic Nuclear Detection Office (DNDO) is charged with developing, acquiring, and deploying equipment to detect nuclear and radiological materials, supporting the efforts of DHS and other federal agencies. According to DHS documents, the total budget authority for S&T and DNDO was over $5.8 billion for fiscal years 2007 through 2010.[1] DHS has experienced challenges in managing its multibillion dollar research and development efforts, and GAO has identified problems with its testing and cost-benefit analyses efforts in this area.

What GAO Found

In managing its multibillion dollar research and development efforts, DHS has experienced cost overruns and delays in the procurement and deployment of technologies and systems needed to meet critical homeland security needs. DHS could help reduce inefficiencies and costs in its research and development program by completing testing efforts before making acquisition decisions and including cost-benefit analyses in its research and development efforts.

DHS has made acquisition decisions without completing testing efforts to ensure that the systems purchased met program requirements. GAO's prior work has shown that failure to resolve problems discovered during testing can sometimes lead to costly redesign and rework at a later date. Addressing such problems during the testing phase before moving to the acquisition phase can help agencies avoid future cost overruns.

• In September 2010, GAO reported that DNDO was simultaneously engaged in the research and development phase while planning for the acquisition phase of its cargo advanced automated radiography system to detect certain nuclear materials in vehicles and containers at ports. DNDO pursued the deployment of the cargo advanced automated radiography system without fully understanding that it would not fit within existing inspection lanes at ports of entry and would slow down the flow of commerce through these lanes, causing significant delays. DHS spent $113 million on the program since 2005. DHS cancelled the acquisition phase of the program in 2007.
• In June 2010, GAO reported that three Coast Guard programs—the Maritime Patrol Aircraft, Response Boat-Medium, and Sentinel Class Patrol Boat—placed orders for or received significant numbers of units prior to completing testing, placing the Coast Guard at risk for needing to make expensive changes to the design of these vessels after production had begun if significant problems were identified during testing. Acquisition cost estimates for these three programs together totaled about $6.8 billion, according to Coast Guard data.
• In October 2009, GAO reported that the Transportation Security Administration (TSA), within DHS, deployed explosives trace portals, a technology for detecting traces of explosives on passengers at airport checkpoints, even though TSA officials were aware that tests conducted during 2004 and 2005 on earlier models of the portals suggested the portals did not demonstrate reliable performance in an airport environment. TSA also lacked assurance that the portals would meet functional requirements in airports within estimated costs. In June 2006, TSA halted deployment of the explosives trace portals because of performance problems, and the machines were more expensive to install and maintain than expected. GAO recommended that TSA ensure that tests are completed before deploying checkpoint screening technologies to airports. The agency concurred with the recommendation and has taken action to address it. For example, TSA has required more recent passenger checkpoint technologies to complete both laboratory tests and operational tests prior to their deployment.

In addition, GAO's prior work has shown that cost-benefit analyses help congressional and agency decision makers assess and prioritize resource investments and consider potentially more cost-effective alternatives. However, DHS has not included cost-benefit analyses in its testing efforts and acquisition decision making.

• In 2006, GAO recommended that DHS's decision to deploy next generation radiation detection equipment, or advanced spectroscopic portals, used to detect smuggled nuclear or radiological materials, be based on an analysis of both the benefits and costs—which GAO later estimated at over $2 billion—and a determination of whether any additional detection capability provided by the portals was worth their additional cost. DHS subsequently issued a cost-benefit analysis, but GAO reported that this analysis did not provide a sound analytical basis for DHS's decision to deploy the portals. In June 2009 GAO reported that an updated cost-benefit analysis might show that DNDO's plan to replace existing equipment with advanced spectroscopic portals was not justified, particularly given the marginal improvement in detection of certain nuclear materials required of advanced spectroscopic portals and the potential to improve the current-generation portal monitors' sensitivity to nuclear materials, most likely at a lower cost. After spending more than $200 million on the program, in February 2010 DHS announced that it was scaling back its plans for development and use of the portals technology.
• In October 2009 GAO reported that TSA had not yet completed a cost-benefit analysis to prioritize and fund its technology investments for screening passengers at airport checkpoints. One reason that TSA had difficulty developing a cost-benefit analysis was that it had not yet developed lifecycle cost estimates for its various screening technologies. GAO reported that this information was important because it would help decision makers determine, given the cost of various technologies, which technology provided the greatest mitigation of risk for the resources that were available. GAO recommended that TSA develop a cost-benefit analysis. The agency has completed a lifecycle cost estimate and collected information for its checkpoint technologies, but has not yet completed any cost-benefit analysis.

In January 2011, DHS reported that it plans to take additional actions to strengthen its research and development efforts. For example, DHS reported that it plans to establish a new model for managing departmentwide investments across their life cycles. DHS reported that S&T will be involved in each phase of the investment life cycle and will participate in new entities DHS is planning to create to help ensure that test and evaluation methods are appropriately considered as part of DHS's overall research and development investment strategies. According to DHS, S&T will help ensure that new technologies are properly scoped, developed, and tested before being implemented. In addition, DHS reported that the new entities it is planning to establish to strengthen management of the department's acquisition and investment review process will be responsible for, among other things, making decisions on research and development initiatives based on factors such as viability and affordability, and overseeing key acquisition decisions for major programs using baseline and actual data.

Actions Needed

GAO's work has highlighted the need for the department to strengthen its research and development efforts by ensuring that (1) testing efforts are completed before making acquisition decisions and (2) cost-benefit analyses are conducted to reduce research and development inefficiencies and costs. The planned actions DHS reports it is taking or has under way to address management of its research and development programs are positive steps and, if implemented effectively, could help the department address many of these challenges. However, it is too early to fully assess the impact of these actions.

GAO has reported that DHS could take further actions to improve its management of research and development efforts and reduce costs in procuring and deploying programs that have not been fully tested. For example, rigorously testing devices using actual agency operational tactics before making decisions on acquisition would help DHS reduce inefficiencies and costs. Further, conducting cost-benefit analyses as part of research, development, and testing efforts would help DHS and congressional decision makers better assess and prioritize investment decisions, including assessing possible program alternatives that could be more cost-effective.

Framework for Analysis

The information contained in this analysis is based on the related GAO products listed under the "Related GAO Products" tab. GAO has ongoing work for the Senate Committee on Homeland Security and Governmental Affairs reviewing the role that S&T has in conducting testing and evaluation of major acquisitions programs prior to implementation.

Area Contact

For additional information about this area, contact Diana Maurer at (202) 512-9627 or maurerd@gao.gov.

Related GAO Products