This is the accessible text file for GAO report number GAO-12-239T 
entitled 'DHS Research And Development: Science and Technology 
Directorate's Test and Evaluation and Reorganization Efforts' which 
was released on November 17, 2011. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office:

Testimony Before the Subcommittee on Cybersecurity, Infrastructure 
Protection, and Security Technologies, Committee on Homeland Security, 
House of Representatives: 

For Release on Delivery: 
Expected at 10:00 a.m. EST:
November 17, 2011: 

DHS Research And Development: 

Science and Technology Directorate's Test and Evaluation and 
Reorganization Efforts: 

Statement of David C. Maurer, Director:
Homeland Security and Justice Issues: 


Chairman Lungren, Ranking Member Clarke, and Members of the 

I am pleased to be here today to discuss our prior work examining the 
Department of Homeland Security's (DHS) Science and Technology 
Directorate (S&T) and Research and Development (R&D) efforts. The 
Homeland Security Act of 2002 created DHS and, within it, established 
S&T with the responsibility for conducting national research, 
development, test and evaluation (T&E) of technology and systems for, 
among other things, detecting, preventing, protecting against, and 
responding to terrorist attacks.[Footnote 1] Since its creation in 
2003, DHS, through both S&T and its components, has spent billions of 
dollars researching and developing technologies used to support a wide 
range of missions including securing the border, detecting nuclear 
devices, and screening airline passengers and baggage for explosives, 
among others. S&T has a wide-ranging mission, which includes 
conducting basic and applied research of technologies,[Footnote 2] and 
overseeing the testing and evaluation of component acquisitions and 
technologies to ensure that they meet DHS acquisition requirements 
before implementation in the field.[Footnote 3] In recent years, we 
have reported that DHS has experienced challenges in managing its 
multibillion-dollar technology development and acquisition efforts, 
including implementing technologies that did not meet intended 
requirements and were not appropriately tested and evaluated. These 
problems highlight the important role that S&T plays in overseeing DHS 
testing and evaluation. 

S&T has reorganized to better achieve its goals and provide better 
assistance to DHS components in developing technologies. In addition 
to the challenge of implementing its varied mission, S&T is also 
managing a decline in available R&D resources. S&T's fiscal year 2011 
appropriation decreased 20 percent from fiscal year 2010 and, while 
its fiscal year 2012 appropriation has not yet been enacted, both the 
House and Senate marks for the agency are lower than what was 
appropriated in fiscal year 2011.[Footnote 4] As a result, S&T has had 
to adjust resources and re-prioritize its efforts. In the past, we 
have reported on issues related to the transformation and 
reorganization of R&D efforts in the federal government, particularly 
related to shifting of priorities and managing a reduction in 
resources.[Footnote 5] In addition, we identified DHS R&D as an area 
for potential costs savings in our March 2011 report regarding 
opportunities to reduce potential duplication in government programs, 
save tax dollars, and enhance revenue.[Footnote 6] Specifically, we 
reported that DHS could take further actions to improve its management 
of R&D and reduce costs by ensuring that testing efforts are completed 
before making acquisition decisions and cost-benefit analyses are 
conducted to reduce R&D inefficiencies and costs. 

My testimony today focuses on the key findings from our prior work 
related to S&T's test and evaluation efforts, S&T's recent 
reorganization efforts, and key findings from our past work related to 
federal R&D. Specifically, this statement will address: 

* the extent to which S&T oversees T&E of major DHS acquisitions and 
what challenges, if any, S&T officials report facing in overseeing T&E 
across DHS; and: 

* S&T's recent reorganization efforts and how key findings from our 
prior work on R&D in the federal government can inform how S&T moves 

This statement is based on reports and testimonies we issued from 
March 1995 to July 2011 related to DHS's efforts to manage, test, and 
deploy various technology programs; transformation of federal R&D; and 
selected updates conducted from July 2011 to the present related to 
S&T's reorganization efforts.[Footnote 7] For the updates, we reviewed 
recent S&T testimonies and documentation related to the reorganization 
as well as information on annual S&T appropriations and budget 
requests from fiscal years 2009 to 2012. For our past work, we 
reviewed DHS directives and testing plans, interviewed DHS, Department 
of Energy, Department of Defense, Environmental Protection Agency, and 
other agency officials, reviewed documentation from these agencies, 
visited laboratory facilities, and examined agency databases, among 
other things. We conducted this work in accordance with generally 
accepted government auditing standards. More detailed information on 
the scope and methodology from our previous work can be found within 
each specific report. 

S&T Could Take Additional Steps to Ensure that DHS T&E Requirements 
Are Met; Officials Cited Challenges to Overseeing T&E across DHS: 

S&T Oversight of DHS Testing and Evaluation: 

In June 2011, we reported that S&T met some of its oversight 
requirements for T&E of acquisition programs we reviewed, but 
additional steps were needed to ensure that all requirements were met. 
[Footnote 8] Specifically, since DHS issued the T&E directive in May 
2009, S&T reviewed or approved T&E documents and plans for programs 
undergoing testing, and conducted independent assessments for the 
programs that completed operational testing during this time period. 
S&T officials told us that they also provided input and reviewed other 
T&E documentation, such as components' documents describing the 
programs' performance requirements, as required by the T&E directive. 
DHS senior level officials considered S&T's T&E assessments and input 
in deciding whether programs were ready to proceed to the next 
acquisition phase. However, S&T did not consistently document its 
review and approval of components' test agents--a government entity or 
independent contractor carrying out independent operational testing 
for a major acquisition--or document its review of other component 
acquisition documents, such as those establishing programs' 
operational requirements, as required by the T&E directive. For 
example, 8 of the 11 acquisition programs we reviewed had hired test 
agents, but documentation of S&T approval of these agents existed for 
only 3 of these 8 programs. We reported that approving test agents is 
important to ensure that they are independent of the program and that 
they meet requirements of the T&E directive. 

S&T officials agreed that they did not have a mechanism in place 
requiring a consistent method for documenting their review or approval 
and the extent to which the review or approval criteria were met. We 
reported that without mechanisms in place for documenting its review 
or approval of acquisition documents and T&E requirements, such as 
approving test agents, it is difficult for DHS or a third party to 
review and validate S&T's decision-making process and ensure that it 
is overseeing components' T&E efforts in accordance with acquisition 
and T&E directives and internal control standards for the federal 
government. As a result, we recommended that S&T develop a mechanism 
to document both its approval of operational test agents and component 
acquisitions documentation to ensure that these meet the requirements 
of the DHS T&E directive. S&T concurred and reported that the agency 
has since developed internal procedures to ensure that the approval of 
test agents and component acquisition documents are documented. 

Challenges in Coordinating and Overseeing T&E across DHS: 

We also reported in June 2011 that S&T and DHS component officials 
stated that they face challenges in overseeing T&E across DHS 
components which fell into 4 categories: (1) ensuring that a program's 
operational requirements--the key performance requirements that must 
be met for a program to achieve its intended goals--can be effectively 
tested; (2) working with DHS component program staff who have limited 
T&E expertise and experience; (3) using existing T&E directives and 
guidance to oversee complex information technology acquisitions; and 
(4) ensuring that components allow sufficient time for T&E while 
remaining within program cost and schedule estimates. 

Both S&T and DHS, more broadly, have begun initiatives to address some 
of these challenges, such as establishing a T&E council to disseminate 
best practices to component program managers, and developing specific 
guidance for testing and evaluating information technology 
acquisitions. In addition, as part of S&T's recent reorganization, the 
agency has developed a new division specifically geared toward 
assisting components in developing requirements that can be tested, 
among other things. However, since these efforts have only recently 
been initiated to address these DHS-wide challenges, it is too soon to 
determine their effectiveness. 

S&T Recently Reorganized and Our Prior R&D Work Could Inform How S&T 
Moves Forward: 

Since 2009, S&T has undertaken a series of efforts related to its 
organizational structure. S&T underwent a new strategic planning 
process, developed new strategic goals, and conducted a reorganization 
intended to better achieve its strategic goals. These efforts were 
implemented after a 2009 National Academy of Public Administration 
study found that S&T's organizational structure posed communication 
challenges across the agency and that the agency lacked a cohesive 
strategic plan and mechanisms to assess performance in a systematic 
way, among other things.[Footnote 9] In August 2010, S&T reorganized 
to align its structure with its top strategic goals, allow for easier 
interaction among senior leadership, and reduce the number of 
personnel directly reporting to the Under Secretary of S&T. 
Additionally, after the Under Secretary was confirmed in November 
2009, S&T instituted a new strategic planning process which helped 
inform the development of new strategic goals. The new strategic goals 
announced in August 2010 include: 

* rapidly developing and delivering knowledge, analyses, and 
innovative solutions that advance the mission of DHS; 

* leveraging its expertise to assist DHS components' efforts to 
establish operational requirements, and select and acquire needed 

* strengthening the Homeland Security Enterprise and First Responders' 
capabilities to protect the homeland and respond to disasters; 

* conducting, catalyzing, and surveying scientific discoveries and 
inventions relevant to existing and emerging homeland security 
challenges; and: 

* fostering a culture of innovation and learning in S&T and across DHS 
that addresses mission needs with scientific, analytic, and technical 

According to S&T, the agency has developed a draft strategic plan that 
provides its overall approach to meeting these strategic goals, which 
is currently in the process of being finalized. 

Moreover, according to testimony by the Undersecretary of S&T in March 
2011, to ensure that individual R&D projects are meeting their goals, 
S&T has committed to an annual review of its portfolio of basic and 
applied R&D and all proposed "new start" projects. According to S&T, 
the review process uses metrics determined by S&T, with input from DHS 
components, that are aligned with DHS priorities. These metrics 

* the impact on the customer's mission; 

* the ability to transition these products to the field; 

* whether the investment positions S&T for the future; 

* whether the projects are aligned with customer requirements; 

* whether S&T has the appropriate level of customer interaction; and: 

* whether S&T is sufficiently innovative in the way it is approaching 
its challenges. 

We are currently reviewing DHS and S&T's processes for prioritizing, 
coordinating, and measuring the results of its R&D efforts for the 
Senate Committee on Homeland Security and Governmental Affairs and we 
will report on this issue next year. 

Our prior work related to R&D at other federal agencies could provide 
insight for S&T as it moves forward with new structures and processes 
operating within potential fiscal constraints. During the 1990s, we 
issued a series of reports on federal efforts to restructure R&D in 
the wake of changing priorities and efforts to balance the federal 
budget. More recently, we have issued reports on R&D issues at the 
Department of Defense (DOD), Department of Energy (DOE), the 
Environmental Protection Agency (EPA), and DHS. Although the specific 
recommendations and issues vary from department to department, there 
are key findings across this body of work that could potentially help 
inform S&T's efforts to meet DHS's R&D needs, as well as Congressional 
oversight of these activities. Since our assessment of R&D efforts at 
DHS is currently under way, we have not determined the extent to which 
these key findings from our prior work are applicable to DHS's R&D 
efforts or the extent to which DHS already has similar efforts under 
way. However, our prior work could provide valuable insights into how 
DHS could leverage the private sector to help conduct R&D, restructure 
R&D efforts in response to fiscal constraints, and develop 
comprehensive strategies to mitigate the risk of duplication and 
overlap. For example: 

* We reported on federal agencies that have restructured their 
research and development efforts in response to fiscal constraints. 
For example, in January 1998, we reported on efforts by federal 
agencies, such as DOD, the DOE National Laboratories, and NASA, to 
streamline their R&D activities and infrastructure. We reported that 
restructuring research, development, testing and evaluation to meet 
current and future needs required interagency agreements and cross-
agency efforts, in addition to ongoing individual efforts.[Footnote 
10] Additionally, we reported on five elements that were useful in the 
successful restructuring of R&D in corporate and foreign government 
organizations. For example, we found that successful restructuring of 
R&D activities included having a core mission that supports overall 
goals and strategies, clear definitions of those responsible for 
supporting that mission, and accurate data on total costs of the 
organization's activities. 

* In addition, we have reported that comprehensive strategies mitigate 
risk of duplication and overlap.[Footnote 11] For example, we reported 
in March 2011 that DOD did not have a comprehensive approach to manage 
and oversee the breadth of its activities for developing new 
capabilities in response to urgent warfighter needs, including 
entities engaged in experimentation and rapid prototyping to 
accelerate the transition of technologies to the warfighter, and 
lacked visibility over the full range of its efforts.[Footnote 12] As 
a result, we recommended that DOD issue guidance that defined roles, 
responsibilities, and authorities across the department to lead its 
efforts. DOD agreed with this recommendation. 

* Within DHS itself, we reported in May 2004 that DHS did not have a 
strategic plan to guide its R&D efforts. We recommended that DHS 
complete a strategic R&D plan and ensure that the plan was integrated 
with homeland security R&D conducted by other federal agencies. 
[Footnote 13] We also recommended that DHS develop criteria for 
distributing annual funding and for making long-term investments in 
laboratory capabilities, as well as develop guidelines that detailed 
how DOE's laboratories would compete for funding with private sector 
and academic entities. DHS agreed with our recommendations. While S&T 
developed a 5-year R&D plan in 2008 to guide its efforts and is 
currently finalizing a new strategic plan to align its own R&D 
investments and goals, DHS has not yet completed a strategic plan to 
align all R&D efforts across the department, as we previously 

* Our work on DOE National Laboratories provides additional insights 
related to oversight of R&D efforts that could be useful for DHS S&T. 
In 1995, we reported that DOE's national laboratories did not have 
clearly defined missions focused on accomplishing DOE's changing 
objectives and national priorities.[Footnote 14] DOE, at that time, 
managed the national laboratories on a program by program basis which 
inhibited cooperation across programs and hindered DOE's ability to 
use the laboratories to meet departmental missions. We recommended, 
among other things, that DOE develop a strategy that maximized the 
laboratories' resources. In responding, DOE said that it had 
undertaken a new strategic planning process which resulted in a 
strategic plan. Though DOE developed a strategic plan intended to 
integrate its missions and programs, in 1998 we reported that the 
laboratories did not function as an integrated national research and 
development system and recommended that DOE develop a comprehensive 
strategy to be used to assess success in meeting objectives, monitor 
progress, and report on that progress.[Footnote 15] DOE acknowledged 
that it needed to better focus the laboratories' missions and tie them 
to the annual budget process, but that it would take time to 

* More recently, we reported in June 2009 that DOE could not determine 
the effectiveness of its laboratories' technology transfer efforts 
because it has not yet defined its overarching strategic goals for 
technology transfer and lacks reliable performance data.[Footnote 16] 
Instead, individual DOE programs such as the National Nuclear Security 
Administration and DOE's Office of Science articulated their own goals 
for technology transfer at the national laboratories. We recommended, 
among other things, that DOE articulate department wide priorities and 
develop clear goals, objectives, and performance measures. DOE 
generally agreed with our findings. 

* Lastly, our work on Environmental Protection Agency (EPA) laboratory 
facilities also offers insights into the importance of planning and 
coordination in managing R&D.[Footnote 17] Specifically, we reported 
in July 2011 that EPA has yet to fully address the findings of 
numerous past studies that have examined EPA's science activities. 
These past evaluations noted the need for EPA to improve long-term 
planning, priority setting, and coordination of laboratory activities, 
establish leadership for agency wide scientific oversight and decision 
making, and better manage the laboratories' workforce and 
infrastructure. We recommended, among other things, that EPA develop a 
coordinated planning process for its scientific activities and appoint 
a top-level official with authority over all the laboratories, improve 
physical and real property planning decisions, and develop a workforce 
planning process for all laboratories that reflects current and future 
needs of laboratory facilities. EPA generally agreed with our findings 
and recommendations. 

Chairman Lungren, Ranking Member Clarke, and Members of the 
Subcommittee, this concludes my prepared statement. I would be pleased 
to respond to any questions that you may have. 

GAO Contact and Staff Acknowledgments: 

For questions about this statement, please contact David C. Maurer at 
(202) 512-9627 or Contact points for our Offices of 
Congressional Relations and Public Affairs may be found on the last 
page of this statement. Individuals making key contributions to this 
statement include Chris Currie, Assistant Director; Emily Gunn and 
Margaret McKenna. Key contributors for the previous work that this 
testimony is based on are listed within each individual product. 

[End of section] 

Related GAO Products: 

[End of section] 

Homeland Security: DHS Could Strengthen Acquisitions and Development 
of New Technologies. [hyperlink,]. Washington, D.C.: July 15, 

Environmental Protection Agency: To Better Fulfill Its Mission, EPA 
Needs a More Coordinated Approach to Managing Its Laboratories, 
[hyperlink,]. Washington, D.C.: 
July 25, 2011. 

DHS Science and Technology: Additional Steps Needed to Ensure Test and 
Evaluation Requirements Are Met. [hyperlink,]. Washington, D.C.: June 15, 

Opportunities to Reduce Potential Duplication in Government Programs, 
Save Tax Dollars, and Enhance Revenue, [hyperlink,]. Washington, D.C.: Mar. 2011. 

Homeland Security: Improvements in Managing Research and Development 
Could Help Reduce Inefficiencies and Costs, [hyperlink,]. Washington D.C.: Mar. 15, 

Warfighter Support: DOD's Urgent Needs Processes Need a More 
Comprehensive Approach and Evaluation for Potential Consolidation, 
[hyperlink,]. Washington, D.C.: 
Mar. 1, 2011. 

Technology Transfer: Clearer Priorities and Greater Use of Innovative 
Approaches Could Increase the Effectiveness of Technology Transfer at 
Department of Energy Laboratories [hyperlink,]. Washington, D.C.: June 16, 

Homeland Security: DHS Needs a Strategy to Use DOE's Laboratories for 
Research on Nuclear, Biological, and Chemical Detection and Response 
Technologies. [hyperlink,]. 
Washington, D.C.: May 24, 2004. 

Department of Energy: Uncertain Progress in Implementing National 
Laboratory Reforms, [hyperlink,]. Washington, D.C.: Sept. 
10, 1998. 

Best Practices: Elements Critical to Successfully Reducing Unneeded 
RDT&E Infrastructure. [hyperlink,]. Washington, D.C.: 
Jan. 8, 1998. 

Department of Energy: National Laboratories Need Clearer Missions and 
Better Management, [hyperlink,]. Washington, D.C.: Jan. 
27, 1995. 

[End of section] 


[1] Pub. L. No. 107-296, § 302(5), 116 Stat. 2135, 2163 (2002). 

[2] According to S&T, basic research includes scientific efforts and 
experimentation directed toward increasing knowledge and understanding 
in the fields of physical, engineering, environmental, social and life 
sciences related to long-term national needs. Applied research 
includes efforts directed toward solving specific problems with a view 
toward developing and evaluating the feasibility of proposed solutions. 

[3] S&T's Test & Evaluation and Standards office is responsible for 
overseeing key requirements that DHS components are required to follow 
in DHS's Test and Evaluation directive. 

[4] The fiscal year 2012 appropriations bill passed by the House of 
Representatives would appropriate about 42 percent less for S&T than 
what was appropriated in fiscal year 2011, while the bill passed by 
the Senate appropriations committee would provide almost 5 percent 

[5] GAO, Homeland Security: DHS Needs a Strategy to Use DOE's 
Laboratories for Research on Nuclear, Biological, and Chemical 
Detection and Response Technologies, [hyperlink,] (Washington, D.C.: May 24, 
2004); Department of Energy: Uncertain Progress in Implementing 
National Laboratory Reforms, [hyperlink,] (Washington, D.C.: Sept. 
10, 1998); Best Practices: Elements Critical to Successfully Reducing 
Unneeded RDT&E Infrastructure, [hyperlink,] (Washington, D.C.: 
Jan. 8, 1998). 

[6] GAO, Opportunities to Reduce Potential Duplication in Government 
Programs, Save Tax Dollars, and Enhance Revenue, [hyperlink,] (Washington, D.C.: Mar. 
2011). See also related GAO products at the end of this statement. 

[7] See related GAO products list at the end of this statement. 

[8] GAO, DHS Science and Technology: Additional Steps Needed to Ensure 
Test and Evaluation Requirements Are Met, [hyperlink,] (Washington, D.C.: June 15, 

[9] National Academy of Public Administration, Department of Homeland 
Security Science and Technology Directorate: Developing Technology to 
Protect America (Washington D.C.: June 2009). 

[10] GAO, Best Practices: Elements Critical to Successfully Reducing 
Unneeded RDT&E Infrastructure, [hyperlink,] (Washington, D.C.: 
Jan. 8, 1998). 

[11] [hyperlink,]. 

[12] GAO, Warfighter Support: DOD's Urgent Needs Processes Need a More 
Comprehensive Approach and Evaluation for Potential Consolidation, 
[hyperlink,] (Washington, D.C.: 
Mar. 1, 2011). 

[13] GAO, Homeland Security: DHS Needs a Strategy to Use DOE's 
Laboratories for Research on Nuclear, Biological, and Chemical 
Detection and Response Technologies, [hyperlink,] (Washington, D.C.: May 24, 

[14] GAO, Department of Energy: National Laboratories Need Clearer 
Missions and Better Management, [hyperlink,] (Washington, D.C.: Jan. 
27, 1995). 

[15] GAO, Department of Energy: Uncertain Progress in Implementing 
National Laboratory Reforms, [hyperlink,] (Washington, D.C.: Sept. 
10, 1998). 

[16] GAO, Technology Transfer: Clearer Priorities and Greater Use of 
Innovative Approaches Could Increase the Effectiveness of Technology 
Transfer at Department of Energy Laboratories, [hyperlink,] (Washington, D.C.: June 16, 

[17] GAO, Environmental Protection Agency: To Better Fulfill Its 
Mission, EPA Needs a More Coordinated Approach to Managing Its 
Laboratories, [hyperlink,] 
(Washington, D.C.: July 25, 2011). 

[End of section] 

GAO’s Mission: 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the 
performance and accountability of the federal government for the 
American people. GAO examines the use of public funds; evaluates 
federal programs and policies; and provides analyses, recommendations, 
and other assistance to help Congress make informed oversight, policy, 
and funding decisions. GAO’s commitment to good government is 
reflected in its core values of accountability, integrity, and 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s website [hyperlink,]. Each 
weekday afternoon, GAO posts on its website newly released reports, 
testimony, and correspondence. To have GAO e mail you a list of newly 
posted products, go to [hyperlink,] and select “E-
mail Updates.” 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black 
and white. Pricing and ordering information is posted on GAO’s 
website, [hyperlink,]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional 

Connect with GAO: 

Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Website: [hyperlink,]; 
Automated answering system: (800) 424-5454 or (202) 512-7470. 

Congressional Relations: 

Ralph Dawn, Managing Director,, (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, DC 20548. 

Public Affairs: 
Chuck Young, Managing Director,, (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, DC 20548.