This is the accessible text file for GAO report number GAO-06-91 
entitled 'Risk Management: Further Refinements Needed to Assess Risks 
and Prioritize Protective Measures at Ports and Other Critical 
Infrastructure' which was released on January 17, 2006. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

United States Government Accountability Office: 

GAO: 

December 2005: 

Risk Management: 

Further Refinements Needed to Assess Risks and Prioritize Protective 
Measures at Ports and Other Critical Infrastructure: 

GAO-06-91: 

GAO Highlights: 

Highlights of GAO-06-91, a report to Congressional Requesters: 

Why GAO Did This Study: 

Congress and the President have called for various homeland security 
efforts to be based on risk management—a systematic process for 
assessing threats and taking appropriate steps to deal with them. GAO 
examined how three Department of Homeland Security components were 
carrying out this charge: 

* the Coast Guard, which has overall responsibility for security in the 
nation’s ports;
* the Office for Domestic Preparedness (ODP), which awards grants for 
port security projects; and
* the Information Analysis and Infrastructure Protection Directorate 
(IAIP), which has responsibility for developing ways to assess risks 
across all types of critical infrastructure. 

GAO’s work focused on identifying the progress each DHS component has 
made on risk management and the challenges each faces in moving 
further. 

What GAO Found: 

The three DHS components GAO studied varied considerably in their 
progress in developing a sound risk management framework for homeland 
security responsibilities. The varied progress reflects, among other 
things, each component’s organizational maturity and the complexity of 
its task (see table below). The Coast Guard, which is furthest along, 
is the component of longest standing, being created in 1915, while IAIP 
came into being with the creation of the Department of Homeland 
Security in 2003. IAIP, which has made the least progress, is not only 
a new component but also has the most complex task—addressing not just 
ports but all types of infrastructure. The Coast Guard and ODP have a 
relatively robust methodology in place for assessing risks at ports; 
IAIP is still developing its methodology and has had several setbacks 
in completing the task. All three components, however, have much left 
to do. In particular, each component is limited in its ability to 
compare and prioritize risks. The Coast Guard and ODP can do so within 
a port but not between ports; IAIP has not demonstrated that it can do 
so either within or between all infrastructure sectors. 

Each component faces many challenges in making further progress. 
Success will depend partly on continuing to improve various technical 
and management processes that are part of risk management. For example, 
obtaining better quality data from intelligence agencies would help DHS 
components estimate the relative likelihood of various types of 
threats—a key element of assessing risks. In the longer term, progress 
will depend increasingly on how well risk management is coordinated 
across agencies, because current approaches in many ways are neither 
consistent nor comparable. Also, weaving risk-based data into the 
annual budget cycle of program review will be important. Supplying the 
necessary guidance and coordination is what the Department of Homeland 
Security was set up to do and, as the Secretary of Homeland Security 
has stated, what it now needs increasingly to address. This is a key 
issue for the department as it seeks to identify relative risks and 
take appropriate actions related to the nation’s homeland security 
activities. 

Progress in Risk Management Is Affected by Organizational Maturity and 
Complexity of Risk Management Task 

[See PDF for image] 

Source: GAO. 

[End of table] 

What GAO Recommends: 

This report contains many recommendations aimed at helping the three 
components face their next risk management challenges. DHS, including 
the Coast Guard, ODP, and IAIP, generally concurred with the report and 
its recommendations. DHS said that all three components have actions 
under way to address many of the recommendations in this report. 

www.gao.gov/cgi-bin/getrpt?GAO-06-91. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Margaret Wrightson at 
(415) 904-2200 or wrightsonm@gao.gov. 

[End of section] 

Contents: 

Letter: 

Executive Summary: 

Background: 

Results in Brief: 

Principal Findings: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Chapter 1: Introduction Risk Management Is a Key Tool for Homeland 
Security: 

Chapter 2: The Coast Guard Has Made Progress in Using Risk Management, 
but Challenges Remain: 

Chapter 3: Stronger Risk Management Approach Could Improve the 
Accountability of the Port Security Grant Program: 

Chapter 4: IAIP Faces Challenges in Meeting Risk Management 
Responsibilities across All Sectors of the Nation's Infrastructure: 

Chapter 5: Overall Observations and Recommendations: 

Appendix I: A Risk Management Framework: 

Appendix II: Comments from the Department of Homeland Security: 

Appendix III: GAO Contacts and Staff Acknowledgments: 

Related GAO Products: 

Tables: 

Table 1: Examples of Risk Management in the Private and Public Sectors: 

Table 2: Summary of Progress Made and Challenges That Remain in the 
Coast Guard's Risk Management Approach: 

Table 3: Port-Level Assessments Conducted by the Coast Guard: 

Table 4: National-Level Assessments Conducted by the Coast Guard: 

Table 5: Examples of Data-Related Challenges in Coast Guard Risk 
Assessments: 

Table 6: Summary of Progress and Challenges in the Port Security Grant 
Program: 

Table 7: Examples of Data-Related Challenges in ODP Risk Assessments: 

Table 8: Examples of Changes in Funding Decisions: 

Table 9: Summary of Progress Made and Challenges That Remain in IAIP's 
Risk Management Approach: 

Table 10: Critical Infrastructure Sectors and Lead Federal Agencies: 

Table 11: A Risk Management Framework: 

Figures: 

Figure 1: Risk Management Framework: 

Figure 2: A Framework for Risk Management: 

Figure 3: Facilities at One of the Nation's Major Ports: 

Figure 4: Sources of Evaluation Criteria Associated with Risk 
Management Phases: 

Abbreviations: 

ASME: American Society of Mechanical Engineers: 

CIP: critical infrastructure protection: 

COSO: Committee of Sponsoring Organizations: 

DHS: Department of Homeland Security: 

HITRAC: Homeland Infrastructure Threat and Risk Analysis Center: 

HSPD: Homeland Security Presidential Directive: 

IAIP: Information Analysis and Infrastructure Protection Directorate: 

IG: Inspector General: 

MARAD: Maritime Administration: 

MSRAM: Maritime Security Risk Analysis Model: 

MTSA: Maritime Transportation Security Act: 

NADB: National Assets Database: 

NIPP: National Infrastructure Protection Plan: 

ODP: Office for Domestic Preparedness: 

OMB: Office of Management and Budget: 

PS-RAT: Port Security Risk Assessment Tool: 

PWCS: Ports, Waterways, and Coastal Security: 

RAMCAP: Risk Analysis and Management for Critical Asset Protection: 

TSA: Transportation Security Administration: 

United States Government Accountability Office: 

Washington, DC 20548: 

December 15, 2005: 

The Honorable Henry A. Waxman: 
Ranking Minority Member: 
Committee on Government Reform: 
House of Representatives: 

The Honorable C. A. Dutch Ruppersberger: 
House of Representatives: 

The threat of terrorism presents a number of risks to our nation's 
seaports and other types of critical infrastructure. The Department of 
Homeland Security (DHS) has three components responsible for the 
security of critical infrastructure related to ports and other 
facilities. The U.S. Coast Guard has responsibility for port security 
overall. The Office for Domestic Preparedness (ODP) is responsible for 
providing port security grants to selected maritime facility owners. 
The Information Analysis and Infrastructure Protection (IAIP) 
Directorate is responsible for working with other federal, state, 
local, and private organizations to identify and protect critical 
infrastructure across the nation. Risk management is a tool for 
assessing risks, evaluating alternatives, making decisions, and 
implementing and monitoring protective measures. This report provides 
an evaluation of the progress made, and challenges faced, by the Coast 
Guard, ODP, and IAIP in using risk management to improve homeland 
security. 

As agreed with your offices, unless you publicly announce its contents 
earlier, we plan no further distribution of this report until 30 days 
after its issue date. At that time, we will provide copies of this 
report to appropriate departments and interested congressional 
committees. This report will also be available at no charge on the GAO 
Web site at http://www.gao.gov. 

If you or your staff have any questions about this report, please 
contact me at (415) 904-2200 or at wrightsonm@gao.gov. Key contributors 
to this report are listed in appendix III. 

Sincerely yours, 

Signed by: 

Margaret T. Wrightson: 
Director, Homeland Security and Justice Issues: 

[End of section] 

Executive Summary: 

Risk management, a strategy for helping policymakers make decisions 
about assessing risks, allocating resources, and taking actions under 
conditions of uncertainty, has been endorsed by Congress and the 
President as a way to strengthen the nation against possible terrorist 
attacks. Risk management has long been used in such areas as insurance 
and finance, but its application to domestic terrorism has no 
precedent. Unlike storms and accidents, terrorism involves an adversary 
with deliberate intent to destroy, and the probabilities and 
consequences of a terrorist act are poorly understood and difficult to 
predict. The size and complexity of homeland security activities and 
the number of organizations involved--both public and private--add 
another degree of difficulty to the task. The task of managing this 
complexity centers on the Department of Homeland Security, which since 
its inception in March 2003 has been faced with the challenge of 
transforming 22 agencies into an organization that can plan, manage, 
and carry out operations effectively. Congress likewise has a key 
oversight role to play in ensuring that DHS's course regarding risk 
management reflects a consensus as to the most prudent and cost- 
effective course of action. 

To assist Congress in its oversight, this report focuses on the 
progress made by three DHS components in applying risk management to 
homeland security activities and the challenges each component faces in 
moving further ahead. For two of these components, GAO's review dealt 
specifically with their risk management activities at the nation's 
seaports, while the review for the third component encompassed a wider 
range of infrastructure. GAO decided to focus a considerable amount of 
this review on seaport security because seaports have been viewed as 
potential terrorist targets or as conduits for importing a weapon of 
mass destruction or where terrorists may enter the country. GAO's focus 
on these three components, while not a comprehensive look across the 
entire department, provides perspective on the degree of progress made 
thus far. Risk management has applications for deliberate acts of 
terror as well as natural disasters, such as hurricanes and 
earthquakes. GAO's research, which was conducted prior to Hurricane 
Katrina, focused on preparations for terrorist attacks, not natural 
disasters. The three components GAO studied are: 

* the Coast Guard, the lead federal agency for port security and the 
agency responsible for developing and coordinating various risk-based 
assessments of critical infrastructure in and around ports; 

* the Office for Domestic Preparedness, administrator of the port 
security grant program, has awarded more than half a billion dollars in 
federal grants to owners and operators of port facilities and vessels; 
and: 

* the Information Analysis and Infrastructure Protection Directorate, 
which has been charged with establishing uniform policies, approaches, 
guidelines, and methodologies for integrating infrastructure protection 
and risk management activities within and across key sectors, such as 
energy, defense, and transportation, including airports, railroads, and 
ports.[Footnote 1] 

Besides describing the progress of and challenges for each component, 
this report also presents GAO's observations about what the three 
components' efforts indicate collectively, both with regard to how far 
the department has come in managing homeland security efforts on the 
basis of risk and what steps could help advance the current level of 
progress. 

Background: 

Seaport security receives particular attention in this report because 
seaports are widely viewed as representing attractive terrorist 
targets, in part because of their importance to the economy. More than 
95 percent of the nation's non-North American foreign trade (and 100 
percent of certain commodities, such as foreign oil) arrives by ship. 
The estimated economic consequences of a successful attack and 
resulting shutdown of this system total billions of dollars. Ports also 
represent attractive targets because they contain a myriad of 
vulnerabilities. In all, the nation's 300-plus ports have about 3,700 
cargo and passenger terminals. Chemical factories, oil refineries, 
power plants, and other facilities are often located in port areas and 
add another set of possible targets. Roads crisscross many ports, 
allowing access by land as well as by water, and the number of people 
working in or traveling through ports is in the millions. The Coast 
Guard has the major responsibility for seaport security, and the port 
security grant program administered by ODP adds to the resources 
available for port security projects. 

Relative to the Coast Guard and ODP, IAIP's homeland security 
responsibilities are by far the widest-ranging. The Homeland Security 
Act of 2002 and Homeland Security Presidential Directive 7 (HSPD-7) 
charge IAIP with establishing a risk management framework across the 
federal government to protect the nation's critical infrastructure and 
key resources.[Footnote 2] The scope of this effort is immense, and the 
effort is one of IAIP's central responsibilities. IAIP's task 
ultimately involves developing an approach that can inform decisions on 
what the nation's antiterrorism priorities should be and identifying 
what strategies and programs will do the most good. IAIP's work is done 
in a setting where numerous and substantial gaps in security remain, 
but resources for closing these gaps are limited. More specifically, 
IAIP is charged with examining and comparing relative risks associated 
with a multitude of possible targets, ranging from specific structures 
(such as dams, chemical plants, and nuclear power plants) to major 
sectors of national infrastructure (such as the banking system, 
computer networks, and water systems). IAIP is also responsible for 
developing policies and guidance that other agencies can use in 
conducting their own risk assessments. 

While federal law and the presidential directive call for the use of 
risk management in homeland security, little specific federal guidance 
or direction exists as to how risk management should be implemented. To 
provide a basis for analyzing component efforts, GAO developed a 
framework for risk management based on industry best practices and 
other criteria. This framework, shown in figure 1, divides risk 
management into five major phases: (1) setting strategic goals and 
objectives, and determining constraints; (2) assessing the risks; (3) 
evaluating alternatives for addressing these risks; (4) selecting the 
appropriate alternatives; and (5) implementing the alternatives and 
monitoring the progress made and results achieved. For all three 
components, GAO applied this framework after conducting a wide range of 
interviews with officials, reviewing plans and activities of the three 
components, and visiting port locations. As part of our work, GAO 
briefed officials of the three components about the various phases of 
the framework and the officials generally agreed with its structure and 
intent. The application of risk management to homeland security is 
relatively new, and the framework will likely evolve as processes 
mature and lessons are learned. 

Figure 1: Risk Management Framework: 

[See PDF for image] 

[End of figure] 

Results in Brief: 

Of the three components GAO reviewed, the Coast Guard had made the most 
progress in establishing a foundation for using a risk management 
approach; its next challenges are to further refine and enhance its 
approach. While the Coast Guard has made progress in all five risk 
management phases, its greatest progress has been made in conducting 
risk assessments--that is, evaluating individual threats, the degree of 
vulnerability, and the consequences of a successful attack. However, 
the assessments are limited in their reliability and completeness, and 
better coordination will be needed with the intelligence community so 
that analysts can develop models that better assess the relative 
probability of various threat scenarios. The Coast Guard has developed 
the ability to compare and prioritize risks at individual. However, it 
cannot yet compare and prioritize relative risks of various 
infrastructure across ports. Other challenges include developing 
performance measures to go along with the more general goals already 
developed for the port security mission, further integrating risk into 
the annual cycle of program and budget review, and developing formal 
policies for reviewing and improving the implementation of a risk 
management approach. The Coast Guard has actions under way to address 
the challenges it faces in each risk management phase. Several of these 
actions are based, in part, on briefings GAO held with agency 
officials. 

ODP has made progress in applying risk management to the port security 
grant program, but like the Coast Guard, it also faces challenges 
across all phases of the risk management framework. For example, ODP 
has set broad risk management goals and has placed more emphasis on 
using risk-based data in its assessments, but it lacks performance 
measures showing what specific outcomes the program aims to achieve, 
and it still faces challenges in such matters as comparing grant 
applications across ports. The grant awards for fiscal year 2004 also 
illustrate some of the challenges in ensuring that criteria for making 
the awards are transparent and consistent. At the end of what was, in 
part, a risk-based assessment process, ODP changed the criteria for 
awarding grants when it decided to give lower priority to applicants 
from large companies on the assumption that the companies were better 
able than other entities to pay for their own improvements. This 
changed 40 percent of the grants awarded, as projects with higher risk 
but greater potential for self-funding gave way to lower-risk projects 
with more limited funding prospects. In the procedures for fiscal year 
2005 awards, ODP clarified the criteria it would use in making the 
awards. 

IAIP, which has the broadest risk management responsibilities of the 
three components and faces the greatest challenges, has made the least 
progress in carrying out its complex risk management activities. Its 
efforts are aligned with high-level strategic goals, but ways to 
measure performance in achieving these goals have yet to be developed. 
IAIP is not as far along as ODP and the Coast Guard in conducting risk 
assessments. While IAIP has provided input to ODP for its risk 
assessment efforts, IAIP's risk assessment responsibilities span much 
broader sectors of the nation's infrastructure than seaports alone, 
making its assessment activities more difficult. This difficulty is 
reflected in the limited progress made. With regard to its risk 
assessment responsibilities, IAIP has yet to successfully (1) develop 
data to determine the relative likelihood of various threat scenarios, 
(2) complete a methodology for comparing and prioritizing key assets, 
or (3) meet requirements set forth in HSPD-7 for issuing policies and 
guidance that other agencies can use in conducting their own risk 
assessments. For example, a DHS consultant issued a risk assessment 
methodology in 2004 for collecting data from industry, but adverse 
comments from reviewers have led to revisions that are still under way. 
IAIP is also challenged in its ability to translate these assessments 
into specific measures to be taken, because after IAIP makes decisions 
about what national priorities should be, it is dependent on the 
actions of others to carry them out. This is particularly true with 
regard to private sector assets where IAIP needs collaboration from the 
owners and operators of private sector infrastructure and their 
regulators. 

GAO made four main observations regarding the experience of these three 
components. 

* A considerable degree of effort has been expended thus far, but much 
work remains to be done. This is particularly true in viewing risk 
management strategically--that is, with a view that goes beyond just 
assessing what the risks are and also integrating the consideration of 
risk into the annual cycle of program and budget review that is already 
in place. 

* The varying degree of progress among the three components tends to 
reflect several characteristics of each component--how long it has been 
at the task of developing a risk management approach, how long it has 
existed as a component and is therefore able to function maturely, and 
how complex its risk management task is. For example, IAIP, which has 
made the least progress, is not only a new component established in 
2003, but also has the most complex risk-related tasks of the three 
components--addressing risk not only at ports, but across all types of 
infrastructure and with multiple federal agencies and nonfederal 
stakeholders. 

* In the near term, all three components' success in risk management 
will depend partly on continuing to make progress on the challenges 
described above. This involves continuing to work on such matters as 
performance measures, basic policies, and enhancements to existing risk 
assessment tools. 

* The final observation is related to a critical longer-term need: more 
guidance and coordination from the department level, both to help 
ensure that individual components such as IAIP are carrying out their 
roles effectively and to ensure that the various responses from 
individual components mesh as effectively as possible with one another. 
In comparing the approaches developed by the three components, GAO 
noted ways in which their efforts were not consistent. The danger is 
that if components develop systems and methodologies that are 
inconsistent, they may end up with incompatible systems that have 
little or no ability to inform spending decisions on homeland security. 
The challenges associated with creating a department that can 
effectively administer a coherent risk management approach to the 
nation's homeland security have been widely acknowledged. IAIP 
recognizes that as DHS's individual components begin to mature in their 
risk management efforts, the need increases for ensuring consistency 
and coherence in these efforts. Supplying this necessary guidance and 
coordination is what DHS was set up to do. 

Principal Findings: 

Coast Guard Has Made Progress in Using Risk Management, but Challenges 
Remain: 

The Coast Guard has made progress across all five phases of risk 
management. In the first phase (goal and objective setting), the Coast 
Guard has established broad strategic goals for port security, 
including, in order of priority: (1) preventing terrorist attacks 
within, and terrorist exploitation of, the maritime domain and (2) 
reducing America's vulnerability to terrorism in the maritime domain. 
It faces challenges in developing objectives that translate these goals 
into more specific and measurable results. Coast Guard officials 
recognize that developing performance measures is a necessary next step 
and have actions under way to develop such measures. 

For the second phase, assessing risks, the Coast Guard has greatly 
expanded the scope of its risk assessment activities since the 
terrorist attacks of September 11, 2001. It has conducted three major 
security assessments at the port level, and collectively these 
assessments have resulted in considerable progress in understanding and 
prioritizing risks within a port. After it initiated port-level 
assessments, the Coast Guard expanded its analysis efforts to the 
national level to gain a more strategic perspective on port security. 
In all, the Coast Guard has conducted three major efforts at the 
national level, focusing more generally on understanding the risk posed 
to various classes of assets (such as bridges or container ships) by 
various types of attacks (such as using explosives or weapons of mass 
destruction). The assessments are limited in their reliability and 
completeness, however, in the degree to which the Coast Guard has (1) 
formal and systematic input from the intelligence community for 
modeling relative probability and likelihood of threat scenarios and 
(2) risk assessment tools allowing comparison and prioritization of 
specific infrastructure across ports. These limitations affect the 
degree to which the Coast Guard is able to determine how best to focus 
its attention on these threats that, from a national perspective, pose 
the greatest risk within the seaport sector. The Coast Guard has 
initiated actions to address these challenges. For example, the agency 
has initiated contact with the intelligence community to obtain better 
data on threat scenarios, and it plans to complete development of an 
assessment tool that will compare the relative risks of high-value 
assets at one port with risks of assets in a different port. 

Enhancing these first two phases of risk management is key to making 
additional progress on the next two phases--evaluating and selecting 
alternatives that reduce risk. While the Coast Guard's efforts have 
resulted in progress in identifying and evaluating alternatives at the 
individual port level, the lack of measurable objectives and sufficient 
information to fully depict threats, vulnerabilities, and consequences 
limits the ability to target the areas with the greatest gaps or 
produce the most cost-effective decisions. Similarly, buttressing 
annual budget review cycles with risk-based data is in its early 
development and more work remains to be done. Finally, with regard to 
the fifth phase--implementation and monitoring--the Coast Guard has 
implemented a number of activities to mitigate risks and has 
demonstrated the ability to evaluate its efforts and make improvements. 
The actions taken have included establishing maritime intelligence 
centers on the Atlantic and Pacific coasts and working closely with 
nonfederal stakeholders to reduce vulnerabilities in and around 
facilities and vessels. However, existing feedback mechanisms are 
insufficient to ensure that Coast Guard field personnel can make their 
headquarters managers aware of ways to improve the process. The Coast 
Guard recognizes the value of formal feedback loops as a means of 
improving its risk management processes, and it has plans to obtain 
formal feedback as part of its future efforts. 

ODP's Port Security Grant Program Illustrates Both Progress and 
Challenges in Implementing Risk Management: 

Like the Coast Guard, ODP has made progress across all five phases of 
risk management. For example, for the first phase, it has set risk 
management goals that support broader maritime goals, such as 
protecting critical infrastructure in harbors, borders, ports, and 
coastal approaches. ODP has not begun to translate these broad goals 
into measurable objectives. Without them, it is difficult to know what 
progress has been made in reducing risk and what security gaps remain. 

ODP has carried out risk assessments, with input from the Coast Guard 
and IAIP, and evaluated mitigation alternatives--the second and third 
phases of the framework--to help determine which ports should receive 
priority for grants. Using risk assessments, ODP narrowed the number of 
ports eligible for grants from 129 to 66 for fiscal year 2005. Other 
recent steps include placing greater emphasis on using threat, 
vulnerability, and consequence data in prioritizing grant applications. 
Along with this progress, however, are several methodological 
challenges that limit such things as the usefulness of data received 
from intelligence agencies and ODP's ability to compare and prioritize 
risks among ports. For example, without data on the relative 
probability of various threat scenarios from the Coast Guard or IAIP, 
ODP may not target the most significant security needs. ODP has not yet 
developed approaches for addressing most of these challenges. 

While ODP has also made progress in developing a risk-based grant 
selection process and mechanisms to monitor what the grants accomplish, 
grant awards for fiscal year 2004 illustrate the challenges involved in 
actually making risk-based decisions. At the end of its process for 
determining which grants to fund, based in part on risk, ODP decided to 
give lower priority to grants involving projects at large companies, on 
the assumption that the companies were better able than other entities 
to pay for their own improvements. For example, one chemical company's 
application for $225,000 to purchase cameras, fencing, and barricades 
was initially ranked 25th out of 287 applications nationwide, but under 
the revised priorities its ranking fell to 236th. Projects initially 
ranked much lower received funding instead. For example, an application 
initially ranked as 279th out of 287 was approved for funding. In all, 
the application of non-risk criteria changed 40 percent of the grants 
awarded. ODP's changes affected the transparency and consistency of the 
awards process, in that (1) the criteria under which applications were 
submitted and initially considered were changed at the end of the 
process, and (2) the role of risk in evaluating the applications was 
obscured, because the resulting awards may not have addressed the most 
severe security gaps. Additionally, there is no guarantee that large 
companies would spend their own funds for security improvements, and it 
is unclear whether there are incentives, such as minimum standards for 
security, that would motivate them to do so. ODP issued revised 
criteria for fiscal year 2005 grants, and in doing so has made the 
process more transparent and consistent. 

IAIP's Progress in Carrying Out Risk Management Has Been Limited: 

IAIP's progress in all five phases of risk management has been limited. 
It has made some progress in developing goals, having issued an Interim 
National Infrastructure Protection Plan in February 2005 that 
identifies a strategy for identifying, prioritizing, and coordinating 
the protection of critical infrastructure and key resources.[Footnote 
3] The interim plan provides some guidance in meeting IAIP's broad 
responsibilities for identifying, comparing, and prioritizing critical 
assets, but it is not a comprehensive document, and IAIP faces several 
challenges in making it more comprehensive. These challenges are 
related to (1) developing performance measures that can be used in 
evaluating progress and (2) establishing milestones and time frames for 
processing and prioritizing assets across the many different 
infrastructure sectors. 

IAIP's progress in risk assessment--the second phase of risk 
management--has been limited in several main respects. For example, 
IAIP has experienced difficulties in carrying out requirements of the 
Homeland Security Act of 2002 that charged IAIP with the responsibility 
of conducting risk assessments of critical infrastructure and key 
resources to determine the risks of particular types of terrorist 
attacks. IAIP's original methodology for this task, called the Risk 
Analysis and Management for Critical Asset Protection, required 
extensive modification after its initial issuance in April 2004. IAIP 
now views it as a tool for engaging industry in a risk management 
dialogue with government. In September 2005, IAIP officials said they 
are developing a National Comparative Risk Assessment to meet the 
immediate need of examining risks within and across sectors, and they 
plan to complete an interim assessment by the end of 2006. Challenges 
to carrying out this timetable include the need to obtain key 
information from other federal agencies and the fact that IAIP still 
needs to award the contract for this effort. One specific issue is the 
approach IAIP has taken in assessing the probability of various threat 
scenarios. The Homeland Security Act calls on IAIP to assess the 
probability of success of terrorist attacks, and during the course of 
GAO's review, IAIP officials said they recognize the importance of 
assessing the relative likelihood of an attack in meeting this 
requirement. IAIP officials said that the lack of intelligence analysis 
and data on such things as the capability and intent of terrorist 
groups hinders their ability to assess probability, but that work is 
under way in this regard. IAIP officials also pointed out that some 
inaccuracy is to be expected in examining the intent and capability of 
an adversary whose plans are concealed and that it will be important to 
reduce the potential of low-confidence assessments having undue 
influence when long-term investment decisions are made. 

IAIP's progress in the three other phases of risk management 
(evaluating alternatives, selecting a solution, and implementing and 
monitoring that solution) will remain limited, in part because of the 
points just discussed--performance goals and a complete risk assessment 
methodology are not in place. Beyond these limitations, however, IAIP 
faces additional challenges. For example, IAIP's role in selection, 
implementation, and monitoring is further complicated because in many 
instances, other entities have primary responsibility for selecting the 
solution. For example, other agencies, such as the Department of 
Defense or the Department of Energy, have primary responsibilities for 
some of the infrastructure sectors covered in IAIP's assessments. 
Additionally, much of the critical infrastructure is owned or operated 
by private industry, and while IAIP does not have authority over them, 
other federal agencies do have authority over infrastructure in 
specific sectors. This condition highlights the importance of 
coordination between IAIP and agencies with such regulatory authority. 
For example, the Nuclear Regulatory Commission, which issues licenses 
to nuclear power plants, has regulatory authority over security matters 
at these facilities. IAIP officials said they use their expertise and 
powers of persuasion to bring about specific actions but in most cases 
cannot compel others to adopt IAIP's recommendations. 

Overall Observations: 

A great amount of effort has been applied. However, much more remains 
to be done than has been accomplished so far. Across all three 
components, the most progress has generally been made on fundamental 
steps, such as conducting risk assessments of individual assets, and 
less progress has generally been made on developing ways to translate 
this information into comparisons and priorities across ports and 
across infrastructure sectors, or applying it to new programs. Progress 
among the three components' efforts has been far from consistent and 
has tended to vary not only with the length of time the component has 
been using a risk-based approach, but also with the component's own 
maturity level and the complexity of its risk management task. 

With regard to next steps that would appear to add the most value to 
making further progress, one key observation is that in the short term, 
progress is heavily dependent on continuing to improve basic policies, 
procedures, and methods for applying risk management. Each component 
has an admittedly difficult set of challenges ahead, but progress has 
to be built on taking these incremental steps and doing them well. An 
area that needs further attention by all three entities is working with 
intelligence communities to develop improved analysis and data so that 
the relative probability of various threat scenarios can be further 
developed. 

The final observation is that in the longer term, progress will become 
increasingly dependent on how well the entire risk management effort is 
coordinated. While absolute compatibility among all components' efforts 
is likely impossible, even with components working in close 
cooperation, strong coordination is important to help ensure that 
component efforts are consistent rather than stovepiped. The risk 
management efforts GAO examined appeared to be fueled by a strong 
concern to make some headway, with coordination and interagency 
consistency a lesser concern. For example, the Coast Guard initiated 
efforts to set up a methodology for assessing and specifying risks 
before IAIP was created; and in the view of IAIP officials, it was 
important to proceed even though they recognized that doing so might 
lead to approaches that would not mesh cleanly with the approach IAIP 
would eventually develop. That approach appeared prudent in the short 
term, in that if the Coast Guard had waited to begin until guidelines 
had been set, it would still be waiting. Now, however, the need for 
coordination is looming larger, and coordination is essential to the 
success of efforts over time. Some of this coordination needs to come 
from IAIP, which is required under presidential directive to issue 
guidelines for other agencies to use, but it has yet to do so. Beyond 
IAIP, DHS has an active role in this regard. This is a key issue for 
the department as it moves from being an organization that is 
essentially in its early stages to one that is increasingly being 
expected to respond in a way that is more organizationally mature. 
Since 2003, GAO has designated the implementation and transformation of 
DHS as high risk because of the numerous challenges in transforming 22 
agencies into one department and the serious implications of failure. 
Translating the concept of risk management into applications that are 
consistent and useful represents one of these challenges, and failure 
to effectively address this could have serious consequences for 
homeland security. In risk management, which the department has 
embraced as the guiding principle behind its policies and operations, 
IAIP's role is to act as an intra-agency and interagency coordinator of 
homeland security activities. Doing so will strengthen its ability to 
weigh risks and inform the decisions made across the homeland security 
responsibilities of the many agencies involved. 

Recommendations for Executive Action: 

GAO is making a number of specific recommendations to the Secretary of 
DHS with regard to the challenges faced by the three components. These 
recommendations, listed specifically at the end of the relevant 
chapters, cover such matters as developing performance goals and 
measures, improving risk assessment methodologies, working with 
intelligence communities to develop better data for risk assessment 
purposes, and (for IAIP) developing guidance for other agencies to use 
in evaluating risk and considering risk mitigation alternatives. 

Agency Comments and Our Evaluation: 

We provided DHS a draft of this report for its review and comment. DHS, 
including the Coast Guard, ODP, and IAIP, generally agreed with our 
findings and recommendations. For instance, DHS said that each DHS 
component we reviewed has actions under way to address recommendations 
made in the report. The comments from each component are summarized at 
the end of the relevant chapters. In addition to commenting on our 
findings and recommendations, DHS provided technical comments under 
separate cover, and we revised the draft report where appropriate. 
Written comments from DHS are reprinted in appendix II. 

[End of section] 

Chapter 1: Introduction: Risk Management Is a Key Tool for Homeland 
Security: 

This is a report about the nation's progress in applying risk 
management to key aspects of homeland security. Risk management is a 
widely endorsed strategy for helping policymakers make decisions about 
allocating finite resources and taking actions in conditions of 
uncertainty. It has been widely practiced for years in such areas as 
insurance, construction, and finance. By comparison, its application in 
homeland security is relatively new--much of it coming in the wake of 
the terrorist attacks of September 11--and it is a difficult task with 
little precedent. The goals for using it in homeland security include 
informing strategic decisions on ways to reduce the likelihood that 
adverse events will occur, and mitigate the negative impacts of and 
ensure a speedy recovery from those that do. Achieving these goals 
involves making policy decisions about what the nation's homeland 
security priorities should be--for example what the relative security 
priorities should be among seaports, airports, and rail--and basing 
spending decisions on what approaches or strategies will do the most 
good at narrowing the security gaps that exist. Risk management has 
been widely supported by the President and Congress as a management 
approach for homeland security, and the Secretary of the Department of 
Homeland Security has made it the centerpiece of agency policy. 

"Homeland security" is a broad term with connotations that resonate 
from the September 11 attacks and other connotations that now resonate 
from the disaster brought on by Hurricane Katrina in August 2005. Risk 
management has applications for deliberate assaults like the September 
11 attacks and natural disasters, such as hurricanes and earthquakes. 
Our research was completed and the report largely written before 
Hurricane Katrina struck. Thus, our work concentrated on components' 
actions in response to terrorism. 

This report examines how three DHS components have applied risk 
management to certain aspects of their homeland security 
responsibilities. The three components are the United States Coast 
Guard, the Office for Domestic Preparedness, and the Information 
Analysis and Infrastructure Protection Directorate. This report looks 
at risk management efforts of the Coast Guard and ODP specifically 
related to seaport security, and for IAIP, it looks at risk management 
efforts related to IAIP's broader responsibilities in assessing 
terrorist threats against all aspects of the nation's 
infrastructure.[Footnote 4] 

Risk Management Has a Long History of Use in Industry and Government: 

Risk management can be described as the continuous process of assessing 
risks, reducing the potential that an adverse event will occur, and 
putting steps in place to deal with any event that does occur.[Footnote 
5] It has been used in the private and public sectors for decades (see 
table 1 for examples). For example, insurance companies use a variety 
of statistical techniques to assess the level of risk for what they are 
insuring. Within government, agencies use risk management to set 
regulations and to protect the environment and the health and safety of 
American taxpayers. Although some risk management methodologies and 
processes can be complex and may require expert advice and support, 
other aspects of risk management--such as setting goals and using 
performance measures to track progress in meeting them--are well 
understood and widely practiced. 

Table 1: Examples of Risk Management in the Private and Public Sectors: 

Private sector examples: 

Type of application: Insurance; 
How risk management is used: Insurance companies evaluate risks when 
insuring businesses and homeowners against natural disasters. They 
assess the probability of natural disasters, such as hurricanes and 
earthquakes, based on past history and the costs resulting from the 
damage caused or the lives lost. On the basis of analysis such as this, 
companies set policies and costs that apply to businesses and 
homeowners. 

Type of application: Engineering; 
How risk management is used: Engineering firms have analyzed risks 
related to safety and security when designing chemical plants, nuclear 
reactors, or bridges. Using risk analysis techniques, they examine the 
possible threats to the safety and security of the structure and 
evaluate ways to address the threat by considering various design 
features that could reduce vulnerabilities or consequences. One example 
is designing double-hulled oil tankers to reduce the risk of an Exxon 
Valdez type oil spill. 

Type of application: Banking and finance; 
How risk management is used: Banks and financial institutions assess 
risks associated with various investment options. For example, spending 
funds on overseas investments could involve assessing political, 
social, and financial risks as well as the potential market share that 
could be gained. Assessments such as these inform decisions on where 
and whether capital should be invested. 

Public sector examples: 

Type of application: Food and Drug Administration; 
How risk management is used: The Food and Drug Administration assesses 
risk associated with diseases related to various types of food. It 
examines whether diseases are linked to types of fish and dairy goods. 
It examines the types and costs of health problems that may occur and 
it recommends and sets policies or regulations aimed at improving food 
safety. 

Type of application: Environmental Protection Agency; 
How risk management is used: The Environmental Protection Agency 
analyzes health risks caused by toxic chemicals, emissions from 
vehicles, and other sources of pollution. It examines the extent to 
which such pollutants may cause health problems and it sets and 
recommends policies or regulations to minimize the risk to the public. 

Type of application: Department of Defense; 
How risk management is used: The Department of Defense uses a risk 
management approach to protect its forces. For example, it has used 
risk management to identify threats and vulnerabilities, and determine 
which assets are the most critical and to make management decisions on 
how to make its bases and related facilities more secure. 

Source: GAO. 

[End of table] 

Application of Risk Management to Homeland Security Is Widely Endorsed 
and Accepted: 

Risk management was part of the nation's approach to assessing 
terrorism before the events of September 11. For example, in the 1990s, 
the Defense Special Weapons Agency assessed risks to evaluate force 
protection security requirements for mass casualty terrorist incidents 
at military bases. Companies under contract to federal agencies such as 
the Department of Energy, the National Security Agency, and the 
National Aeronautics and Space Administration used risk assessment 
models and methods to identify and prioritize security requirements. 
The Federal Aviation Administration and the Federal Bureau of 
Investigation did joint threat and vulnerability assessments on 
airports determined to be high risk. When we reviewed two of these 
efforts in the late 1990s, we found a lack of formal risk assessment 
requirements and made several recommendations to integrate risk-based 
data into decision-making processes.[Footnote 6] 

What September 11 changed was the intensity and magnitude of this task. 
The September 11 attacks were clearly a transformational event for the 
nation, in that they called attention to vulnerabilities throughout the 
nation's infrastructure, not just in aviation security. While there 
might always have been a concern, for example, about the consequences 
of an accident in a chemical factory in a highly populated area, now, 
these consequences had to be viewed not just from the standpoint of a 
potential accident, but as something a terrorist could exploit. 
Potential targets multiplied, and the scope of work to be done became 
much greater. Homeland security spending rose from about $21 billion in 
fiscal year 2001 to a proposed $50 billion in fiscal year 2006. 

Risk management has received widespread support and interest from 
Congress, the President, and the Secretary of DHS as a tool that can 
help set priorities on how to protect the homeland. In this setting, 
numerous and substantial gaps in security exist, but resources for 
closing these gaps are limited and must compete with other national 
priorities. Policymakers in the legislative and executive branches have 
endorsed risk management as a technique that can inform decisions on 
setting relative priorities and on making spending decisions. 

In view of the widespread support that risk management has gained, 
federal agencies are now required to assess risks. The Homeland 
Security Act of 2002 calls for a comprehensive assessment of risk 
related to vulnerabilities of critical infrastructure and key 
resources, notably (1) the risk posed by different forms of terrorist 
attacks, (2) the probability that different forms of an attack might 
succeed, and (3) the feasibility and efficacy of countermeasures to 
deter or prevent such attacks.[Footnote 7] Two congressionally 
chartered commissions, the 9/11 Commission and the Gilmore Commission, 
support the use of data on risks to help inform the difficult decisions 
that must be made in allocating limited federal funds for security 
measures. The President has issued policies directing the heads of 
seven major departments or agencies to assess risks. The past and 
present Secretaries of DHS have stated that actions of the department 
will be guided through the use of risk management. 

Department of Homeland Security Has Broad and Challenging 
Responsibilities in Applying Risk Management: 

Congress has charged DHS with lead responsibilities in carrying out or 
coordinating homeland security programs and for applying risk 
management in carrying out this responsibility. For two main reasons, 
integrating a risk management approach into its business practices is a 
major management challenge that faces DHS. 

* First, relative to many other fields such as insurance or finance, 
terrorism is a relatively new application for risk management. The 
sources of the risk are intelligent adversaries with malevolent intent 
with whom there is relatively little domestic experience. Unlike the 
insurance or banking industries, which have extensive historical data 
that are used to assess risks, DHS lacks such data on domestic 
terrorism, and this limits any detailed analysis in assessing risk. As 
a result, the probabilities and consequences of a terrorist act are 
poorly understood and difficult to predict and greater reliance on 
expert judgment is required. In January 2005, we identified risk 
management as an emerging high-risk area. At that time, we noted that 
DHS had not completed any risk assessments mandated by the Homeland 
Security Act.[Footnote 8] 

* Also, the size and complexity of homeland security activities add 
another dimension of difficulty to the task. Since its inception in 
March 2003, DHS has been faced with the challenge of transforming 22 
agencies into one department in a way that results in an organization 
with effective planning, management, and operations while carrying out 
its critical mission of securing the homeland. Since 2003, we have 
designated implementing and transforming DHS as high risk, because DHS 
had to transform these many agencies--several with major management 
challenges--into one department.[Footnote 9] Besides the challenge it 
poses at the federal level, risk management also crosses jurisdictional 
boundaries and involves state and local governments and private 
industry stakeholders, and it requires a multidisciplinary approach 
involving intelligence, law enforcement, strategic planning, and 
program activities that address threats and vulnerabilities. 

Within DHS, we examined the progress of three DHS components--the Coast 
Guard, ODP, and IAIP--in administering risk management as part of their 
management processes. Two of these components (Coast Guard and ODP) 
have responsibilities related to seaport security. IAIP's 
responsibilities are much broader and more difficult--it is responsible 
for coordinating and assessing homeland risks across the federal 
government. Here is an overview of the three components. 

* The United States Coast Guard. The Coast Guard is the lead federal 
agency for the security of the nation's ports. Its responsibilities 
include protecting ports, the flow of commerce, and the maritime 
transportation system from terrorism. As the lead in domestic maritime 
security, the Coast Guard has a robust presence at the national, 
regional, and port levels. The Coast Guard protects more than 300 ports 
and 95,000 miles of coastline. By providing a secure environment, the 
Coast Guard keeps maritime transportation open for the transit of 
commercial goods, as well as assets and personnel from the armed 
forces. In carrying out its mission, the Coast Guard has, among other 
activities, conducted local and national assessments of security risks 
at the nation's ports. The role of the Coast Guard in applying risk 
management to port security is discussed in more detail in chapter 2. 

* The Office for Domestic Preparedness. Within the Office of State and 
Local Government Coordination and Preparedness, the Office for Domestic 
Preparedness is responsible for administering federal homeland security 
assistance programs for states and localities, including the port 
security grant program. Since 2002, the program has awarded over $500 
million in grants to state, local, and industry stakeholders to improve 
security in and around their facilities or vessels. The role of ODP in 
applying risk management to port security grants is discussed in more 
detail in chapter 3. 

* Information Analysis and Infrastructure Protection Directorate. The 
Information Analysis and Infrastructure Protection Directorate is 
responsible for, among other things, identifying and assessing current 
and future threats to the homeland, mapping those threats against known 
vulnerabilities, recommending protective measures, issuing warnings, 
and offering advice on preventive and protective action. IAIP is 
responsible for cataloging key critical infrastructure, then analyzing 
various characteristics to prioritize this infrastructure for the 
entire nation. These priorities are then to be used to direct 
protective measures for port security as well as across all other kinds 
of infrastructure. The role of IAIP in applying risk management to 
ports and other infrastructure is discussed in more detail in chapter 
4. 

Seaports Are an Important Focus in the Homeland Security Response: 

Seaport security receives substantial focus in this report because 
seaports have been widely regarded as vulnerable to attack. One reason 
is that the nation's seaports and inland waterways play a vital role in 
the nation's economy and national security. From an economic 
perspective, ports are critical links in the commercial trade and 
transportation systems, with more than 95 percent of the nation's non- 
North American foreign trade, including 100 percent of foreign oil, 
entering the country through seaports. The range of commodities 
involved includes not only a wide variety of consumer and agricultural 
products, but also cargo considered dangerous such as liquefied 
petroleum gas. A significant portion of this waterborne trade comes via 
cargo containers that are expected to move in and out of ports quickly, 
in keeping with industry expectations of just-in-time delivery. Port 
facilities are also used to ship military cargo abroad, and the 
Departments of Defense and Transportation have designated about 17 
ports as "strategic" to support wartime mobilization, deployment, and 
resupply. Finally, not only are ports key hubs in our transportation 
system, they also function as centers of industrial, commercial, and 
financial activity. As such, they are home to many assets that are 
deemed to be among the nation's most critical infrastructure, which is 
to be protected under the USA PATRIOT Act of 2001 and the Homeland 
Security Act of 2002. 

A second reason that seaports are potentially vulnerable is the wide 
range of targets and attack possibilities they encompass. Facilities 
such as container terminals, where containers are transferred between 
ships and railroad cars or trucks, must be able to screen vehicles 
entering the facility and routinely check cargo. Chemical factories and 
other installations where hazardous materials are present must be able 
to control access to areas containing dangerous goods or hazardous 
substances. Vessels, ranging from oil tankers and freighters to 
tugboats and passenger ferries, must be able to restrict access to 
onboard areas, such as the bridge or other control stations critical to 
the vessels' operation. Possible terrorist scenarios range from the use 
of improvised explosive devices to attack ferries to the use of 
recreational boats to ram key infrastructure in and around ports. 

A Framework for Risk Management: 

While there is a consensus that risk management should be applied to 
homeland security programs, doing so is a complex task that has few 
precedents and little specific guidance. The Homeland Security Act and 
presidential directives have called for the use of risk management. 
However, they did not define how risk management was to be 
accomplished. Given that there are no established universally agreed 
upon set of requirements or processes for risk management of homeland 
security, we developed a framework that can be broadly applied to a 
range of settings, such as analyzing security in the maritime sector 
and other environments. We did so by gathering, reviewing, and 
analyzing an extensive amount of public and private sector work; 
interviewing experts from private consulting companies in the areas of 
risk management and risk computer-modeling; interviewing experts on 
terrorism; and utilizing our own past work in this area. We also 
solicited comments and feedback from academic experts in risk 
management. As part of our work, we briefed officials of the three DHS 
components about the various phases of the framework, and the officials 
generally agreed with its structure and intent. The application of risk 
management to homeland security is relatively new, and the framework 
will likely evolve as processes mature and lessons are learned. 

The framework we developed is a conceptual synthesis of risk management 
approaches that we use as criteria to assess the adequacy of DHS's risk 
management systems (see fig. 2). For further information on the 
framework and how we developed it, see appendix I. 

Figure 2: A Framework for Risk Management: 

[See PDF for image] 

[End of figure] 

This framework may be applied governmentwide and at various 
organizational levels, from departmental down to individual programs. 
The figure illustrates the cyclical nature of this approach, and while 
the phases are generally linear, changes can be made at any step in the 
process as new information becomes available. The five major phases of 
risk management are detailed below. 

Strategic Goals, Objectives, and Constraints: 

According to the framework, management decisions are to be made in the 
context of the organization's strategic plan, with clearly articulated 
goals and objectives that flow from the plan. Performance measures that 
are clear, concise, and measurable are linked to the broader goals and 
can be used to measure progress toward these goals. An organization's 
program and risk planning documents address risk-related issues that 
are central to its mission.[Footnote 10] However, various constraints 
can take many forms and have an impact on risk related strategies. For 
example, some constraints may be imposed by statute, organizational 
policy, or budget restrictions. Managers at different levels within an 
agency or organization may encounter various constraints that differ 
with the scale of the operation. 

Risk Assessment: 

Risk assessment helps decision makers identify and evaluate potential 
risks facing key assets or missions so that countermeasures can be 
designed and implemented to prevent or mitigate the effects of the 
risks.[Footnote 11] In our framework, risk assessment is a function of 
threat, vulnerability, and consequence. The product of these elements 
is used to develop scenarios and help inform actions that are best 
suited to prevent an attack or mitigate vulnerabilities to a terrorist 
attack, in conjunction with the risk-based evaluation of alternatives 
undertaken while considering cost and other factors. 

* Threat is the probability that a specific type of attack will be 
initiated against a particular target/class of targets. It may include 
any indication, circumstance, or event with the potential to cause the 
loss of or damage to an asset. It is based on an understanding of an 
adversary's intention, motivation, history of attacks, and capability 
to do damage. Analysis of threat-related data is a critical part of 
risk assessment. Information for characterizing threat can be gained 
from a variety of sources, such as the intelligence and law enforcement 
community, as well as from past activities of various terrorist groups. 
Understanding an underlying pattern of attacks on target types is 
useful in predicting future terrorist events and planning mitigation 
strategies. However, the unexpected threats not contained in the 
historical record of terrorist groups also need to be considered. 
Ultimately, one purpose of assessing threats is to assign relative 
probabilities to various types of attacks. 

* The vulnerability of an asset is the probability that a particular 
attempted attack will succeed against a particular target or class of 
targets. It is usually measured against some set of standards, such as 
availability/predictability, accessibility, countermeasures in place, 
and target hardness (the material construction characteristics of the 
asset). Each of these four elements can be evaluated based on a 
numerical assignment corresponding to the conditional probability of a 
successful attack. The probability that a particular vulnerability 
could be successfully exploited is, in part, a function of the 
effectiveness of the antiterrorism countermeasures. 

* The consequence of a terrorist attack is characterized as the 
expected worst case or worse reasonable adverse impact of a successful 
attack. The consequence to a particular asset can be evaluated when 
threat and vulnerability are considered together. The outcome of a 
terrorist attack may include many forms, such as the loss of human 
lives, economic costs, and adverse impact on national security. 

Another closely related element taken into consideration is criticality 
(that is, the relative importance) of the asset involved. Criticality 
involves the prioritization of assets based on factors such as the 
potential for loss of life and the economic implications for the 
livelihood, resources, or wealth of the area, region, or country if the 
asset were to be lost. Layers of effective security countermeasures 
increase the likelihood that a terrorist attack will be unsuccessful as 
risk is reduced. 

Alternatives Evaluation: 

Risks can be reduced through various antiterrorism countermeasures or 
countermeasure systems designed to prevent an attack or mitigate the 
impact of an attack. Two concepts here are key to evaluating 
countermeasure alternatives. The first is that countermeasures should 
be evaluated against specific risk assessments to determine the extent 
to which risks can be reduced by the countermeasure being considered. 
The second concept is the role of costs to both public and private 
sources, as costs are a critical element in the application of 
countermeasures. In our framework, cost-benefit analysis is critical in 
assessing alternatives, because it links the benefits derived from risk-
reducing alternatives to the costs associated with implementing and 
maintaining them. 

Management Selection: 

Management selection in our framework is informed by the outputs in the 
preceding phases. Having assessed risks and evaluated countermeasure 
options, management selects the blend of intervention strategies and 
activities across the entire spectrum of goals, objectives, and 
components of risk that achieves the greatest expected risk reduction 
in relation to cost for both the short and the long term among the 
various proposed alternatives. However, the technical analysis of 
alternatives is not likely to resolve or fully capture the numerous 
elements of concern to management. Decision makers may employ various 
risk-reducing strategies. However, preferences and value judgments will 
influence decisions about which strategies to employ. For example, 
corporate culture may influence decision makers to concentrate 
countermeasures on a relatively few critical assets, while others may 
value distributional impacts, that is, some organizations may be more 
willing than others to distribute resources over a wider array of 
assets. Management selection is an important task, and decisions are 
made with the information that is available. Our guidelines for 
effective internal controls dictate that once decisions are reached, 
they, along with the rationales for them, should be documented in order 
to inform future actions. 

Implementation and Monitoring: 

This phase in the framework involves the implementation of the selected 
countermeasures. Following implementation, monitoring is essential in 
order to help ensure that the entire risk management process remains 
current and relevant, and reflects changes in the effectiveness of the 
alternative actions and the risk environment in which it operates. It 
is crucial to exploit any and all information sources, exercises, 
gaming, modeling and simulation, analysis of real world events, and 
sharing of information in a data sparse environment. Measurable 
objectives show the degree to which activities, timelines, support 
functions, service delivery, and spending are consistent with goals and 
implemented in accordance with the planning process. Program evaluation 
is an important tool for assessing the efficiency and effectiveness of 
the program. In addition to simply monitoring the implementation of the 
system and making adjustments, the entire risk management planning 
process should be periodically revisited. Since technology and 
information change at a rapid pace, countermeasures in place today may 
be outdated tomorrow and may become more susceptible to being breached. 
In addition, consultation with external subject area experts can 
provide a current perspective and an independent review in the 
formulation and evaluation of the program. 

Objectives, Scope, and Methodology: 

Our overall aim was to provide a perspective on how three DHS 
components have applied risk management as it relates to homeland 
security in general, or to port security in particular. More 
specifically, this report addresses the following objectives: 

* What progress has the Coast Guard made in applying risk management to 
its port security mission, and what challenges does it face in moving 
further? 

* What progress has ODP made in applying risk management to its 
administration of the port security grant program, and what challenges 
does it face in moving further? 

* What progress has IAIP made in applying risk management to comparing 
and prioritizing critical infrastructure with one another and what 
challenges does it face in moving further? 

* Are there key observations that can be drawn from all three of these 
efforts with regard to how far the three components have come in risk 
management as it applies to terrorism? 

To determine what progress the Coast Guard has made in applying risk 
management to its port security mission and the challenges that it 
faces, we met with Coast Guard officials responsible for port security 
risk assessment efforts to discuss the progress they have made and the 
challenges that remain. We discussed risk management efforts and 
challenges with Coast Guard officials at four ports--Baltimore, 
Maryland; Charleston, South Carolina; Houston, Texas; and Seattle, 
Washington--who were responsible for risk management activities. We 
judgmentally selected these ports because of their geographic 
distribution, and the results from our interviews cannot be generalized 
to ports nationwide. In addition, we reviewed documents of the Port 
Security Assessment Program, the Port Security Risk Assessment Tool, 
Area Maritime Security Plans, the National Risk Assessment Tool, the 
National Maritime Security Profile, and the National Maritime Strategic 
Risk Assessment. We also reviewed key legislation such as the Maritime 
Transportation Security Act of 2002 and prior GAO reports on maritime 
security. Finally, we reviewed threat assessments produced by the 
National Maritime Intelligence Center and the Transportation Security 
Agency to gain a more complete understanding of the challenges. 

To determine what progress ODP has made in applying risk management to 
its administration of the port security grant program and the 
challenges that it faces, we compared fiscal year 2004 and fiscal year 
2005 ODP port security grant program procedures. In order to understand 
the grant process and the risks related to individual ports, we 
reviewed the risk assessment tools used by ODP officials, including the 
Coast Guard's Port Security Risk Assessment Tool.[Footnote 12] We 
reviewed and summarized a database listing fiscal year 2004 grant 
applications and awards to determine the extent to which criteria for 
awards coincided with the receipt of grant awards. We reviewed the 
Inspector General's (IG) January 2005 report of the port security grant 
program and discussed the recommendations contained in the report with 
ODP officials. We examined procedural changes made by ODP, in response 
to the IG recommendations and other factors, to the 2005 grant 
application process. We did not review the fiscal year 2005 award 
decisions because we had completed our fieldwork before award decisions 
were announced, in September 2005. We met with Coast Guard, Maritime 
Administration, ODP, and IAIP officials involved in the port security 
grant program process. We also reviewed pertinent legislation, such as 
appropriations for the grant program for successive fiscal years. 

To determine what progress IAIP has made in applying risk management to 
comparing and prioritizing critical infrastructure, and what challenges 
it faces in moving ahead, we reviewed key legislative and executive 
documents, such as the Interim National Infrastructure Protection Plan, 
the Homeland Security Act of 2002, Homeland Security Presidential 
Directives 7 and 8, national strategies, and DHS's strategic plan. We 
met with IAIP officials responsible for identifying and prioritizing 
threats, vulnerabilities, and consequences across different types of 
critical infrastructure to determine the obstacles they face in making 
such evaluations and the challenges they face in making progress in 
this area. We reviewed documents, such as the Risk Analysis and 
Management for Critical Asset Protection and the Buffer Zone Protection 
Program. We interviewed Office of Management and Budget (OMB) officials 
responsible for oversight of issues involving infrastructure protection 
to obtain their views on risk management practices across the federal 
government. 

To determine whether there are key observations that can be drawn from 
the three components we reviewed, we analyzed and synthesized the 
findings we developed to identify challenges that remain in applying 
risk management to homeland security. We compared the three components' 
progress in applying risk management principles to their respective 
tasks, identified common experiences in applying risk management, and 
drew conclusions about issues that may need addressing. We reviewed 
numerous documents, including pertinent statutes and presidential 
directives, GAO reports on high-risk programs in the federal 
government, the National Strategy for the Physical Protection of 
Critical Infrastructures and Key Assets, and testimony by the Secretary 
of DHS. 

We performed our work in accordance with generally accepted government 
auditing standards between May 2004 and November 2005. 

[End of section] 

Chapter 2: The Coast Guard Has Made Progress in Using Risk Management, 
but Challenges Remain: 

The Coast Guard has established a foundation for applying risk 
management to port security; its next challenges are to refine and 
strengthen its approach. The Coast Guard has made progress in all five 
phases of risk management (see table 2). For the first phase--goals and 
objectives--the Coast Guard has established broad strategic goals for 
port security, and its next challenge is to translate these goals into 
specific, measurable objectives that can be used to assess performance. 
The Coast Guard has actions under way to address this challenge--a key 
step in determining the extent to which its actions actually reduce 
risk. The Coast Guard has made the most progress in the second phase-- 
conducting risk assessments. Six separate but related assessment 
efforts, covering both individual ports and the nation as a whole, have 
given the Coast Guard a clearer sense of the vulnerabilities that 
exist. However, current assessments are limited in terms of their 
methodology, and they do not allow the Coast Guard to compare and 
prioritize relative risks across ports--limitations that the Coast 
Guard recognizes and is taking steps to address. Enhancing these first 
two risk management phases is key to making further progress on the 
next two phases--evaluating risk mitigation alternatives and selecting 
a particular alternative for action. Without measurable objectives and 
more complete methodologies, the risk management process may not be 
able to target the most significant security concerns or determine the 
most cost-effective approach to take in providing reasonable 
protection. Additionally, weaving data produced from the risk 
management process into the annual cycle of program review remains a 
challenge. Finally, with regard to the fifth phase--implementation and 
monitoring--the Coast Guard has demonstrated the ability to evaluate 
its efforts and make improvements. However, more extensive and more 
formal feedback mechanisms would help ensure that Coast Guard 
headquarters managers can inform field staff about actions taken as a 
result of the comments received about the risk management process. The 
Coast Guard has actions under way to improve feedback loops. 

Table 2: Summary of Progress Made and Challenges That Remain in the 
Coast Guard's Risk Management Approach: 

Risk management phase: Strategic goals, objectives, and constraints; 
Examples of progress made: High-level strategic goals have been set for 
port security nationwide and for port locations across the country; 
Examples of remaining challenges: High-level goals have not been 
translated into measurable objectives. The Coast Guard recognizes the 
importance of developing measurable objectives and is working to do so. 

Risk management phase: Risk assessment; 
Examples of progress made: Several types of risk assessments have been 
conducted at both the port and the national level. They have given the 
Coast Guard the ability to compare and prioritize infrastructure within 
a port; 
Examples of remaining challenges: Data on threats, vulnerabilities, and 
consequences have limitations; Methods have not been developed to allow 
the Coast Guard to compare and prioritize risks across ports; Coast 
Guard officials agreed they have challenges and are taking action to 
address them. 

Risk management phase: Alternatives evaluation; 
Examples of progress made: Using local risk assessments, the Coast 
Guard has developed alternative approaches to prevent attacks and 
reduce vulnerabilities; 
Examples of remaining challenges: At the national level, the Coast 
Guard's methodology for evaluating alternatives is limited. National 
risk assessments generally lack cost and benefit data on alternative 
ways to mitigate port security risks. The Coast Guard is taking steps 
to address this challenge by examining benefits (reductions in risk) 
and the estimated costs in doing so. 

Risk management phase: Management selection; 
Examples of progress made: Coast Guard officials have been able to use 
expert knowledge or data from risk assessments to select specific 
alternatives, such as establishing security zones around key 
infrastructure, improving security around ferries and cruise ships, and 
coordinating security improvements (such as fences, gates, and cameras) 
around key infrastructure; 
Examples of remaining challenges: Methodological limits in risk 
assessments and alternatives evaluation hinder the quality of data that 
informs management decisions. Informing the annual cycle of program 
review with data from risk management processes has been limited. The 
Coast Guard recognizes these challenges and has actions under way to 
address them. 

Risk management phase: Implementation and monitoring; 
Examples of progress made: The Coast Guard has implemented improvements 
to some of its risk assessment tools to make them stronger and has 
invited feedback from staff on how processes are working; 
Examples of remaining challenges: Existing feedback mechanisms are 
limited to largely informal processes, reducing communication between 
headquarters and field staff about actions taken as a result of the 
comments or feedback provided. The Coast Guard plans to include formal 
feedback loops in one of its risk assessment tools by the end of 2005. 

Source: GAO analysis of the Coast Guard's risk management efforts. 

[End of table] 

Coast Guard Homeland Security Activities Revolve Heavily around the 
Maritime Domain: 

The Coast Guard is the lead federal agency responsible for protecting 
domestic ports. In this role, the Coast Guard must identify, evaluate, 
and mitigate many kinds of security challenges. Ports are often 
sprawling enterprises that contain key infrastructure besides docks, 
piers, ships, barges, and warehouses. Many ports are also home to power 
plants, chemical factories, bridges and tunnels, and a variety of other 
assets of critical importance to the nation's economy and its defense. 

Coast Guard expenditures and activities for port security have risen 
dramatically since the terrorist attacks of September 11. The Coast 
Guard estimates that its budget for port security has jumped from about 
$250 million in fiscal year 2001 to about $1.5 billion in fiscal year 
2005. Since the terrorist attacks, the Coast Guard has carried out a 
myriad of port security activities, including increasing its 
intelligence capabilities, carrying out more harbor patrols and vessel 
escorts, establishing security zones, and working more extensively with 
federal, state, local, and industry stakeholders on port security 
matters as required by the Maritime Transportation Security Act of 2002 
(MTSA). 

While much of the Coast Guard's homeland security efforts center 
specifically on ports, these activities are part of the agency's 
broader mission of Ports, Waterways, and Coastal Security (PWCS). This 
mission involves protecting the maritime domain and marine 
transportation system; preventing terrorist attacks, and responding to 
and recovering from attacks that do occur.[Footnote 13] As part of the 
PWCS mission, the Coast Guard aims to develop greater "maritime domain 
awareness"--that is, improving port stakeholders' understanding about 
anything associated with the global maritime environment that could 
adversely affect the security, safety, economy, or environment of the 
United States. Maritime domain awareness seeks to identify threats as 
soon as possible and far enough away from domestic ports to eliminate 
or mitigate the threat. Several Coast Guard efforts are under way to 
help address both port security and marine domain awareness. In 
particular: 

* The Coast Guard is planning to expand its sector command centers, 
where officials can receive data 24 hours a day on maritime activities. 
Currently, the Coast Guard plans to develop sector command centers at 
35 ports.[Footnote 14] 

* The Coast Guard is involved in a major recapitalization effort-- 
called the Integrated Deepwater System--to replace and modernize the 
agency's aging fleet of aircraft and vessels, including improved and 
integrated command, control, communications and computers, 
intelligence, and surveillance and reconnaissance capabilities. Since 
the terrorist attacks of September 11, the Coast Guard has revised its 
plans, and the Deepwater program now includes improving maritime domain 
awareness and maritime security capabilities as part of its mission. 
This program is scheduled to take 20 years and cost between $19 billion 
and $24 billion. 

Federal statutes and presidential directives call for the Coast Guard 
to use risk management in its homeland security efforts. MTSA, for 
example, calls for the Coast Guard and other port security stakeholders 
to carry out a variety of risk-based tasks, including assessing risks 
and developing security plans for ports, facilities, and 
vessels.[Footnote 15] The Coast Guard's progress across the various 
risk management phases is thus a key part of its homeland security 
mission. The remainder of this chapter discusses the Coast Guard's 
progress and challenges on each of the phases in GAO's framework. 

Goal-Setting: High-Level Goals Are in Place, with Efforts Under Way to 
Set Measurable Objectives: 

The Coast Guard has been able to make some progress in the first phase 
of the risk management framework, in that it has established high-level 
strategic goals for its PWCS mission. The Coast Guard set its national 
strategic goals for port security in December 2002.[Footnote 16] In 
order of priority, the goals were as follows: 

* preventing terrorist attacks within, and terrorist exploitation of, 
the maritime domain; 

* reducing America's vulnerability to terrorism in the maritime domain; 

* protecting population centers, critical infrastructure, maritime 
borders, ports, coastal approaches, and the boundaries and seams 
between them; 

* protecting the U.S. marine transportation system while preserving the 
freedom of the maritime domain for legitimate pursuits; and: 

* minimizing the damage and expediting the recovery from attacks that 
may occur within the maritime domain as either the lead federal agency 
or a supporting agency. 

Working with federal, state, local, and industry stakeholders involved 
in port security, the Coast Guard has also developed security plans for 
port areas across the country. These plans reflect the characteristics 
and needs of the individual ports, and in general, they aim to deter a 
terrorist incident and improve communication among port stakeholders. 
These plans are specific to each port location and are aligned with 
higher-level port security goals. 

While the Coast Guard has set broad goals for its port security 
mission, it still faces challenges in developing objectives into more 
specific and measurable results that measure progress toward these 
goals. So far, the Coast Guard has expressed its port security 
objectives in terms of activity levels, such as conducting patrols, 
escorting vessels, and inspecting cargo. While such activities may have 
contributed to improved security in and around the nation's ports, 
using them as measures may not systematically target areas of higher 
risk and may not result in the most effective use of resources, because 
these measures are not pointed toward outcomes. They describe what 
levels of activity, or outputs, the Coast Guard is providing, but they 
do not provide an indication of what these activities are 
accomplishing. Doing so requires measures that are clearly tied to 
results. Such measures would indicate the extent of progress made and 
help identify the security gaps that still remain.[Footnote 17] 

Developing measurable objectives is a complex and difficult task, but 
Coast Guard officials recognize that doing so is a necessary next step 
and plan to have such objectives developed in fiscal year 2006. In 
September 2005, the Coast Guard stated that it plans to develop a 
measure of its performance that will be based on an assessment of 
threat, vulnerability, and consequence. The Coast Guard plans to 
develop objectives for reducing overall risk. As part of this process, 
the Coast Guard plans to assess the impact of its activities in 
reducing threats, vulnerabilities, and consequences. 

Risk Assessments: Progress Has Been Substantial, but Challenges Remain 
for Addressing Limitations in Assessment Data and Methodology: 

The Coast Guard's risk management activities have centered primarily on 
this phase of the risk management process, with assessments being 
conducted at both port and national levels. Further progress on the 
quality of these assessments is challenged by several types of 
limitations in the data and the methodology being used. 

Several Sets of Assessments Have Been Completed at the Port and 
National Levels: 

Following the terrorist attacks of September 11, and consistent with 
MTSA's directives, the Coast Guard has greatly expanded the scope of 
its risk assessment activities. Before the attacks, these assessments 
centered on matters such as ecological damage and general marine 
safety. In 1999, for example, the Coast Guard adopted a risk management 
approach for its marine safety and environmental mission area. 
Assessments of key infrastructure in and around ports came largely 
after the attacks. Three such assessments have been done at the port 
level (see table 3).[Footnote 18] These assessments included data on 
threats, vulnerabilities, and consequences--the three types of 
information used in evaluating risk: 

* Threats. The assessments gathered information on plausible threat 
scenarios, such as using weapons of mass destruction, ramming a vessel 
or facility, or detonating devices underwater. In general, local Coast 
Guard personnel or other federal and nonfederal stakeholders decided 
what threat scenarios to include in the assessment based on their 
knowledge of the port. 

* Vulnerabilities. The assessments evaluated threat scenarios against 
potential targets, such as passenger vessels, bridges, or terminals, to 
assess the degree to which these potential targets were vulnerable to 
attack. 

* Consequences. Finally, the three assessments addressed the potential 
outcomes of successfully carrying out a threat against a potential 
target. These consequences included such matters as loss of life, 
damage to the environment, damage to property, and economic disruption. 

Table 3: Port-Level Assessments Conducted by the Coast Guard: 

Port-level risk assessment: Port Security Risk Assessment Tool (PS- 
RAT); 
Description: Implemented in November 2001, PS-RAT is a computer- based 
tool for determining the risk associated with specific attack scenarios 
against key infrastructure or vessels in local ports. It was used to 
compare and prioritize risks among critical infrastructures at a port. 
In November 2002, the Coast Guard improved the tool to address 
inconsistencies in data among ports, and it included additional factors 
for mitigation, such as recoverability from an attack. 

Port-level risk assessment: Port Security Assessment Program; 
Description: Begun in August 2002 and completed in March 2005, this 
program produced a vulnerability assessment of 55 of the nation's most 
strategic commercial and military ports. To identify which ports were 
of the most strategic importance, the Coast Guard considered such 
factors as cargo volume, ferry and cruise ship traffic, population 
density around the port, and presence of critical infrastructure. 

Port-level risk assessment: Area Maritime Security Plans; 
Description: Required under the Maritime Transportation Security Act of 
2002, these plans describe a communication and coordination framework 
for port stakeholders and law enforcement officials to follow in 
addressing security vulnerabilities and responding to incidents. The 
Coast Guard has completed plans for all 43 designated port areas. 

Source: GAO analysis of the Coast Guard's port-level assessments. 

[End of table] 

These assessments have resulted in considerable progress in 
understanding and prioritizing risks within a port. In particular, the 
Port Security Risk Assessment Tool (PS-RAT), a computer program that 
includes possible threat scenarios, allows the Coast Guard to assess 
risks and develop relative rankings for infrastructure at each port 
location. At one port, PS-RAT was used to compare and inform priorities 
on over 1,000 critical items of infrastructure. 

Three other efforts have focused on assessments at the national level 
(see table 4).[Footnote 19] These efforts have relied in part on data 
and information generated from the local assessments described above, 
but they have also incorporated additional information. For example, 
the National Maritime Security Profile integrated available information 
from the intelligence community--a step that had not been carried out 
in any of the local risk assessments. Another assessment, the National 
Maritime Strategic Risk Assessment, sought to develop risk profiles for 
each of the Coast Guard's strategic goals; it examined specific mission 
areas, including port security, search and rescue, and law enforcement, 
and it sought input from field commanders on ways to mitigate key 
risks. According to Coast Guard staff, this was the first attempt at a 
large-scale strategic risk assessment that sought to assess the status 
of the maritime domain. 

Table 4: National-Level Assessments Conducted by the Coast Guard: 

National-level assessments: National Risk Assessment Tool; 
Description: Implemented in February 2002, this tool provided a 
foundation for strategically evaluating the risks in the maritime 
domain. It incorporated 50 types of infrastructure and 12 possible 
attack scenarios and included information on threat, vulnerability, and 
consequences. Information from the local tool (PS-RAT) was used in 
developing the results. 

National-level assessments: National Maritime Security Profile; 
Description: Developed in 2003-2004, this profile assessed critical 
infrastructure, possible threats, vulnerabilities, and consequences. It 
used PS-RAT and the national risk assessment tool, among other methods, 
to develop the profile, and it gathered input from the intelligence 
community to improve data on threats for its PWCS mission. 

National-level assessments: National Maritime Strategic Risk 
Assessment; 
Description: Launched in August 2004, this is an effort to communicate 
risks in each of the Coast Guard's missions. As part of this risk 
assessment, the Coast Guard used information from the National Maritime 
Security Profile. Among other things, this effort identified possible 
intervention strategies for addressing risk areas. 

Source: GAO analysis of the Coast Guard's national-level assessments. 

[End of table] 

While the port-level assessments focus on specific assets and 
infrastructure at each location, national assessments have focused more 
generally on understanding the risk posed to various classes of assets 
(such as container ships, barges, power plants, or bridges and tunnels) 
by various types of attacks (such as using explosives, taking control 
of an asset, or using weapons of mass destruction). These national 
assessments address, for example, whether the maritime domain is at 
greater risk from a takeover of a power plant or from a weapon of mass 
destruction planted on a container ship. The national assessments do 
not compare risks faced by specific assets at one port with risks faced 
by specific assets at another port. 

Key Challenges Involve Improving Data and Methodology: 

The progress in conducting risk assessments is tempered by a number of 
challenges that remain in making these assessments more robust tools 
for informing the risk management process. These challenges are 
numerous and complicated, and this chapter illustrates some of the 
important issues the Coast Guard faces in making its risk assessments 
more useful. The challenges discussed here involve (1) improving the 
threat, vulnerability, and consequence data on which the assessments 
are based and (2) addressing methodological limitations that affect the 
reliability, completeness, and applicability of the risk assessments 
themselves. 

Limitations of Data on Threats, Vulnerabilities, and Consequences: 

Our review of the Coast Guard's processes and our discussions with 
Coast Guard personnel surfaced a number of data-related problems that 
limit the reliability and completeness of the risk assessments (see 
table 5). For example, the tools require information not only about the 
types of threats a facility may face, but also about the probability of 
such threats occurring. The threat data received by the Coast Guard 
from the intelligence community do not allow the Coast Guard to model 
these probabilities, thus limiting the value of the output. The 
problems we identified have implications for the ability to effectively 
compare risks faced by the various types of port-related 
infrastructure. 

Table 5: Examples of Data-Related Challenges in Coast Guard Risk 
Assessments: 

Data type: Threats; 
Summary of challenge: The information received from intelligence 
sources is generally useful, but it lacks the detail that allows the 
Coast Guard to model the relative probability of various threat 
scenarios. For example, the Coast Guard cannot assign a relative 
probability to various threat scenarios, affecting the ability to 
characterize threats without either understating or overstating them. 
In practice, the calculation of threat was essentially held constant 
for Coast Guard-wide analysis of PS-RAT data. 

Data type: Vulnerabilities; 
Summary of challenge: The Coast Guard's tools for assessing risk 
currently do not take into account (1) reductions in vulnerability that 
stem from the Coast Guard's actions (such as security patrols or other 
monitoring) or (2) the effect that multiple strategies (such as fencing 
and guards) may have on reducing vulnerabilities. As a result, the 
tools may overstate the degree of vulnerability that exists. 

Data type: Consequences; 
Summary of challenge: The Coast Guard's tools measure the direct 
effects of a terrorist attack, such as loss of lives and property 
damage, but they do not consider the secondary effects, such as loss of 
jobs that may occur. This limitation likely understates the overall 
consequences resulting from an attack. 

Source: GAO analysis of the Coast Guard's risk assessment efforts. 

[End of table] 

These challenges are complex and technical, and the following examples 
illustrate the kinds of limitations they pose: 

* Limitations in threat-related data. The intelligence information the 
Coast Guard normally receives about threats is not specific enough for 
all of the threat scenarios in the National Maritime Security Profile 
or the PS-RAT.[Footnote 20] For example, the type of threat data that 
Coast Guard personnel could use to model threats includes data on the 
presence of terrorist cells nationally and internationally, the 
capability and intent of terrorist groups as they relate to specific 
types of attack in and around ports, and the specific target groups. 
Increasing the quality of this information would improve the quality of 
the output. For example, when the Coast Guard received and integrated 
higher-quality information about some threats from its Intelligence 
Coordination Center, it modified data for 80 of about 300 threat 
scenarios in its National Maritime Security Profile. 

* Limitations in vulnerability-related data. The Coast Guard's risk 
assessment tools were designed to evaluate existing security in and 
around a building or vessel, but according to Coast Guard officials, 
the baseline established by the tools excludes areawide actions the 
Coast Guard has taken to reduce vulnerabilities in and around ports, 
such as conducting more patrols, creating operational centers, or 
establishing security zones in and around key ports. The baseline also 
excludes actions taken by local port stakeholders, such as increasing 
the number of harbor patrols conducted by local law enforcement. The 
Coast Guard designed its tools this way because the primary purpose of 
the tools was to provide a port-level risk ranking of assets. Using 
this information, the Coast Guard could then use tools, such as the PS- 
RAT, to measure the benefit and value of any interventions for a 
specific vessel, facility, or asset type that it initiates against the 
original baseline. Coast Guard officials recognize that another tool 
would be useful in determining the overall vulnerabilities that exist 
after all Coast Guard actions have been taken. 

* Limitations in consequences-related data. The Coast Guard's 
assessment of consequences is limited to direct effects of a terrorist 
attack, such as loss of lives and property damage; it does not consider 
important secondary effects, such as follow-on effects to the economy, 
including loss of jobs or increased energy costs that may occur months 
after an attack. This limitation likely understates the overall 
consequences that result from an attack and may distort relative risks 
associated with various threat scenarios. Coast Guard officials note 
that estimating secondary effects is important but difficult since 
there is a lack of accepted methods for doing so. A second limitation 
is that there is no commonly agreed upon value for a consequence such 
as death or injury, or the symbolic effect of destroying a national 
symbol such as the Statue of Liberty. For example, the Coast Guard's 
model places a dollar value of $1 million (in 2005 dollars) on the loss 
of a life. Other components, such as the Environmental Protection 
Agency, use a monetary value of $6.1 million (in 1999 dollars). The 
value chosen can affect the priorities that emerge from using the risk 
assessment tools. 

We discussed these data-related limitations with Coast Guard officials, 
who generally agreed with our observations. The Coast Guard has since 
planned or started several actions designed to address some of these 
limitations. Coast Guard officials said they plan to improve PS-RAT 
based in part on the limitations discussed above.[Footnote 21] For 
example, they made changes in procedures for obtaining information from 
the Intelligence Coordination Center and have focused on improving the 
quality of information received from the intelligence 
community.[Footnote 22] In addition, the Coast Guard plans to improve 
vulnerability and consequence data. For example, the Coast Guard plans 
to assign weights to different levels of consequences. Coast Guard 
officials said they hope to accomplish these changes by the end of 
2005. 

Methodological Limitations: 

Two key methodological limitations affect the use of risk assessment 
data as a tool for informing decision makers on relative risks across 
port locations. The first limitation relates directly to the ability to 
compare and prioritize one port with another, while the second 
limitation relates to the process used to rank and prioritize 
individual threats. 

* Risks cannot be compared between ports. PS-RAT results were not 
designed to compare the risks at one port with risks at 
another.[Footnote 23] While PS-RAT allows the Coast Guard to compare 
and prioritize key infrastructure within a port, it does not produce a 
risk ranking that permits the Coast Guard to compare and prioritize 
infrastructure across ports. Interport comparisons, while theoretically 
possible, are difficult to actualize in practice. In general terms, the 
difficulties stem largely from the fact that, for each port, multiple 
scenarios must be considered; the scenarios that are deemed most 
relevant to each port, however, will differ from port to port. For 
example, ports that support passenger ferries and container cargo may 
be exposed to different risks than ports that primarily support bulk 
cargo. Comparisons are further limited because Coast Guard personnel at 
different ports use different methods to input data into PS-RAT. For 
example, Coast Guard officials at some field offices have summarized 
information on a type of asset, such as all bridges at a port location, 
while officials at other locations have developed data on each bridge 
in the area of responsibility. We discussed this limitation with the 
Coast Guard officials, and during the course of our review, the Coast 
Guard initiated work on a Maritime Security Risk Assessment Model 
(MSRAM)--a model that should permit the Coast Guard to compare the 
relative risks of high-value assets at one port with assets at a 
different port. The model is to include an analysis of various threat 
scenarios, vulnerabilities, and consequences. Overall, the MSRAM is to 
score and characterize risk associated with individual assets, 
including the estimated likelihood of an attack, the vulnerability of 
the asset should an attack occur, and the impacts of a successful 
attack. The Coast Guard has also requested data from the Intelligence 
Coordination Center on intent and capability that could improve the 
estimate of relative likelihood of various threat scenarios. Coast 
Guard officials said they plan to implement the MSRAM by the end of 
2005. 

* Risks viewed as less probable could surprise the Coast Guard. The 
Coast Guard evaluates hundreds of threat scenarios that are deemed 
plausible--focusing attention, ultimately, on those threats that, 
together with identified vulnerabilities and consequences, pose the 
greatest overall risk. For example, in the National Maritime Security 
Profile, the Coast Guard classifies the risk of various threat 
scenarios as very high, high, medium, low, and very low, and it centers 
attention on scenarios that it estimates are very high or high risk. 
The Coast Guard's approach, although a useful starting point, may not 
be as reliable as the process would appear to suggest given the data 
limitations we describe above. How agencies like the Coast Guard deal 
with scenarios that receive low rankings is important in addressing the 
possibility of strategic surprise--an attack scenario that may not be 
identified or given high priority in the initial risk assessment 
process. Without sensitivity analysis or formal feedback loops to 
reassess all scenarios and therefore provide greater assurance that the 
rankings are as reliable as possible, the risk of being unprepared for 
strategic surprise may increase.[Footnote 24] The Coast Guard addresses 
this issue by making appropriate adjustments in priorities when 
tactical and strategic information call for such changes. We discussed 
this issue further with agency officials, and partly on the basis of 
these discussions, the Coast Guard is taking additional steps to 
address this issue. It is doing so by (1) increasing coordination among 
risk stakeholders at all levels to improve checks throughout the risk 
management cycle, (2) making refinements to threat data by requesting 
the Intelligence Coordination Center to provide estimates of capability 
and intent of terrorist groups, (3) including time horizons for various 
scenarios, and (4) leveraging independent assessments conducted by 
different subject matter experts as a way of checking its risk 
assessment work. 

Evaluation of Alternatives: Ability to Evaluate Alternatives Is 
Greatest at the Local Level: 

Just as the Coast Guard's ability to assess risk is stronger at the 
individual port level than across ports, its ability to evaluate 
various alternatives for addressing these risks is greater at the port 
level as well. PS-RAT was specifically developed to help local Captains 
of the Port concentrate and prioritize their resources and evaluate 
alternative methods of risk reduction. Data from PS-RAT help identify 
vulnerabilities within a port and can be used in improving security 
measures related to the area maritime security plans. PS-RAT is not 
designed to work, however, above the port level. At the national level, 
the Coast Guard has conducted qualitative evaluations of the potential 
benefits of various alternatives for reducing risk levels, such as 
improved information sharing through the use of interagency operational 
centers, waterborne patrols, and escorting ships. In addition, it is 
assessing the potential reduction in risk of different strategies for 
improving awareness of the maritime domain. 

The effectiveness of such evaluations, both within and between ports, 
is influenced to a large degree by the performance standards and goals 
that are set, as well as the reliability and completeness of the risk 
assessments that are conducted. To the extent that goals are missing 
and risk assessments produce data that do not completely and reliably 
depict threats and vulnerabilities, the prospective evaluation of 
benefits and costs of future mitigation strategies may not target the 
areas with the greatest gaps or lead to the most cost-effective 
decisions. 

In examining various alternatives, a key consideration will be 
measuring the overlapping benefits of different strategies as a means 
of developing data on what set of alternatives may provide the most 
improved security in return for the resources expended. These 
overlapping benefits and costs may involve Coast Guard actions, such as 
expanding the number of operational centers at port locations or 
procuring new ships and aircraft, or actions by others, such as more 
inspections of container cargo shipments or developing better 
intelligence capability. The Coast Guard's evaluation of alternatives 
generally involves examining its own possible actions, such as 
developing proposals to expand the number of operational centers, 
without placing the cost of such actions in context with other 
alternatives that have already been deployed or that could be used. In 
discussions with us, Coast Guard officials said addressing this issue 
was part of the Coast Guard's efforts to further refine its risk 
management approach. For example, the Coast Guard is working toward 
examining alternatives by assessing the degree to which they reduce 
risk in exchange for the cost involved. 

Management Selection: Local Strategies Have Been Selected but Would Be 
Better Informed by Improvements in Phases for Goal Setting and Risk 
Assessment: 

In regard to management selection--the fourth phase of the framework-- 
the Coast Guard has used its port-level assessments to select specific 
mitigation strategies to manage vulnerabilities in and around 
facilities. For example, local Captains of the Port have used the 
assessment information in coordination with input from local 
stakeholders to (1) establish security zones around key port 
infrastructures; (2) improve security in and around passenger vessels; 
and (3) coordinate security improvements, such as fences, cameras, and 
barriers around port infrastructures. At the national level, the Coast 
Guard is designing and planning to implement an array of radar systems, 
sensors, and information systems to identify and track possible threats 
in the maritime domain. One element of this effort is the establishment 
of maritime intelligence fusion centers that cover the Pacific and 
Atlantic coasts. These fusion centers are providing intelligence to the 
Coast Guard intelligence network in the field, and the centers share 
information with interagency partners, such as the Navy and Customs and 
Border Protection. 

In general, progress in this area is affected heavily by the same 
factors affecting progress in evaluating alternatives: progress in 
setting measurable performance objectives and improving the reliability 
and completeness of risk assessments. The various phases of the risk 
management model build and rely on one another, and in this case the 
quality and reliability of the results are heavily affected by the 
quality and reliability of efforts in the first two phases. 

Another major challenge will be to strategically integrate risk-based 
data into other management systems, such as the annual cycle of program 
and budget review for assessing how to deploy resources among ports. 
Coast Guard officials acknowledged that using risk-based data to inform 
the annual budget cycle is an important step and are taking further 
steps to address them. For example, the Coast Guard's MSRAM tool will 
compare risks at one port with those at another port, and the data 
produced could inform the Coast Guard's annual budget review of 
programs and resource allocation. In addition, based in part on our 
discussions with Coast Guard officials, the Coast Guard's resource 
planning guide for fiscal years 2008-2012 calls attention to the 
importance of joining these processes. The guide states that by 
"employing principles of risk management and using our understanding of 
strategic risks to Coast Guard mission performance, we will be able to 
make better decisions regarding investment, re-investment and base 
management priorities." 

Implementation and Monitoring: A Foundation for Continuous Improvement 
Is in Place; Challenges Remain in Refining It and Developing Formal 
Policies: 

The Coast Guard has made progress in the fifth phase of the framework-
-implementation and monitoring--by improving existing tools and systems 
in a variety of ways. For example: 

* The Coast Guard has improved PS-RAT; version 2, developed in November 
2002, by providing such improvements as greater detail on consequence 
data. Additionally, the MSRAM will include improvements in the quality 
of threat information in its analysis. 

* The Coast Guard used the Port Security Assessment Program to further 
refine area maritime security plans by offering intelligence and other 
information to the local Captains of the Port on security issues the 
plan may not have covered. 

* In November 2004, the Coast Guard started conducting the National 
Maritime Strategic Risk Assessment to obtain comments and feedback from 
Coast Guard area and headquarters staff on its mission areas, including 
its port security mission. Among other things, the staff provided input 
on ways that the Coast Guard could improve the manner in which it 
carries out its port security mission and reduce any gaps in coverage. 
For example, staff identified opportunities for leveraging resources, 
such as strengthening the partnership with nonfederal stakeholders 
involved in maritime security, as well as potential gaps in the port 
security mission, such as training, equipment, and technology. This 
information was summarized by a Coast Guard consultant and was provided 
to managers in Coast Guard headquarters. 

In some cases, this progress is limited to a degree by a lack of a 
formal policy for taking action on feedback that is received about ways 
to improve the risk assessment and management approach. For example, 
when the Coast Guard receives feedback from field staff on ways to 
improve its strategic approach, there are no formal policies for 
addressing the issues raised. Without such policies, there is little 
assurance that the feedback received will inform decisions to improve 
risk management practices and ultimately port security. For example, a 
Coast Guard official at one port noted that while there are informal 
communication channels to offer feedback on the PS-RAT, there are no 
formal procedures for doing so, and that such procedures would be 
useful in addressing concerns of local users when using the tool. 
Recognizing that there was no formal feedback process for the PS-RAT, 
the Coast Guard developed an ad hoc methodology to gather feedback from 
the users of the tool. In March 2005, Coast Guard headquarters hosted a 
workshop to obtain input from Coast Guard field offices on ways to 
improve the PS-RAT. The workshop resulted in a list of possible 
modifications to the PS-RAT and a plan to develop the MSRAM. Coast 
Guard officials acknowledge that a formal feedback process would be 
beneficial, and they plan to include one as part of the Coast Guard's 
MSRAM. 

Identifying and addressing organizational barriers to the Coast Guard's 
ability to improve or carry out its risk management approaches is a 
final consideration for the monitoring and implementation phase. One 
key organizational challenge is building and sustaining expertise and 
skills for effectively designing and using the risk management tools, 
techniques, and models necessary for managing the Coast Guard's efforts 
in carrying out its port security and maritime domain awareness 
responsibilities. The Coast Guard provides training on risk-based 
decision making at its training center. Recently, the Coast Guard 
revised its officer evaluation form by including "risk assessment" as a 
key leadership competency for its officers. Partly on the basis of the 
briefings we provided to agency officials, the Coast Guard plans to 
carry out workshops to examine ways to integrate risk management into 
the PWCS strategic plan and activities. Applying risk management tools 
and techniques is a complex undertaking, and it requires a managed 
effort to maintain and build organizational expertise and skills to do 
the job well. 

Conclusions: 

The foundation the Coast Guard has established for risk management is 
generally sound. It represents a strong commitment on the Coast Guard's 
part to using risk management effectively in making decisions. Further, 
the Coast Guard, often acting on discussions held during the course of 
this review, has actions under way to address many of the concerns we 
identified. The most difficult work ahead revolves around 
systematically integrating risk-based information into management 
systems, including the annual cycle of program review, that can help 
inform decisions about how to deploy resources and security measures 
among ports and to examine the value of new programs in addressing 
security gaps that remain. In this regard, the Coast Guard's efforts 
are still in the early phases of development. As the Coast Guard moves 
forward, it is especially important that the agency develop ways to 
establish a stronger linkage between the various local and national 
risk assessment efforts under way. For example, area maritime security 
plans are not comparable, and security risks at one port location 
cannot be compared with the risks identified at another location. As a 
result, the collective value of these individual efforts is diminished. 
Developing ways to establish a stronger linkage would likely increase 
the value of the work. 

Recommendations for Executive Action: 

We are not making recommendations in those areas where the Coast Guard 
has actions well under way. The recommendations below are designed 
primarily to spotlight those areas in which additional steps are most 
needed to strengthen agency efforts to implement a risk management 
approach to the Coast Guard's port security activities. Accordingly, we 
recommend that the Secretary of Homeland Security direct the Commandant 
of the Coast Guard to take action in the following two areas: 

* Risk assessment: Develop plans to establish a stronger linkage 
between local and national risk assessment efforts. This effort could 
involve strengthening the ties between local assessment efforts, such 
as area maritime security plans, and national risk assessment 
activities. 

* Alternatives evaluation and management selection: Ensure that 
procedures for these two processes consider the most efficient use of 
resources. For example, one approach involves refining the degree to 
which risk management information is integrated into the annual cycle 
of program and budget review. 

Agency Comments and Our Evaluation: 

In commenting on a draft of chapter 2, DHS, including the Coast Guard, 
generally agreed with our findings and recommendations. DHS said that 
the report notes that the Coast Guard has made progress in all five 
risk management phases and is taking action to address the challenges 
that remain. In addition to commenting on our recommendations, the 
Coast Guard provided several technical comments under separate cover, 
and we revised the report when appropriate. Written comments from DHS 
are in appendix II. 

[End of section] 

Chapter 3: Stronger Risk Management Approach Could Improve the 
Accountability of the Port Security Grant Program: 

The Office for Domestic Preparedness within the Department of Homeland 
Security has made progress in applying risk management to the port 
security grant program but faces challenges in strengthening its 
approach, as demonstrated in part by its experience in awarding past 
grants. Examples of progress--and challenges--can be found across all 
five risk management phases (see table 6). ODP has established overall 
goals for the grant program but faces challenges in setting specific 
and measurable program objectives, in part because this effort hinges 
on similar action by other federal agencies. ODP's progress, with input 
from the Coast Guard and IAIP, has been greatest in conducting the 
actual risk assessments. It assessed 129 ports and, using risk-based 
prioritization, narrowed to 66 the number of ports eligible to apply 
for fiscal year 2005 grants. Its methods for assessing risks and 
evaluating mitigation alternatives, however, still are limited in their 
ability to prioritize relative risks across ports or to calculate the 
costs and benefits of various alternatives. Finally, while ODP has also 
made progress in developing a risk-based grant selection process and 
mechanisms to monitor what the grants accomplish, it has not always 
completely relied on a risk-based process. For example, grant awards 
for fiscal year 2004 illustrate the trade-offs that occur when 
attempting to award grants to applicants that are at greater risk and, 
at the same time, provide funds to applicants that have the larger 
financial need. At the end of what was, in part, a risk-informed 
assessment process, ODP change the criteria for awarding grants when it 
decided to give lower priority to grant applications involving projects 
at large companies, on the assumption that these companies were better 
able than other entities to pay for their own security improvements. 
This changed 40 percent of the grants originally recommended for an 
award, as projects with higher risk but greater potential for self- 
funding gave way to lower-risk projects with more limited funding 
prospects. ODP changed the process for fiscal year 2005 by clarifying 
the criteria it would use in awarding grants and by requiring 
applications from private entities to match at least 50 percent of the 
total amount requested. 

Table 6: Summary of Progress and Challenges in the Port Security Grant 
Program: 

Risk management phase: Strategic goals, objectives, and constraints; 
Examples of progress made: High-level strategic goals have been set for 
the port security grant program; 
Examples of remaining challenges: The program is missing measurable 
program objectives to show the progress that has been made. 

Risk management phase: Risk assessment; 
Examples of progress made: In its May 2005 guidelines, ODP has 
prioritized spending decisions by identifying 66 key seaports that are 
eligible for awards, and it is placing greater reliance on the use of 
risk assessments at port locations.[A]; 
Examples of remaining challenges: Threat, vulnerability, and 
consequence data have limitations. The degree to which risk assessments 
compare and prioritize risk across ports remains a challenge. 

Risk management phase: Alternatives evaluation; 
Examples of progress made: Unlike efforts in previous years, the fiscal 
year 2005 effort examined alternative solutions proposed by nonfederal 
stakeholders and ODP, and the Coast Guard assessed the cost and 
benefits of the projects; 
Examples of remaining challenges: The degree and extent to which 
proposals can be accurately evaluated and benefits calculated for risk 
reduction remains uncertain. ODP is working with the Coast Guard to 
deal with this challenge. 

Risk management phase: Management selection; 
Examples of progress made: For fiscal year 2005, criteria for 
management selections include the prioritization of projects based on 
the criticality of ports and proposals that reduce vulnerabilities to 
certain threat scenarios. These risk-based criteria were not used in 
prior fiscal years; 
Examples of remaining challenges: For fiscal year 2004, internal 
controls for documenting management decisions were not followed. For 
fiscal year 2005, ODP documented grant award decisions in a database. 

Risk management phase: Implementation and monitoring; 
Examples of progress made: In fiscal year 2005, ODP has made a number 
of improvements to better monitor implementation of its risk management 
process; 
Examples of remaining challenges: For fiscal year 2004, additional 
improvements were needed to obtain formal feedback from grant program 
stakeholders. For fiscal year 2005, ODP has developed formal feedback 
loops from grant program stakeholders. 

Source: GAO analysis of ODP's port security grant program. 

[A] The Coast Guard provided various data on ports, such as amount of 
total cargo, domestic cargo, international cargo, number of passengers 
using ferries, and number of passengers using cruise ships. ODP 
prioritized ports by evaluating these data to determine a list of the 
66 highest-risk port areas. 

[End of table] 

ODP Manages Port Security Grants: 

The port security grant program was established in fiscal year 2002 
under the purview of the Transportation Security Administration (TSA), 
which became part of DHS in March 2003.[Footnote 25] Because of 
organizational changes within DHS, the grant program has been 
administered by ODP within the Office of State and Local Government 
Coordination and Preparedness since May 2004. ODP was transferred from 
the Department of Justice to DHS upon passage of the Homeland Security 
Act of 2002. 

The grant program provides assistance to nonfederal stakeholders for 
making security improvements at the nation's ports. During fiscal years 
2002-2004, grants from the program totaled about $560 million and 
covered such concerns as more fencing, cameras, and communications 
equipment. For fiscal year 2005, the appropriations act for DHS 
provided $150 million for port security grants.[Footnote 26] The 2005 
program focused on three primary concerns: (1) protection against 
improvised explosive devices carried by small craft, underwater craft, 
or vehicles; (2) enhanced explosives detection capabilities for the 
owners and operators of vehicle ferries and associated facilities (as 
shown in fig. 3); and (3) facility security enhancements in the 
nation's highest-risk ports. The program is designed to operate in 
coordination with federal partner agencies and industry.[Footnote 27] 
Grantees are selected through a competitive process. 

Figure 3: Facilities at One of the Nation's Major Ports: 

[See PDF for image] 

[End of figure] 

For fiscal year 2006, the Office of Management and Budget had proposed 
consolidating the port security grant program with other homeland 
security grant programs. Known as the Targeted Infrastructure 
Protection Program, the program would have consolidated funding for 
ports, transit, and other critical infrastructure into one program. 
However, DHS's appropriations act for fiscal year 2006 maintained 
separate funding for the port security grant program. In particular, 
the act provided a $175 million appropriation for the port security 
grant program that, "shall be awarded based on risk and 
threat."[Footnote 28] 

In January 2005, the DHS Office of the Inspector General issued a 
report on the Port Security Grant Program that covered the program's 
second and third rounds of grants--through fiscal year 2003.[Footnote 
29] This report made a number of recommendations, and in response, ODP 
initiated a number of changes to the program, according to ODP grant 
program officials. Our discussion of the grant program reflects the 
changes ODP had made at the time of our report. 

Goal Setting: Overall Goals Have Been Set; Developing Performance 
Measures and Leveraging Federal Dollars Remain Challenges: 

Progress has been made on setting goals--the first phase of GAO's risk 
management framework--for the port security grant program. Congress and 
the Administration have laid out broad policy goals for maritime 
security and for the program. Congress's stated purpose in establishing 
the program was to finance the costs of enhancing facility and 
operational security at critical national seaports.[Footnote 30] The 
Administration has set the program in the context of the December 2004 
Presidential Directive on Maritime Security Policy, which cites several 
broad policy goals for maritime security, including preventing 
terrorist attacks in the maritime domain and reducing vulnerability to 
such attacks; protecting U.S. population centers, critical 
infrastructure, borders, harbors, ports, and coastal approaches; and 
maximizing awareness of security issues in the maritime domain in order 
to respond to identified threats.[Footnote 31] 

DHS's application guidelines for fiscal year 2005 grants reflect the 
context of these broad policy goals. They state that the program 
reflects congressional and executive intent "to create a sustainable 
effort for the protection of critical infrastructure from terrorism, 
especially explosives and nonconventional threats that would cause 
major disruption to commerce and significant loss of life," and they 
link the program to specific national priorities specified in the 
nation's security planning framework.[Footnote 32] Other ways in which 
the 2005 grant program reflects a goal-oriented approach are its 
efforts to apply the grants to locations that are viewed as the 
nation's highest-risk ports (discussed in more detail below) and to 
focus the grants on such specific concerns as protection against 
improvised explosive devices. 

While broad policy and program goals have been set, challenges to 
further progress on this risk management phase take two main forms. The 
first is translating the program's broader goals into measurable 
objectives. One difficulty in doing so is that other federal partner 
agencies have yet to spell out measurable objectives at the national 
level as related to protecting key infrastructure. For example, as 
discussed in chapter 2, the Coast Guard's Maritime Strategy for 
Homeland Security describes strategic approaches and priorities, but it 
does not include measurable objectives as part of its 
approach.[Footnote 33] In addition, DHS has yet to set performance 
measures for programs related to the implementation of programs for 
protecting critical infrastructures--another program area that the 
grant program supports.[Footnote 34] We discuss this further in chapter 
4. 

A second challenge involves determining an appropriate way to consider 
two different federal concerns about grant programs: ensuring that 
grants address key needs while at the same time ensuring that they make 
the most efficient use of federal dollars. This challenge exists for 
several reasons: 

* Federal and nonfederal partnership for addressing key needs. First, 
the federal government is not the only potential source of revenue for 
addressing security needs. Ports are often a complex mixture of public 
sector and private sector infrastructure. For example, public entities 
such as port authorities or local and state governments may own or 
operate seaport facilities and roadways, while private companies and 
interests may own and operate factories, warehouses, oil refineries, 
and railways.[Footnote 35] Ports can produce benefits that are public 
in nature (such as general economic well-being) and distinctly private 
in nature (such as generating profits for a particular company). The 
public benefits they produce can also be distinctly local in nature, 
such as sustaining a high level of economic activity in a particular 
state or metropolitan area. Thus, state and local governments, like 
private companies, also have a vested interest in ensuring that their 
ports can act as efficient conduits of trade and economic activity. 
Given that homeland security threats can imperil this activity, it can 
be argued that all of these stakeholders should invest in the continued 
stability of the port. 

* Leveraging federal dollars. Second, in many federal grant programs, 
the desired outcome is that federal grants supplement what other 
stakeholders are willing to spend. If a grant program is not designed 
to encourage supplementation, the danger is that other stakeholders 
will rely solely on the federal funds and choose to use their own funds 
for other purposes. This practice is known as substitution, and the net 
result is that limited federal funds cannot be stretched, or leveraged, 
to the degree they otherwise could be. In prior work addressing this 
issue in certain other grant programs, we found that on average, every 
additional federal grant dollar resulted in about 60 cents of 
substitution.[Footnote 36] 

Although the design of a grant program is not part of the risk 
management framework, it is an important issue because it is one key to 
accomplishing the dual aims of targeting funds to projects that address 
the highest risk while discouraging the replacement of state, local, 
and private funds with federal money. ODP's approach for 2005 has been 
to formalize a matching requirement for private sector stakeholders but 
not for public sector stakeholders. For fiscal year 2005 grants, ODP 
required that applications from industry match at least 50 percent of 
the total amount requested. 

This situation illustrates the complexity of addressing the most 
significant security needs while considering the degree to which 
nonfederal stakeholders should share in the cost of security 
improvements. For fiscal year 2005, the program encourages but does not 
require public sector or nonprofit entities to match federal funds, 
according to ODP officials. Depending on the value placed on reducing 
the substitution of federal funds for local funds, the design of the 
port security grant program offers a way to improve the fiscal impact 
of federal dollars. There is disagreement among policymakers about 
where the emphasis should be on this aspect of grant programs. Some 
might see the substitution of federal funds for local funds as 
reasonable given differences in fiscal capacity, while others may view 
homeland security as a shared fiscal responsibility. If policymakers 
place greater value on reducing the substitution of federal funds for 
local funds, strengthening matching requirements for such entities 
offers one option. The 2006 appropriation for the port security grant 
program includes a federal matching requirement.[Footnote 37] ODP has 
not yet issued its 2006 port security grant guidance clarifying how it 
will implement this requirement. One way to implement the requirement 
involves using a sliding scale for matching federal funds depending on 
the fiscal capacity of the grant applicant. Such a scale could range, 
for example, from an 80 percent matching requirement for Fortune 500 
companies to a 25 percent matching requirement for those entities that 
have less in monetary resources.[Footnote 38] 

Risk Assessments: The Funding Distribution Model Is Becoming Better 
Able to Consider Risk, but Methodological Challenges Remain: 

ODP has made progress in carrying out risk assessments--the second part 
of the risk management framework--but the progress made is balanced by 
the additional methodological challenges that remain. ODP's progress is 
reflected in changes made to the program for fiscal year 2005, in both 
port-level and national-level assessments. Among other things, ODP has 
placed greater emphasis on using threat, vulnerability, and consequence 
data in prioritizing applications and port locations. Nonetheless, 
various methodological challenges remain in the assessment tools or the 
assessments themselves. For example, ODP is still limited in its 
ability to compare and prioritize applications from one port with those 
from a different location. 

Procedures for the Fiscal Year 2005 Port-Level and National-Level 
Assessments Have Been Strengthened: 

At the port level, a key difference in ODP's fiscal year 2005 grant 
procedures is that input from other stakeholders plays a more prominent 
role in the award decisions. ODP obtained such input in prior years but 
did not formally consider it in making decisions. For example, in 
making its determinations for fiscal year 2004 grants, ODP sought input 
from the Coast Guard Captains of the Port and from regional directors 
of the Maritime Administration. These officials reviewed and ranked 
port security grant applications based on their knowledge of the port 
location and the results of various assessments, including the PS-RAT. 
ODP officials said they considered this input, but they did not 
integrate rankings into the evaluation forms used in the ODP assessment 
process. In this regard, the DHS IG found that ODP should place greater 
emphasis on risk reduction as part of the grant review 
process.[Footnote 39] In responding to the IG's findings, for fiscal 
year 2005, the rankings made by Coast Guard, Maritime Administration, 
and state officials are part of the formula ODP uses to make final 
decisions on grant awards.[Footnote 40] Additionally, the 2005 program 
places greater emphasis on applications that are consistent with area 
maritime security plan priorities. 

ODP's adjustments to its fiscal year 2005 procedures are even greater 
at the national level, where it has made a concerted effort to narrow 
the program to ports of greatest concern, and to use threat, 
vulnerability, and consequence data to rank and prioritize both ports 
and applications. For grants made in fiscal year 2004, ODP considered 
applications from all ports in making awards. For the 2005 program, ODP 
worked with IAIP and the Coast Guard to develop a list of eligible 
ports. The agency first identified the largest ports based on volume. 
According to ODP officials, a risk formula was developed to rank each 
of the ports. In May 2005, ODP determined that, on the basis of this 
assessment, 66 of the 129 largest-volume ports across the country were 
eligible for grant awards.[Footnote 41] ODP further prioritized the 66 
ports by dividing them into four tiers--tier 1 representing those ports 
with the highest risk and tier 4 representing ports with the lower 
risk. ODP provided a set amount of money to each tier, and ports in 
these tiers competed against each other for funding. In carrying out 
its analyses, ODP also placed greater reliance on threat, 
vulnerability, and consequence information. 

Beyond reducing the number of ports eligible for the grant program, 
another change ODP made at the national level was to prioritize 
possible threat scenarios that grant funds should address, using input 
from Coast Guard officials. ODP has given priority to applications that 
prevent or detect threats arising from improvised explosive devices. 
Such devices pose a threat to transportation systems across the nation 
and have been used by terrorists in the past. Specifically, for the 
2005 grant program, ODP has given priority to the following threat 
scenarios: 

* preventing and detecting improvised explosive devices delivered via 
small craft, 

* preventing and detecting underwater attacks from such devices, and: 

* preventing and detecting vehicle-born improvised explosive devices on 
ferries. 

Challenges Remain for Improving Methods and Data: 

ODP's assessment methodology, while improved, still faces challenges. 
Progress in using risk assessment data to manage the grant program has 
limitations in methods and data for informing award decisions. These 
challenges, if addressed, would provide decision makers with more 
reliable and complete data on which to prioritize and award grant 
funds. The challenges fall into two main categories: (1) incomplete 
threat, vulnerability, and consequence data and (2) methodological 
inability to fully compare grant applications from one port with those 
from another port. 

Incomplete Data on Threats, Vulnerabilities, and Consequences: 

Our review of ODP's risk assessment approach and our discussions with 
ODP and Coast Guard personnel identified several challenges related to 
data on threats, vulnerabilities, and consequences (see table 
7).[Footnote 42] Many of these challenges mirror the challenges faced 
by the Coast Guard that we described earlier in chapter 2. 

Table 7: Examples of Data-Related Challenges in ODP Risk Assessments: 

Data type: Threats; 
Summary of challenge: Data on threats provide a useful starting point. 
However, ODP is challenged, just as the Coast Guard is, in conducting 
risk assessments without data on the relative likelihoods associated 
with various threat scenarios. Information from the intelligence 
community on such things as the presence of national or international 
terrorist cells, and on the capability and intent of terrorist groups 
as they relate to types of scenarios and the specific infrastructure 
type, would enhance ODP's ability to model relative probabilities of 
threat scenarios. 

Data type: Vulnerabilities; 
Summary of challenge: ODP's assessment of vulnerabilities does not take 
into account actions that have already been taken to reduce 
vulnerabilities (such as more patrols, fencing, and guards). While ODP 
has prioritized certain scenarios, the vulnerability assessments are 
not linked to specific threat scenarios, thus limiting the value of 
vulnerability scores. 

Data type: Consequences; 
Summary of challenge: Similar to its assessment of vulnerabilities, 
ODP's assessments of consequences are not linked to specific threat 
scenarios. Instead, ODP uses values such as population density near a 
port or cargo tonnage to measure consequences. 

Source: GAO analysis of ODP's risk assessment for the port security 
grant program. 

[End of table] 

The following examples illustrate the challenges posed by the 
limitations in threat, vulnerability, and consequence data. 

* Limitations in threat-related data. ODP's current characterization of 
threat cannot be interpreted as an estimate of relative probability of 
threat scenarios--a key element of risk assessment--in and around 
ports.[Footnote 43] The problems we pointed out in chapter 2 about 
threat data are applicable here as well: The threat information 
available from intelligence agencies does not provide the type of data 
that could be used to more fully inform ODP's decisions. More complete 
data would include such things as information on the presence of 
national or international terrorist cells, the capability and intent of 
terrorist groups as they relate to types of attack, and on the specific 
infrastructure types that have been attacked. ODP officials said that 
the scarcity of threat data limits their ability to inform the decision-
making process and that decisions are based on the best available 
combination of data and expert judgment. Because threat data are 
limited, ODP bases its models on several proxies for risk. For example, 
ODP used volume of cargo that moves into and out of ports as a way to 
develop its list of the 129 ports that could initially be candidates 
for the grant program. ODP then assessed the relative risk at these 
ports by using, among other things, a threat variable represented by 
the number of "credible threats and incidents" and the number of 
"vessels of interest" (i.e., suspicious vessels) that use a port 
facility. These data originated from the intelligence community, the 
Coast Guard, and IAIP. ODP recognizes, however, that threat data on 
ports are scarce, and data on the number of credible threats and 
incidents and vessels of interest may not represent actual threats, but 
instead could also represent other law enforcement problems, such as 
illegal migrants or theft of goods and merchandise. The challenges in 
developing reliable data on probability affect the overall risk 
assessment for a port area, given a specific attack scenario. Without 
data or models that measure the relative probability of various threat 
scenarios, ODP may not target the most significant security concerns. 

Limitations in vulnerability-related data. As we described in chapter 
2, the Coast Guard's PS-RAT excludes from its analysis reductions in 
vulnerabilities resulting from security measures that have already been 
taken by the Coast Guard, such as inspecting passenger vehicles that 
board ferries, escorting high-interest vessels, and establishing 
security zones around the port. ODP's assessment of vulnerabilities 
involves specific aspects of individual port areas themselves, namely, 
data on the length of the channel leading to a port, the number of port 
calls by all ships, and the number of tankers that use a port. While 
such data are representative of the intrinsic vulnerabilities of a 
port's location, they do not include such factors as guards, fences, 
and cameras that are already in place; security zones that have been 
established; or escorts of high-interest vessels that occur. As a 
result, the assessment may overstate vulnerabilities for port 
locations. Also important is the fact that ODP's approach to evaluating 
port area vulnerabilities excludes consideration of specific threat 
scenarios. For example, while they have identified certain threat 
scenarios as priorities, the vulnerability indicators are not linked 
directly to scenarios, such as the use of improvised explosive devices. 
Without this linkage to the threat component of the risk assessment 
process, ODP's approach to vulnerability assessments may not consider 
certain factors, such as the number of areas in a port location where 
recreational vessels are unmonitored, and how such factors may be 
conducive to certain threat scenarios, such as loading improvised 
explosive devices on such vessels. 

Limitations in consequences-related data. The values used by ODP to 
describe the consequences of a terrorist attack may not accurately 
depict the damages resulting from a terrorist attack. In valuing the 
consequences of an attack, ODP focuses on people-related, economic- 
related, and national security-related measures.[Footnote 44] For 
example, ODP uses maximum population density within 10 miles of a port 
and the average number of daily ferry passengers to estimate the 
consequences of a terrorist attack at a port, and it aggregates these 
measures. Similarly, ODP uses international and domestic tonnages, the 
amount of containerized cargo, and the dollar value of foreign trade to 
measure the economic consequence of a terrorist attack. While these 
provide a useful starting point, there are two issues related to this 
approach. First, without linking these data to the relative probability 
of various threat scenarios, ODP's estimate of consequences is limited 
to inherent characteristics of a port rather than the estimated 
consequences of various attack scenarios. In contrast, agencies such as 
the Coast Guard estimate the number of lives lost and assign a dollar 
value to the loss of human lives. Second, ODP aggregates the estimated 
number of lives lost with the dollar value of foreign trade. Doing so 
raises questions about the reliability and meaning of the final output. 

Methodological Limitations: 

As was the case with the Coast Guard, a key methodological limitation 
affects one goal of risk assessments--informing decision makers on 
relative risks across port locations. This limitation relates directly 
to the ability to compare the relative risks of facilities at one port 
with those at another port. When field review teams rank and prioritize 
grant applications, they use several sets of data, including 
information provided on the application, personal knowledge of the 
port, and the results of the PS-RAT, which gives ranking information 
for a given port area on vulnerability and consequence.[Footnote 45] As 
discussed in chapter 2, PS-RAT, however, cannot now be used to compare 
the risks at one port with those at another. PS-RAT allows ODP to 
compare and prioritize key infrastructure within a port, but it does 
not produce a risk ranking that permits ODP to compare and prioritize 
infrastructure across ports. Coast Guard efforts to develop a tool that 
examines relative risks across ports will aid ODP in addressing this 
limitation. 

Alternatives Evaluation: ODP Recognizes the Importance of Evaluating 
Alternatives, but Tools for Doing So Are Limited: 

Evaluation of alternatives--the third phase of GAO's risk management 
framework--is an area that ODP recognizes as being an important part of 
awarding grants, and the changes for fiscal year 2005 represent 
progress in this area. One change for fiscal year 2005 involves 
additional steps to consider benefits and costs. When ODP asks local 
Coast Guard Captains of the Port to review applications, one criterion 
it asks them to apply is which projects offer the highest potential for 
risk reduction for the least cost. For 2005, ODP plans to augment 
evaluation by conducting an analysis of costs and benefits of the 
project (which considers the potential for risk reduction), and it 
shares the results with the Coast Guard. 

As part of this assessment, ODP plans to break out applications from 
the 66 eligible port locations into four tiers and give applications 
from ports that are in higher tiers more priority and more money. Port 
areas with the highest risk are assigned to tier 1 and port areas with 
the lowest risk are assigned to tier 4. 

ODP's ability to assess proposed security improvements, like the Coast 
Guard's, is influenced by the program goals and performance measures 
that the component sets and the reliability and completeness of the 
risk assessments that it carries out. However, when measurable 
objectives are missing, the degree to which security gaps remain and 
the extent to which progress has been made remain unclear. Similarly, 
while PS-RAT provides a starting point for evaluating the proposed 
measures and the extent to which the measure narrows security gaps 
within a port, it was not designed to compare and prioritize relative 
risks from one port to relative risks in a different port. This 
condition limits ODP's ability to compare the benefits of proposed 
security measures from an applicant at one port location with benefits 
of proposed measures at a different port. 

Management Selection: ODP Has Addressed Problems in Documenting 
Differences between Initial Selection Recommendations and Final 
Selection Decisions: 

Earlier in this chapter, we discussed the importance--and difficulty-- 
of balancing potentially conflicting goals, such as ensuring that funds 
are directed to projects of the greatest risk while at the same time 
stretching limited federal dollars to the maximum degree. ODP's 
selection of grants for 2004 illustrates the challenges in applying a 
risk-based approach to awarding grants and the trade-offs involved in 
attempting to balance risk and financial need as criteria. In order to 
achieve accountability, federal standards for internal controls require 
that all transactions and other major events need to be documented. 
Basically, about 40 percent of ODP's final selection decisions were 
different from the initial recommendations of lower-level evaluators, 
without documentation from reviewers explaining why they disagreed with 
the initial recommendations. According to officials involved in the 
program, before the final selections were made, the Secretary of 
Homeland Security issued guidance indicating that Fortune 500 companies 
should be able to pay for their own security improvements and that 
ferries and port authorities should receive higher priority in the 
final award decisions than other applicants. ODP officials said that 
the fact that they followed this new guidance affected the final 
ranking of grant applicants. The tension between self-funding and 
security priorities illustrates the need for effective internal 
controls to ensure that procedures are followed and that the resulting 
selection decisions are transparent and clearly documented. ODP has 
taken action to address this problem. 

In the 2004 grant award process, ODP's initial assessments resulted in 
recommendations for funding 154 specific proposals.[Footnote 46] 
However, our analysis of the final grant awards showed that about 40 
percent of the initial 154 applications that were recommended for a 
grant award by lower-level evaluators that examined grant applications 
did not receive an award. Table 8 shows examples of the types of 
changes that occurred. 

Table 8: Examples of Changes in Funding Decisions: 

Examples of projects initially recommended for funding but ultimately 
not approved: One applicant requested grant funds to improve security 
involving the construction of two main entrance access barricades, 
perimeter lighting, and additional cameras. The applicant offered to 
share costs--about 20 percent of the $468,000 that it was requesting. 
After its initial review, ODP staff ranked this application as the 
third highest and recommended it for an award. The final decision 
ranked this applicant 228 out of 287 applicants, and the application 
did not receive an award. According to staff familiar with this 
project, the applicant received a lower ranking because it was a 
Fortune 500 company. 

Examples of projects initially recommended for funding but ultimately 
not approved: Another applicant requested $225,000 in grants to 
purchase cameras, fencing, and barricades around its facility, and it 
committed to matching the requested amount. The initial headquarters 
review recommended that the applicant receive an award and it ranked 
the project as 25th out of 287 applicants. Local Coast Guard and MARAD 
officials ranked this project as their top priority for the port, 
noting that this chemical plant produces material that is highly 
hazardous and that improving security at this facility had the Coast 
Guard's "highest recommendation." This applicant did not receive an 
award and it was ranked 236 out of 287 applications because it was a 
Fortune 500 company. 

Examples of projects initially recommended for funding but ultimately 
approved: Another applicant requested a $1.1 million grant to augment 
an existing police department surveillance and camera system of port 
facilities on an around-the-clock basis. The initial headquarters 
review did not recommend this application for an award and it was 
ranked 279th out of 287 applicants. Comments from the initial review 
showed that there was concern the project was not cost-effective 
because "it appeared to duplicate another effort by the port or the 
state and the project provided moderate risk reduction to identified 
vulnerabilities." The reviewers noted that the applicant did not offer 
to share in the cost of the project. However, ODP awarded $800,000 to 
the applicant. 

Examples of projects initially recommended for funding but ultimately 
approved: A fourth applicant requested $1,105,200 to install protective 
film on windows at its terminal location. Grant program procedures 
stated that preference would be given to projects that prevent, deter, 
and detect (an attack) over a project that involved new installation or 
replaced existing infrastructure. On the basis of these guidelines, the 
Coast Guard staff at one port location ranked this application as 50th 
out of 55 applicants in its port zone. According to the staff, they 
were instructed to raise the scores for this application, and as a 
result, the applicant received an award for its application. 

Source: GAO analysis of 2004 grant fund database. 

[End of table] 

These examples illustrate the trade-off between awarding grants to 
applicants that are assessed, in part, based on risk or on providing 
funds to applicants that have a financial need. The net result is that 
when federal funds are used in this fashion, they may not address the 
most severe security gaps in and around ports because there is no 
guarantee that private sector firms will spend their own funds for 
security improvements since it is unclear whether there are incentives, 
such as minimum standards for security, that would motivate them to do 
so.[Footnote 47] The competing goals of addressing the most significant 
security needs and providing financial assistance to those entities 
with less fiscal capacity call attention to the difficulty of achieving 
a balance between these objectives and the need for selection decisions 
that clearly document the trade-off. 

In order to achieve a transparent process for accountability purposes, 
federal standards for internal control require that all transactions 
and other significant events need to be clearly documented, and that 
documentation should be readily available for examination.[Footnote 48] 
However, internal control procedures for documenting decisions, 
including changes in project ranking, were not followed. The result is 
that the rationale for award decisions was not always available. In 
cases where an applicant's ranking would change by over 100 places, 
there was no documentation that described the reason for the change. In 
several cases where changes such as this occurred, it was noted that 
the final selection board concurred with the original ranking, but no 
reason was provided for the reprioritization. DHS's IG review also 
found this lack of documentation. The IG recommended that the reviewers 
be required to document their decisions in the grants management 
system, particularly when the decisions are inconsistent with 
recommendations from a lower level of review. DHS generally concurred 
with the IG's recommendations and stated that it would require 
reviewers to document their decisions as part of the 2005 grant 
program. Our work showed that in the fiscal year 2004 grant program-- 
the most recent round prior to the issuance of the IG's report--such 
documentation was still missing. For fiscal year 2005, ODP instituted 
additional measures to ensure that decisions were documented as part of 
the review process. 

Implementation and Monitoring: ODP Has Instituted a Monitoring Plan for 
Grant Awards: 

ODP has made substantial progress in creating procedures that address 
the fifth phase of the framework--implementation and monitoring; the 
challenge lies in carrying them out. As we have described in this 
chapter, ODP has taken a number of actions to base spending decisions 
on risk-based data; it has, among other things, (1) developed 
procedures for prioritizing port locations, (2) prioritized threat 
scenarios, and (3) more closely aligned risk assessments to port 
locations. In addition, ODP has coordinated its efforts with the Coast 
Guard, the intelligence community, and other key stakeholders in 
developing the procedures for the 2005 grant program. For the fiscal 
year 2005 grant program, ODP has established procedures for monitoring 
recipients after they have been funded. The monitoring consists of 
follow-up after a project has been implemented to help ensure that the 
project has been implemented in accordance with the grant award, 
including timelines, budgets, and programmatic criteria are being met. 
In addition, ODP requires that progress reports be submitted biannually 
and a final project report be submitted within 120 days after the end 
of the project. 

A challenge that remains this phase of the risk management framework is 
developing processes for feedback to improve the process used in 
awarding grants. For example, our review of the 2004 grant awards show 
that local officials, including Coast Guard officials and grant 
applicants, did not receive feedback from ODP on why projects they 
designated as high priority did not receive funding when lower-priority 
projects did. According to ODP officials, they have instituted 
processes for providing feedback to and obtaining feedback from grant 
program stakeholders as part of the fiscal year 2005 program. 

Conclusions: 

The principles of risk management apply to the port security grant 
program, and the lessons learned call attention to additional actions 
that could build on the progress that has already been made. Depending 
on the value placed on reducing the substitution of federal funds for 
local funds, there are opportunities to further leverage federal 
dollars in the program's design, and by leveraging federal funds, 
additional security needs in and around ports could be addressed. 
Additionally, without having performance measures in place, it is hard 
to gauge what progress has been made and what security gaps remain. The 
development of such measures should offer greater insights on the 
extent to which funds are narrowing security gaps that exist or helping 
to identify security needs that surface. The lessons of the program 
also provide numerous insights into the way that multiple stakeholders, 
such as the Coast Guard and IAIP, contribute to the way in which ODP 
uses risk-based data in administering the program. As we described in 
chapter 2, the Coast Guard has several efforts under way to improve its 
risk assessment of ports, including developing data on the relative 
probability of threat scenarios, improving data on consequences and 
vulnerabilities that are linked to various threat scenarios, and 
comparing risks among ports. ODP should be able to use the results of 
such efforts to help in awarding grants that are consistent with the 
priorities identified. 

Recommendations for Executive Action: 

To strengthen ODP efforts to implement a risk management approach to 
its port security grant program, we recommend that the Secretary of 
Homeland Security direct the Executive Director for ODP to undertake 
the following three actions: 

* Clarify, in its grant guidance, the conditions under which greater 
leveraging of federal dollars should be included as a strategic goal 
for the port security grant program. 

* Develop measurable objectives for managing the grant program's 
progress toward achieving strategic goals and use these measures to 
gauge progress and make adjustments to the program. 

* Coordinate efforts with the Coast Guard and IAIP to use more reliable 
risk assessment data as they become available. At a minimum, such data 
should include (1) the relative likelihood of various threat scenarios, 
(2) consequences and vulnerabilities that are linked to terrorist 
scenarios, and (3) a comparison of risks across ports. 

Agency Comments and Our Evaluation: 

In commenting on a draft of chapter 3, DHS, including ODP, generally 
agreed with the findings and recommendations. Specifically, DHS said 
that the recommendations are reasonable given that most of our review 
took place prior to changes that ODP made to the program in fiscal year 
2005. DHS said that it appreciated our efforts to review the fiscal 
year 2005 port security grant program requirements even though most of 
our work had been completed. DHS stated that several of the 
recommendations have already been addressed, and it noted that the 
remaining ODP-related recommendations will be addressed in the fiscal 
year 2006 port security grant program, at least to the extent possible. 
In addition to commenting on our findings and recommendations, DHS 
provided technical comments under separate cover, and we revised the 
draft report where appropriate. Written comments from DHS are in 
appendix II. 

[End of section] 

Chapter 4: IAIP Faces Challenges in Meeting Risk Management 
Responsibilities across All Sectors of the Nation's Infrastructure: 

The Information Analysis and Infrastructure Protection Directorate 
within DHS faces broad and extensive challenges in meeting its risk 
management responsibilities and thus far has made limited progress. 
Relative to the Coast Guard and ODP, IAIP's risk management 
responsibilities are much wider and more difficult: Instead of 
comparing risks across port assets, it must find ways to compare risks 
at ports with risks in other sectors, such as public health, energy, 
and banking and finance. Challenges remain across all of the phases of 
GAO's risk management framework (see table 9). For the first phase 
(goals and objectives), while IAIP's efforts are anchored to strategic 
goals in various executive branch strategies and an interim national 
plan, IAIP's challenge is to continue developing the national plan to 
provide performance measures and associated steps and milestones. In 
the second phase, IAIP has begun several key risk assessment efforts 
but has had limited success in completing them. For example, IAIP faces 
challenges in developing data on the relative likelihood of various 
threat scenarios--a key part of the assessments it must conduct under 
the Homeland Security Act of 2002--because the information produced by 
the intelligence community is of limited use for risk assessment 
purposes, according to IAIP officials. IAIP has plans to develop such 
data by coordinating its efforts more closely with the intelligence 
community. Additionally, IAIP has yet to successfully complete the 
difficult task of comparing and prioritizing assets within and across 
sectors, but it plans to have an interim assessment done by the end of 
2006. IAIP's challenge in the final three phases (evaluating 
alternatives, selecting approaches, and implementation and monitoring) 
are related heavily to IAIP's unique role: It recommends what the 
security priorities should be to other federal agencies and nonfederal 
stakeholders and recommends how best to address them, but it is largely 
dependent on other stakeholders, public or private, to take any 
actions. The decision to implement security improvements is made by 
stakeholders alone. Moreover, IAIP acknowledges that it can further 
leverage work that has already been done in this area by other federal 
agencies that have regulatory authorities over certain private sector 
infrastructure owners. IAIP's challenges center on developing credible 
guidelines and approaches that can leverage work already done and 
foster concurrence in risk analysis results and encourage actions to be 
taken. 

Table 9: Summary of Progress Made and Challenges That Remain in IAIP's 
Risk Management Approach: 

Risk management phase: Strategic goals, objectives, and constraints; 
Examples of progress made: Strategic goals have been laid out in 
various national strategies, and IAIP issued the Interim National 
Infrastructure Protection Plan (NIPP), which, among other things, is 
intended to guide the process for identifying, comparing, and 
prioritizing critical assets within and across sectors; 
Examples of remaining challenges: As IAIP works to complete the interim 
NIPP, it will be challenged to develop performance measures and 
detailed timelines or target dates for identifying and prioritizing 
critical infrastructure. 

Risk management phase: Risk assessment; 
Examples of progress made: IAIP has developed a national database of 
critical infrastructure assets and a series of benchmark threat 
scenarios to be used to analyze potential attacks. IAIP has used these 
scenarios to develop data collection instruments for two types of 
assets (nuclear plants and chemical plants) to assess their 
vulnerabilities; 
Examples of remaining challenges: IAIP faces challenges in developing a 
methodology so that it can develop data on the relative likelihood of 
various threat scenarios--a key element of risk assessment it must 
conduct under the Homeland Security Act. It also faces challenges in 
developing a methodology for prioritizing assets within and across 
sectors. 

Risk management phase: Alternatives evaluation; 
Examples of progress made: IAIP has developed tools for owners and 
operators of selected critical infrastructure assets to estimate the 
consequences of an attack and perform vulnerability assessments. This 
information is a prerequisite when valuing costs and benefits and 
prioritizing among different assets; 
Examples of remaining challenges: IAIP and the owners or operators of 
critical infrastructure may not agree on the costs and benefits of 
protective actions. Developing the methodology for prioritizing assets 
will also be important for progress in this phase. IAIP plans to 
develop procedures in 2006 for quantifying costs and benefits. 

Risk management phase: Management selection; 
Examples of progress made: IAIP is pursuing partnerships to encourage 
the responsible owners and operators to implement IAIP recommendations. 
IAIP is also developing a risk management decision support framework to 
facilitate government authorities' selection of risk management policy, 
programs, and budgetary options; 
Examples of remaining challenges: In most cases, IAIP does not have 
authority to make the selection; its role in this regard is advisory. 
Thus, its challenge is to develop ways to help ensure that owners, 
operators, and state and local government authorities make informed 
choices, ensure that federal decisions are informed by risk-based data, 
and leverage the regulatory authorities of other agencies. 

Risk management phase: Implementation and monitoring; 
Examples of progress made: IAIP has limited responsibilities in 
implementing programs that result in improved infrastructure 
protection. It provides funds to ODP through a Buffer Zone Protection 
Program that involves efforts to reduce vulnerabilities in and around 
facilities and assets; 
Examples of remaining challenges: IAIP is similarly challenged by a 
lack of authority in implementing protective measures to protect 
critical infrastructure. Also, IAIP cannot require state and local 
governments to use federal funds on specific infrastructure protection 
measures. In addition, IAIP is challenged to get intelligence data 
specific enough to develop measures for determining whether protective 
actions are actually deterring or minimizing the impact of terrorist 
attacks. 

Source: GAO analysis of IAIP's risk management practices. 

[End of table] 

IAIP Plays a Key Role in Evaluating Risk across Infrastructure Sectors: 

While risk management is one of several tools for the Coast Guard and 
ODP, risk management is central to one of IAIP's key missions, which is 
to establish a risk management framework across the federal government 
to protect the nation's critical infrastructure and key 
resources.[Footnote 49] The Homeland Security Act of 2002 made IAIP 
responsible for critical infrastructure protection (CIP) functions, 
charging IAIP with broad responsibility for developing a comprehensive 
national plan for securing the key resources and critical 
infrastructure of the United States. IAIP's statutory responsibilities 
require it to conduct risk management activities on a national scale 
and to gather the information needed to do so from other federal 
agencies, state and local government agencies, and private sector 
entities.[Footnote 50] By statute, IAIP is responsible for: 

* identifying, detecting, and understanding threats in light of actual 
and potential vulnerabilities to the homeland; 

* conducting comprehensive assessments of the vulnerabilities of the 
key resources and critical infrastructure of the United States; 

* conducting risk assessments to determine the risks posed by 
particular types of terrorist attacks and how likely they are to 
succeed, as well as the feasibility and potential efficacy of various 
countermeasures; 

* identifying priorities for protective and support measure by DHS, 
other federal agencies, state and local governments, the private 
sector, and other entities; and: 

* recommending measures to protect the critical infrastructure and key 
resources of the United States in coordination with other federal 
agencies and in cooperation with state and local governments, the 
private sector, and other entities.[Footnote 51] 

In December 2003, the President issued Homeland Security Presidential 
Directive-7, Critical Infrastructure Identification, Prioritization, 
and Protection, which established the framework in which IAIP carries 
out its responsibility of coordinating the overall national CIP effort. 
Current CIP policy, as described in HSPD-7, defines responsibilities 
for DHS, sector-specific agencies, and other departments and agencies. 
It instructs federal departments and agencies to identify, prioritize, 
and coordinate the protection of critical infrastructure to prevent and 
deter attacks, and mitigate the effects of any attacks that may occur. 

To ensure the coverage of critical sectors, HSPD-7 designated sector- 
specific agencies for the critical infrastructure sectors identified in 
the National Strategy for Homeland Security (see table 10). These 
agencies are responsible for infrastructure protection activities in 
their assigned sectors, which include coordinating and collaborating 
with relevant federal agencies, state and local governments, and the 
private sector to carry out their responsibilities and facilitating the 
sharing of information about physical and cyber threats, 
vulnerabilities, incidents, potential protective measures, and best 
practices. For example, the transportation sector, for which the 
Department of Homeland Security is assigned responsibility, includes 
the movement of people and assets that are vital to the nation's 
economy, mobility, and security. The maritime shipping infrastructure, 
a component of the transportation sector, includes ports and their 
associated assets, ships, passenger transportation systems, and other 
maritime transportation assets. Each sector may also include a number 
of systems and "key assets"--some of which include individual targets 
whose attack could cause large-scale human casualties or property 
destruction, or profoundly damage national prestige and 
confidence.[Footnote 52] 

Table 10: Critical Infrastructure Sectors and Lead Federal Agencies: 

Sector: Agriculture; 
Sector-specific agency: Department of Agriculture and Department of 
Health and Human Services. 

Sector: Banking and finance; 
Sector-specific agency: Department of the Treasury. 

Sector: Chemicals and hazardous materials; 
Sector-specific agency: Department of Homeland Security (IAIP). 

Sector: Defense industrial base; 
Sector-specific agency: Department of Defense. 

Sector: Emergency services; 
Sector-specific agency: Department of Homeland Security (IAIP). 

Sector: Energy; 
Sector-specific agency: Department of Energy. 

Sector: Food; 
Sector-specific agency: Department of Agriculture and Department of 
Health and Human Services. 

Sector: Government; 
Sector-specific agency: Department of Homeland Security (Federal 
Protective Service). 

Sector: Information technology and telecommunications; 
Sector-specific agency: Department of Homeland Security (IAIP). 

Sector: Postal and shipping; 
Sector-specific agency: Department of Homeland Security (TSA). 

Sector: Public health and health care; 
Sector-specific agency: Department of Health and Human Services. 

Sector: Transportation; 
Sector-specific agency: Department of Homeland Security (TSA). 

Sector: Drinking water and water treatment systems; 
Sector-specific agency: Environmental Protection Agency. 

Source: GAO analysis of the President's national strategy documents and 
HSPD-7. 

[End of table] 

Under HSPD-7, the overall national CIP effort is to be coordinated by 
DHS, a responsibility carried out by IAIP, subject to the DHS 
Secretary's direction and control, as provided in the Homeland Security 
Act of 2002. The DHS Secretary has several overarching CIP 
responsibilities under HSPD-7, including identifying, prioritizing, and 
coordinating CIP within and across sectors, with an emphasis on 
critical infrastructure and key resources that could be exploited to 
cause catastrophic health effects or mass casualties. In addition, the 
Secretary is required to establish uniform policies, approaches, 
guidelines, and methodologies for integrating CIP and risk management 
activities within and across sectors, along with metrics and criteria 
for related programs and activities. 

Strategic Goals, Objectives, and Constraints: High-Level Goals in Place 
and Interim Plan Drafted, but Performance Measures and Milestones Have 
Yet to Be Developed: 

A number of strategic goals are in place to guide IAIP's broad 
responsibilities. They stem from the Homeland Security Act of 2002 and 
the following executive branch documents: 

* The National Strategy for the Physical Protection of Critical 
Infrastructures and Key Assets.[Footnote 53] Issued in February 2003, 
this strategy identified a set of national goals and objectives and 
outlined the guiding principles that underpin efforts to secure the 
critical infrastructure and key resources of the United States. The 
strategy recognizes that adequate protection of critical infrastructure 
requires (1) comprehensive threat assessment and analysis; (2) 
effective and efficient risk assessment; and (3) security baselines, 
standards, and guidelines. 

* Homeland Security Presidential Directives 7 and 8. Both directives 
were issued on December 17, 2003. HSPD-7 established a national policy 
for federal departments and agencies to enhance the protection of 
critical infrastructure and key resources and made DHS responsible for 
establishing uniform policies, approaches, guidelines, and 
methodologies for integrating nationwide infrastructure protection and 
risk management activities. HSPD-8 calls for a national preparedness 
goal that balances the potential threat and magnitude of terrorist 
attacks with the resources required to prevent, respond to, and recover 
from them--a risk management approach calling for an estimate of the 
likelihoods and expected consequence of possible terrorist attacks that 
takes finite resources into account. 

* The Interim National Infrastructure Protection Plan (interim NIPP). 
The Secretary of DHS assigned IAIP the responsibility for developing a 
national infrastructure protection plan. The interim NIPP was released 
in February 2005.[Footnote 54] It calls for the use of a risk 
management framework that takes into account threats, vulnerabilities, 
and consequences when comparing and prioritizing critical 
infrastructure and deciding what actions to take to protect them. This 
framework is intended to be carried out both within sectors and 
nationally across sectors. It is to contain steps for narrowing the 
overall set of assets to those that are most critical at a national 
level. 

We have reported that the interim NIPP is not a comprehensive document, 
and IAIP faces several challenges in making it more 
comprehensive.[Footnote 55] These challenges are in two main areas: 

* Performance measures. Although the interim NIPP did not contain 
performance metrics to measure effectiveness, it recognized that they 
are needed and calls on IAIP and the sector-specific agencies to 
develop them. According to IAIP officials, this is being done in two 
phases. For the first phase, IAIP has identified a set of basic core 
metrics that can be used to evaluate performance across all sectors, as 
called for in the interim NIPP. IAIP is also working with agencies 
responsible for specific sectors to develop a supplemental set of 
metrics for each sector. The intent of this measurement process is to 
provide DHS and the sector-specific agencies with feedback on where and 
how they should focus their resources to be most effective. According 
to IAIP officials, the second phase involves, in part, monitoring the 
progress of each sector in implementing the risk management framework 
laid out in the interim NIPP. To date, however, IAIP and the sector- 
specific agencies have not completed the performance metrics called for 
in the interim NIPP. 

* Milestones and timelines. The interim NIPP did not contain milestones 
for the development of sector-specific plans or timelines of target 
dates for identifying and prioritizing critical infrastructure. 
According to IAIP officials, the final version of the NIPP, after 
undergoing interagency review, will be released in 2006, and it will 
contain milestones and timelines for the initial phase of developing 
performance metrics. It is not clear, however, if this final version 
will contain milestones and timelines for sector-specific plans or for 
completing the process of prioritizing critical infrastructure. 

Risk Assessment: IAIP Faces Substantial Gaps and Has Made Limited 
Progress: 

IAIP's progress in risk assessment has not been as extensive as the 
Coast Guard's or ODP's. Its area of greatest progress is in developing 
a national database of assets that constitutes the nation's critical 
infrastructure. In two other key respects, however, it faces major 
challenges in carrying out requirements specified in law or policy 
directives. These challenges are in developing adequate data on threats 
and creating a methodology for making cross-sector comparisons. 

Progress Is Greatest on Database of Critical Assets: 

IAIP has developed the National Assets Database (NADB), an inventory of 
approximately 80,000 assets, as a starting point in being able to 
evaluate and prioritize them. The NADB includes such facilities as 
nuclear power plants, pipelines, bridges, stadiums, and locations such 
as Times Square. Assets in the NADB are gathered from a variety of 
public and private sector sources, including federal, state, and local 
databases; prior studies containing lists of infrastructure and 
resources; and sector-specific data collection activities. The database 
is revised as assets are added and removed in collaboration with state 
and local officials and with representatives of sector-specific 
agencies. According to IAIP officials, this database is intended to 
produce baseline data that will later allow for assessments of 
vulnerabilities by location, within sectors, and across 
sectors.[Footnote 56] 

Development of Threat Data Faces Challenges: 

IAIP has begun work to develop threat scenarios and analyze them. The 
Homeland Infrastructure Threat and Risk Analysis Center (HITRAC), 
staffed by sector specialists and intelligence analysts with 
backgrounds from the intelligence community, is responsible for 
generating these plausible threat scenarios--called benchmark threats. 
HITRAC has developed 16 benchmark threats, such as a suicide bomber, a 
vehicle-borne improvised explosive device, and a weapon of mass 
destruction. IAIP faces two substantial challenges, however, in 
completing this work. 

* Relative probability for threat scenarios not yet developed. First, 
IAIP faces challenges in developing a way to differentiate the relative 
probability of various threats. Under the Homeland Security Act of 
2002, IAIP must perform risk assessments to determine the risks posed 
by particular types of terrorist attacks, including an assessment of 
the probability of success of such attacks.[Footnote 57] IAIP officials 
stated that a lack of intelligence data and law enforcement data limits 
their ability to develop the relative probability for various threat 
scenarios, and for this reason, they have focused their initial efforts 
on developing vulnerability and consequence data. According to IAIP 
officials, the intelligence community--including the intelligence 
components of DHS--has been unable to provide detailed intelligence on 
threats to most sectors, infrastructure, assets, or asset 
types.[Footnote 58] Assigning equal likelihood to various threat 
scenarios would mean IAIP's risk assessments will not include key 
threat data on the capabilities and intent of terrorist groups, the 
history of terrorist threats on various asset types, or the existence 
of terrorist cells domestically or internationally. And because data on 
the relative likelihood of threat scenarios are not included, the 
assessments will emphasize high-consequence events that may have a low 
probability of occurring. This approach is bound to result in 
potentially unreliable or incomplete data on where to establish 
priorities. In September 2005, IAIP officials told us that they 
recognize the importance of developing data on the relative likelihood 
of an attack type, and that doing so is part of their responsibility 
for meeting the requirements of the Homeland Security Act. Officials 
told us that they plan to assess the likelihood of threats by having 
HITRAC develop consistent comparative threat assessments by integrating 
intelligence sources, law enforcement data, and suspicious activity 
reporting with subject matter expertise. IAIP officials caution, 
however, that the directorate may not be able to estimate the relative 
likelihood of some threat scenarios, and as a result, some assessments 
may emphasize high-consequence events that have a low probability of 
occurring. Also, officials indicated that some inaccuracy is to be 
expected when HITRAC examines the intent and capability of an adversary 
whose plans are concealed and that it will be important to reduce the 
potential of low-confidence assessments having undue influence when 
long-term investment decisions are made. 

* Vulnerability assessments not yet developed for many infrastructure 
sectors. Second, IAIP has not yet developed vulnerability assessments 
for the full spectrum of infrastructure sectors. As of August 2005, 
IAIP had managed the development of vulnerability assessment 
questionnaires for two components of critical infrastructure--nuclear 
facilities and chemical facilities. Initially, IAIP's contractor was 
scheduled to assess asset types in 8 of the 18 sectors and key assets 
by the end of 2005. However, according to IAIP officials this work will 
be done by the end of 2006.[Footnote 59] IAIP officials did not have an 
estimate as to when the assessments would be complete for all other 
sectors of critical infrastructure that it is responsible for 
assessing.[Footnote 60] 

Methodology for Cross-Sector Comparisons Has Experienced Setbacks: 

IAIP has experienced setbacks in its attempts to meet HSPD-7's 
requirement to develop a strategy for identifying, prioritizing, and 
coordinating the protection of critical infrastructure. To prioritize 
the protection of this infrastructure, IAIP has been working for about 
2 years on a methodology for assessing vulnerabilities of critical 
infrastructure to help inform comparisons of risks on an intrasector 
and cross-sector basis. This methodology has been delayed because of 
methodological concerns, and IAIP's schedule for completing various 
other activities needed to meet the requirement is dependent on the 
methodology and now appears uncertain. 

IAIP developed an analytical assessment tool known as the Risk Analysis 
and Management for Critical Assets Protection methodology (RAMCAP). 
RAMCAP was begun in November 2003, when DHS awarded the American 
Society of Mechanical Engineers (ASME) a $1.6-million grant to develop 
an overarching methodological guide for the private sector to assess 
its terrorism security risks. Originally, DHS expected RAMCAP to 
advance homeland security efforts by providing a usable, affordable 
vulnerability and risk assessment methodology for owners and operators 
to use that would inform risk management decision in the private 
sector. Now, IAIP views it as an independent effort that will 
complement IAIP risk assessment efforts at comparing risks within and 
across sectors. 

When RAMCAP was released for comment in April 2004, ASME received 
comments from over 100 officials from industry, academia, and 
government. A peer review process produced additional comments. The 
comments centered on several issues, including concern about the amount 
of resources needed to assess risks and the limited benefits in 
creating self-assessments for industry without knowing what the purpose 
of the effort involved.[Footnote 61] In June 2004, ASME altered its 
approach based on this feedback and began a broad outreach effort to 
involve industry in developing vulnerability assessment modules. 

In December 2004, IAIP awarded a $4 million contract to ASME to 
continue its efforts in developing RAMCAP and vulnerability assessment 
modules for owners and operators of asset types in eight 
sectors.[Footnote 62] After developing pilot modules for the chemical 
and nuclear industries, in August 2005, ASME produced a new working 
draft of RAMCAP. According to IAIP, RAMCAP may undergo minor revisions 
as more modules are completed. IAIP officials stated that more 
substantial revisions to RAMCAP will likely occur when IAIP broadens 
its approach to infrastructure protection by examining the entire 
systems and interdependencies within and across sectors. 

In September 2005, IAIP informed us that it is developing a National 
Comparative Risk Assessment for the 18 critical infrastructure sectors 
and key assets, and it plans to complete an interim assessment by the 
end of 2006. The assessment has several phases. It involves sector- 
specific agencies identifying the top 100 high-risk systems and assets 
in their sector based on potential consequences. IAIP officials said it 
requested agencies to develop their lists by July 2005. However, the 
degree to which agencies will be able to fully respond to the request 
is uncertain. For example, as of July 2005, the Transportation Security 
Agency had begun but not yet completed a risk assessment for the 
passenger rail sector, and it did not indicate when the assessment 
would be done. Additionally, TSA did not expect to have the first 
version of its sector-specific plan--the plan that describes its risk 
assessment methodology--until February 2006. Until agencies such as TSA 
complete their assessments and develop their sector-specific plans, 
they will not be able to determine relative risks within their sector 
in a consistent fashion.[Footnote 63] 

Whether IAIP will be able to complete this interim assessment by the 
end of 2006 may be complicated by two other factors. First, in 
September 2005, IAIP officials said that IAIP still needs to award the 
contract for this effort, and officials did not provide a schedule for 
when this would occur. Second, the degree and extent to which IAIP will 
need to obtain input from the private sector in developing this 
assessment tool is unclear. In developing the vulnerability assessment 
modules for sector-specific industries, IAIP's consultant and industry 
groups worked extensively with each other in developing the modules. If 
similar coordination and communication efforts are needed to complete 
the interim assessment, it may call for additional time and resources. 

According to IAIP officials, once the initial rankings are made, 
subject matter experts will review the results and the results of 
previous analyses will be used to refine the rankings within a sector 
and ultimately across sectors. IAIP plans to use the interim results to 
inform homeland security grant programs and serve as a basis for 
further risk management efforts. The results of the vulnerability 
assessment modules that are being developed will inform the national 
assessment and will act as a basis for interaction with the private 
sector, according to IAIP. 

The interim National Comparative Risk Assessment is intended to meet 
the immediate need of examining relative risk within and across 
sectors. However, there are limitations on the degree to which the 
interim assessment will produce comparable, consistent, and reliable 
data for setting national priorities. First, only vulnerability 
assessments of asset types in 8 of 18 sectors and key assets are 
scheduled for completion by the end of 2006, while the schedule for 
completing assessments on other sectors or asset types is unclear-- 
suggesting that vulnerability data on some sectors will be more 
reliable than data on other sectors. Second, the interim assessment 
will likely involve a heavy emphasis on consequence-related data, and 
less information on the relative likelihood of threat scenarios and 
data on vulnerability--two major components of risk assessment. Third, 
as sector-specific agencies develop their list of high-risk assets, it 
is unclear whether they will do so in a consistent and uniform fashion, 
because the overarching framework that guides these actions is not yet 
in place. In September 2005, we reported that completing the element of 
the framework that defines concepts, terminology, and metrics for 
assessing risk limits DHS's ability to compare risk across 
sectors.[Footnote 64] IAIP recognizes that engaging sector-specific 
agencies in assessing risk in a consistent way is key. IAIP now intends 
the National Comparative Risk Assessment to provide such guidance, but 
until this assessment is issued, it is unclear where the guidance will 
come from. Limitations such as these, according to IAIP, are driven to 
some degree by resource and time constraints. 

Evaluating Alternatives: Lack of Guidance and Consensus on Costs and 
Benefits Creates Challenges: 

The alternatives evaluated by IAIP differ somewhat from the 
alternatives evaluated by the Coast Guard (as discussed in chapter 2) 
or ODP (as discussed in chapter 3). The Coast Guard evaluates 
alternatives related directly to how it deploys its own resources, such 
as increasing security patrols, boarding suspicious vessels, creating 
and enforcing security zones, and using its regulatory powers to force 
maritime facilities to adopt protective measures. ODP evaluates 
alternatives that relate directly to its grant awards; the alternatives 
are the various funding proposals for security-related infrastructure 
improvements. In contrast, the alternatives that IAIP evaluates are 
generally not based on actions IAIP can take. Instead, they are based 
on identifying, from a national standpoint, the (1) areas of greatest 
risk and (2) infrastructure protection strategies that offer the 
greatest benefit for the cost involved. In this regard, IAIP faces 
several challenges. 

One challenge faced by IAIP in evaluating alternatives is the 
difficulty in valuing costs and benefits in a homeland security 
setting--an important tool for evaluating alternatives. OMB has 
guidance for federal agencies on how to evaluate alternative government 
actions, but this guidance is of limited use in assessing alternatives 
for risk management for homeland security programs. OMB has identified 
various tools it considers useful in planning, such as cost-benefit 
analysis, capital budgeting, and regulatory decision making.[Footnote 
65] However, such tools are difficult to apply to homeland security 
expenditures, even when such application is encouraged in the National 
Strategy for Homeland Security, because the benefits of homeland 
security investments are hard to quantify. Because OMB guidance is 
relatively silent on acceptable treatment of nonquantifiable benefits, 
there is a lack of criteria to guide federal analysts in conducting 
risk management. In response to our inquiries, OMB officials told us 
that they have not been involved in implementing Homeland Security 
Presidential Directive-7, which is related to critical infrastructure 
protection. In addition, they said that they have not been developing, 
nor did they have plans to start developing, guidance on risk 
management methodologies for federal agencies to use for homeland 
security programs. They said that they would rely on DHS and IAIP to 
take the federal government lead in developing such methodologies. IAIP 
officials said they plan to develop procedures in fiscal year 2006 for 
quantifying costs and benefits of mitigation strategies. 

Another challenge faced by IAIP in evaluating alternatives, at least in 
terms of ranking assets and protective actions to prevent and mitigate 
attacks, is that other entities responsible for taking such protective 
actions may use different criteria for evaluation. That is, the 
federal, state, local, or private sector entities that own and operate 
much of the nation's critical infrastructure may disagree with IAIP on 
how to evaluate alternatives through assessing benefits and costs or 
other types of evaluation. This lack of consensus could lead to two 
separate evaluations of alternatives--one by IAIP and one by the entity 
that owns and operates the asset. While IAIP may view certain assets or 
protective actions as critical, those responsible for the assets and 
protective actions may view the assets and actions as marginal or not 
necessary at all, or vice versa. According to IAIP officials, state and 
local government and industry stakeholders will benefit from using the 
same assessments, but the value that is placed on the assessment may 
differ from one stakeholder to another. As an example, Washington state 
emergency management officials told us that their initial listing and 
ranking of critical assets was much different than that developed by 
IAIP. State officials may place greater weight on attack scenarios that 
result in impacts on children or disadvantaged communities. Industry 
may place greater weight on scenarios that disrupt the long-term 
viability of a business. IAIP's challenge is discerning whether federal 
risk concerns are managed appropriately and that the costs for managing 
risks are assigned as much as possible to the authority that benefits 
from the activity. In addition to these considerations, Congress also 
has a role in appropriating federal funding of protective programs. 

Management Selection: IAIP Challenged in Protecting Infrastructure 
because of Its Limited Role in Selecting Alternative Protective 
Measures: 

As with the alternatives evaluation part of GAO's framework, IAIP's 
management selection differs from how the Coast Guard or ODP makes 
decisions. While the Coast Guard decides what protective measures to 
take and ODP decides which applicants receive funding for port 
security, IAIP is not in a position to direct others on how to act. 
IAIP officials said that while they have the responsibility of helping 
set risk-based priorities concerning where resources should be spent 
for protecting critical infrastructure, IAIP does not have the 
authority to direct the management of these resources in many cases. 
For example, IAIP does not have authority to compel owners and 
operators of critical infrastructure to take action. Instead, IAIP 
recommends the relative priority of critical infrastructure and 
specific protective measures. Other entities, such as owners, 
operators, or agencies with more direct regulatory responsibilities, 
can then act on IAIP recommendations and technical advice. The one 
exception is IAIP's Buffer Zone Protection Program, which we discuss in 
the section following on implementation and monitoring. 

At the departmental level, DHS may use the IAIP priority list to direct 
one of its component agencies to take specific actions. As examples, 
IAIP priorities could be used by U.S. Customs and Border Protection to 
increase the scrutiny of cargo containers at a specific port terminal, 
by the Secret Service to increase security for the President, by the 
Federal Emergency Management Agency to help improve local response 
capabilities near specific facilities, or by ODP to provide grants to 
specific facilities. In some cases, DHS components may have some 
authority over private infrastructure owners--such as the Coast Guard's 
regulatory powers related to implementing the Maritime Transportation 
Security Act. For example, the Coast Guard approves vulnerability 
assessments and mitigation strategies developed by owners and operators 
of facilities and vessels and it reviews whether the owners and 
operators are complying with their plans. However, in most cases, DHS 
and its components do not have authority over the owners and operators 
of critical infrastructure, particularly in the private sector. 

Thus, the challenge for IAIP (and DHS at the department level) is that 
it generally does not have authority over the entities that make 
management decisions to select among alternative protective measures. 
In such cases, IAIP's role is limited to providing information on how 
it views the relative strengths and weaknesses of the alternatives and, 
sometimes, technical advice on how these entities could improve the 
security of their assets. IAIP officials said they use their expertise 
and powers of persuasion to get the owners and operators to take 
specific protective actions. IAIP officials also said that IAIP works 
closely with industry groups to set standards and promote voluntary 
compliance. 

Certain protective measures could have application across multiple 
threat scenarios. The interim NIPP does not describe what IAIP's plans 
are for analyzing its benchmark threat scenarios and developing 
information that could help owners and operators reduce risk by 
protecting their facilities with countermeasures that address multiple 
scenarios. For example, improving security operations could reduce the 
risk of multiple threats, such as suicide bombers, truck bombs, or 
weapons of mass destruction. Having sufficient emergency supplies could 
address the consequences of casualties or damaged infrastructure that 
occur from various attack modes. By not having plans to develop 
information on what countermeasures could address multiple threat 
scenarios, IAIP is limited in it ability to provide information that 
informs owners and operators of facilities of ways to protect their 
facilities in a cost-effective fashion. According to IAIP, it is 
considering this in the cost-benefit framework that is being developed. 

IAIP's lack of authority over the owners and operators of critical 
infrastructure highlights the importance of coordination among 
different federal agencies. Several federal agencies with lead 
responsibilities for specific sectors of infrastructure do have 
regulatory authorities that could be used to set security standards and 
compel protective measures. For example, the Department of the Treasury 
and the Securities and Exchange Commission and other agencies have 
regulatory authorities over financial markets and can compel them to 
take certain protective measures. While DHS generally does not have 
authority over many of the assets in those sectors in which it has lead 
responsibility, some other federal agencies do. For example, the 
Nuclear Regulatory Commission, which issues licenses to commercial 
nuclear power plants, has clear regulatory authority over security 
matters at these facilities. While each sector has a Government 
Coordinating Council that includes representatives from federal 
agencies involved in the sector, IAIP recognizes that it can further 
leverage work that has already been done by other agencies that have 
regulatory authority over certain private sector owners. This condition 
brings attention to the need for IAIP to coordinate with these agencies 
to leverage federal authority in areas where oversight already exists. 

Implementation and Monitoring: IAIP Is Also Challenged by Its Limited 
Role in Implementation and Limited Information on Effectiveness: 

IAIP has a limited role in implementing its own protective programs, as 
well as in monitoring the effectiveness and level of implementation of 
security risk management programs broadly. An example of IAIP's role in 
implementing protective programs is the Buffer Zone Protection Program. 
Managed by the Protective Security Division within IAIP, the program is 
designed to make it more difficult for terrorists to conduct planning 
activities or successfully launch attacks from the immediate vicinity 
of critical infrastructure and key resource targets. The goal of the 
program is to assist state and local government, local law enforcement, 
and owners and operators in preventing, defending against, preparing 
for, and mitigating the impacts of terrorist attacks. The program does 
so by making grants available, through ODP, to state and local law 
enforcement to implement buffer zone protection plans "outside the 
fence" of private facilities, as well as by conducting workshops and 
technical assistance visits. 

However, IAIP's role in implementing protective measures is much more 
limited "inside the fence" because IAIP does not own or operate any 
assets, have regulatory authority over those entities that own or 
operate the assets, or provide funds for such entities. Owners and 
operators of the infrastructure assets--be they federal, state, local, 
or private--are responsible for implementing the protective actions 
needed. As previous GAO work has shown, public policy tools have been 
used as an approach to encourage increased private sector critical 
infrastructure protection efforts even when regulatory authority is 
lacking.[Footnote 66] In terms of monitoring implementation of actions 
to increase the protection of critical infrastructure, however, IAIP 
does have a role. Its mission is to assess the overall state of 
critical infrastructure protection in the nation. 

In its monitoring role, IAIP faces a number of challenges. The first 
challenge is that there are currently no performance measures to 
evaluate the effectiveness of infrastructure protection. The interim 
NIPP, while calling for such measures, does not offer any. According to 
IAIP officials, such performance metrics will not be available before 
2006. One of the difficulties encountered is that it is hard to develop 
performance measures to gauge homeland security activities that are 
directed at modifying terrorist behavior. For example, it is difficult 
to determine whether measures to improve the protection of critical 
infrastructure have a deterrent effect on terrorists. In some cases, 
deterring terrorists from attacking "hard" targets (those that are 
heavily protected) might have the effect of directing the terrorist 
toward attacking "soft" targets (those that are lightly protected). 

Conclusions: 

IAIP's role in risk management is critical because of its breadth. IAIP 
has overall responsibility to identify critical assets across all 
sectors of the nation's infrastructure, as well as to play a lead role 
in developing methodologies and guidance for analyzing risks and 
assessing the benefits and costs of security alternatives. Progress in 
most areas of responsibility has been limited, and much challenging 
work remains to be done. In particular: 

* Good threat data are critical to conducting risk management. During 
the course of our review, IAIP recognized the importance of developing 
information on the relative probability to various threat scenarios. 
Until better data on threats are developed, risk-based data may not 
fully inform decisions on where to establish priorities. 

* Even with better threat data, assets cannot be compared across 
sectors without a methodology for doing so. Currently, IAIP is 
developing a methodology to do this, but it is not yet comprehensive, 
nor has it been applied. Until such a methodology is fully developed, 
IAIP will be challenged in conducting a national-level assessment of 
risks, a key element of IAIP's core responsibility. Without a 
methodology, it will not be possible for IAIP to make a determination 
of relative risks that could help inform decisions on resource 
allocation. 

* Comprehensive planning and performance measures are necessary to 
clarify what needs to be done and to determine progress in critical 
infrastructure protection. The National Infrastructure Protection Plan 
is still in interim form, leaving open many questions about how 
specific sectors will be protected and how performance will be 
measured. Without target dates for completing sector-specific plans and 
performance measures, it will not be possible to determine IAIP's 
progress in these areas. Additionally, without having plans to develop 
information on what protective measures could address multiple threat 
scenarios, IAIP's ability to inform owners and operators of ways to 
protect their facilities in a cost-effective fashion is limited. 

In all, the lack of progress leaves many decision makers basically on 
their own to develop a way to determine where scarce resources need to 
be applied against almost unlimited numbers of assets to maximize the 
protection of critical infrastructure and security for the homeland. 

Recommendations for Executive Action: 

To help ensure the development of risk management approaches to 
homeland security activities, we recommend that the Secretary of 
Homeland Security direct the Undersecretary for IAIP to undertake the 
following three actions: 

* Work with the intelligence community to develop ways to better assess 
terrorist threats and use available information and expert judgment to 
develop a relative probability for various terrorist scenarios and 
provide this information to sector-specific agencies; 

* As tasked by presidential directive, develop a methodology for 
comparing and prioritizing risks of assets within and across 
infrastructure sectors by including data on the relative probability of 
various threat scenarios; 

* In completing the National Infrastructure Protection Plan, include 
target dates for completing sector-specific plans, developing 
performance measures, and identifying protective measures that could 
address multiple threat scenarios. 

Agency Comments and Our Evaluation: 

In commenting on a draft of chapter 4, DHS, including IAIP, said that 
IAIP is taking steps to address recommendations in the report. In 
regard to our recommendation on working with the intelligence community 
to develop better threat data, DHS said it is working with the Coast 
Guard on a pilot project to do so. The pilot effort will be evaluated, 
improved upon, and then more broadly applied with stakeholders in the 
intelligence community pending their acceptance, according to DHS. In 
regard to our recommendation on developing a methodology for comparing 
and prioritizing risks within and across sectors, DHS responded that 
for fiscal year 2006 grants, a risk analysis methodology was applied 
that considers a small set of assets across sectors. With respect to 
our recommendation on completing the National Infrastructure Protection 
Plan, DHS stated that a draft plan is out for comment and that the 
issue of identifying protective measures that could address multiple 
threat scenarios is being addressed by IAIP. DHS also submitted 
technical comments under separate cover and we made changes where 
appropriate. Written comments from DHS are in appendix II. 

[End of section] 

Chapter 5: Overall Observations and Recommendations: 

Taken together, what do the efforts and experiences of these three 
components suggest about where DHS is with regard to managing homeland 
security efforts on the basis of risk? In our view, there are two key 
overall observations related to the degree of progress made, and two 
more related to next steps that need to take place. With regard to 
progress made, the first observation is that while considerable effort 
has been applied, much more remains to be done than has been 
accomplished so far. Across all three components, the most progress has 
generally been made on fundamental steps, such as conducting risk 
assessments of individual assets, and the least amount of progress has 
generally been made on developing ways to translate this information 
into comparisons and priorities across ports or across infrastructure 
sectors. Second, and closely related, progress among the three 
components' efforts has varied not only with the length of time the 
component has been at the job, but also with the complexity of its risk 
management task. With regard to next steps that would appear to add 
most value to making further progress, one key observation is that in 
the short term, progress is heavily dependent on continuing to make 
steady progress at improving basic policies, procedures, and methods 
for risk assessments and other phases of the risk management framework 
outlined in chapters 2, 3, and 4. Each component has an admittedly 
difficult set of challenges ahead, but progress has to be built on 
taking these incremental steps and doing them well. The final 
observation is related to a critical longer-term need: more guidance, 
direction, and coordination from DHS. The challenges and difficulties 
associated with creating a coordinated, coherent risk management 
approach to the nation's homeland security have been widely 
acknowledged since the events of September 11 and the creation of DHS. 
One of the presidential directives calls on DHS to provide such 
guidance, but the agency has yet to do so. As individual components 
begin to mature in their risk management efforts, the need for 
consistency and coherence becomes even greater. Without it, the 
prospects increase for efforts to fragment, clash, and work at cross 
purposes. 

First Observation: Much Remains to Be Done: 

There is a long way to go in implementing risk management successfully 
in port security--and an even longer way to go in implementing risk 
management in homeland security in general. One main reason is the 
sheer amount of work that must be done. Five years ago, before the 
September 11 attacks, the various terrorist scenarios seemed more 
remote and less certain than the hard reality brought on that day. 
September 11 changed this perspective dramatically. The work involved 
is immense and cuts across many jurisdictional boundaries. Federal 
agencies are called on to strengthen their partnerships with one 
another and to work more closely with thousands of state, local, and 
industry stakeholders. 

A second major reason is that applying risk management to terrorism has 
no well-established precedent. Parts of the private and public sectors 
have used risk management principles for decades. However, doing so in 
a homeland security setting is a highly difficult task that remains in 
its embryonic phases. The components we reviewed face daunting 
challenges in weaving a concern for risk into the annual cycle of 
management review and budget decisions. Across the federal government, 
this challenge is magnified and complicated because of the number of 
agencies charged with carrying out risk management. This is an 
extraordinarily difficult effort with no clear road map of ways to 
strategically integrate a concern for risk into management decisions. 

The fact that so much work remains is not the result of inaction by 
federal agencies. In the agencies and programs we examined, activities 
were often extensive and wide-ranging. Some activities, such as IAIP's 
attempts to develop risk assessment criteria for its comparisons across 
risk sectors, have had limited success, compounding the problem. The 
underlying point, however, is that this is an extraordinarily difficult 
effort with no clear and direct precedent to act as a guide. 
Implementing risk management in port and homeland security will take 
time and care, and this challenge will require ingenuity in adopting 
risk management techniques to this new application in a cost-effective 
way. 

The progress that has occurred to date in the agencies and programs we 
examined has been primarily in the activity that most people would 
perhaps associate most readily with risk management--conducting 
assessments to determine what the risks are at specific ports and 
facilities. While much remains to be done even there, progress has 
generally been slower on ways to approach risk management 
strategically--that is, with a clear set of measurable objectives, a 
clear knowledge of the options available for addressing risks and the 
trade-offs involved with these options, and evaluation and feedback 
mechanisms for continuing to refine and strengthen the approach. 

Observation Two: Progress Varies by Component and Reflects Key 
Characteristics of the Component and the Scope of Its Risk Management 
Efforts: 

The three components we studied have made varying degrees of progress 
in risk management, and to a degree their progress is related to three 
main factors: how long they have been at the task, how organizationally 
stable they are, and the scope of what they are trying to do. The Coast 
Guard, for example, is furthest along among the three components, 
reflecting in part where it stands in relationship to all three of 
these factors. It has been at the task the longest of the three 
components, having begun work on implementing risk management in its 
port security efforts immediately after the September 11 attacks. Its 
primary risk assessment tool at the port level, PS-RAT, was implemented 
in November 2001, and by August 2002, prior to the creation of DHS and 
the port security framework called for under the Maritime 
Transportation Security Act of 2002, it had begun security assessments 
at major U.S. ports. To a degree, these early efforts were learning 
experiences that required changes, but the Coast Guard was able to 
build on its early start. The Coast Guard also had the greatest 
organizational stability of the three components. It moved into DHS as 
an already established entity with an organizational culture spanning 
decades, and its organization and mission were not significantly 
altered by moving into DHS. Finally, with regard to the scope of its 
risk management activities, the Coast Guard's work is specific to port 
locations, where it has direct and primary responsibility for carrying 
out security responsibilities. With its focus on ports, the Coast Guard 
does not have to address a number of the critical infrastructure 
sectors laid out in national preparedness policy, such as banking and 
finance, information and technology, and public health. Even so, the 
Coast Guard's experience to date shows that as the scope of activity 
widens, even within a single sector, complexities develop. For example, 
the Coast Guard has prioritized risks within individual ports, and it 
has actions under way to assess risks across ports, but using this 
information to strategically inform the annual program review and 
budget process will require further attention. 

ODP has made somewhat less progress than the Coast Guard. Relative to 
the Coast Guard's progress, its progress reflects a later start, an 
organization with much less institutional maturity, and a different 
role from the Coast Guard's in that ODP provides grant money rather 
than directly setting security policy. ODP was transferred from the 
Department of Justice to the Department of Homeland Security in 2003. 
While ODP's early grant approval efforts had some risk management 
features in place, its main strides in risk management have come in the 
procedures recently adopted for the fiscal year 2005 grants. In moving 
toward risk management, ODP has found ways to allow information from 
the Coast Guard and IAIP to inform its decision making. This is an 
encouraging and important sign, because the success of risk management 
efforts depends in part on the ability of agencies with security 
responsibilities to share and use each others' data and expertise. 
Although both the Coast Guard and the port security grant program 
administered by ODP have port security as their focus, ODP's more 
limited scope of responsibility has also had an effect on its risk 
management efforts. First, because ODP's role is to award grants that 
support federal priorities in port security, its progress in risk 
management depends to a degree on the progress made by federal agencies 
in determining what their own port security performance measures should 
be. Second, ODP's funding priorities are subject to criteria other than 
risk, as the fiscal year 2004 grant awards demonstrate. That year, 
after creating an initial list of awardees based in part on risk, and 
without regard to ability to pay, ODP extensively revised the list and 
awarded grants to entities considered to have fewer funding 
alternatives. 

Of the three components, IAIP is the least far along in its risk 
management efforts. All three factors have had an effect on this 
progress: IAIP has been at its task for a relatively short time; it is 
a new component; and relative to the Coast Guard and ODP, the scope of 
its efforts is much broader and more difficult. IAIP was created under 
the Homeland Security Act of 2002, giving the directorate little time 
to acquire institutional maturity. In addition to taking on difficult 
tasks like risk management, IAIP faces other institutional challenges, 
such as establishing new management systems, developing sound human 
capital practices, and integrating its efforts with those of the rest 
of DHS. Further, the scope of its risk management activities extends 
well beyond the port-focused activities of the Coast Guard or ODP. IAIP 
is responsible for conducting risk assessments for every critical 
infrastructure segment in the nation. As demonstrated by the experience 
of its RAMCAP methodology for comparing risk across sectors, IAIP 
remains challenged in meeting that responsibility. Its lack of progress 
reflects the same lesson that emerges from the Coast Guard's experience 
in trying to expand the focus of risk assessments beyond a single port: 
The complexity of risk management appears to grow exponentially as the 
focus expands beyond a single location or single type of 
infrastructure. This complexity may help explain IAIP's lack of 
progress, but IAIP is unable at this time to provide adequate assurance 
to Congress or the country that the federal government is in a position 
to effectively manage risk in national security efforts. Steps have 
been small; by far the biggest work is yet to come. 

Observation Three: In the Short Term, Further Progress Is Still Heavily 
Dependent on Completing and Improving Basic Policies, Procedures, and 
Methods: 

Acknowledging that the nation still has far to go in establishing a 
sound risk management approach to security should not obscure the need 
to continue taking small, but critical, steps--building on incremental 
advances. The three components we reviewed have actions under way to 
improve their risk management approach, and their experience indicates 
that much of the immediate work should remain focused on basic steps 
needed to implement all components of the full risk management 
framework. The recommendations we make in chapters 2, 3, and 4 include 
component-specific steps for what needs to be done. In overview, these 
specific recommendations cluster around several major themes related to 
the five phases: 

* Setting strategic goals, objectives, and constraints: While all three 
components have broad-scale goals in place, none has yet tied these 
goals to specific and measurable results. Without such measures, it is 
difficult to gauge what progress has been made in improving security 
and what security gaps remain. The Coast Guard is furthest along: It 
has tied its goals to activity levels, such as the number of patrols 
conducted or vessels inspected, and it is working toward developing 
outcome-based measures. This is a good step, but without such measures 
in place, it is not possible to see how programs reduce risks, improve 
security, or identify gaps in security that remain. All three 
components would benefit from specifying in clear and measurable terms 
what their efforts are designed to accomplish. 

* Conducting risk assessments: All three components can improve their 
risk assessment techniques. All three were challenged by a general lack 
of detailed information on capabilities and intentions of terrorist 
groups as this relates to various threat scenarios. They took different 
approaches in response: The Coast Guard, for example, used threat 
scenarios as substitutes for detailed threat information and is working 
on assigning likelihood to each in order to determine where risks might 
be greatest, while IAIP evaluated the consequences of certain possible 
attacks and focused its analysis of vulnerabilities on the attacks with 
the greatest consequences. Approaches that do not include information 
on the likelihood of various threat scenarios have limitations that 
affect the degree to which agencies are able to determine how to best 
focus their efforts on areas of greatest risk. Efforts to strengthen 
both data, methodology, and policy would increase the reliability of 
their results. 

* Evaluating alternatives: All three components face problems in 
measuring the costs and benefits of different measures for preventing 
or mitigating terrorist attacks. These include developing ways to 
measure costs incurred by a broad range of public and private 
stakeholders and developing ways to measure benefits (such as 
deterrence) when these benefits may not be quantifiable. These 
difficulties are particularly great for IAIP, which must be able to 
measure costs and benefits associated with mitigation strategies that 
reduce vulnerabilities at critical infrastructure in all sectors. These 
difficulties are compounded by the complexity in valuing costs and 
benefits in the area of homeland security when either the costs or the 
benefits are difficult to quantify or are not valued in monetary terms. 

* Management selection: The three components face different challenges 
in this area, because each has different types of alternatives 
available in making decisions. The Coast Guard has the most direct 
control over security efforts; it can, for example, decide what 
protective measures to take with its own assets, and it has authority 
over other stakeholders to implement the Maritime Transportation 
Security Act of 2002. Its challenges lie mainly in what has already 
been discussed above--strengthening methods for risk assessment and 
alternatives evaluation and integrating this effort with the annual 
cycle of program review--so that management can make the most informed 
decisions about these efforts. ODP affects security efforts less 
directly; it can only consider facilities that have applied for grants, 
and it has no direct authority over port facilities in general, as does 
the Coast Guard. ODP has worked with the Coast Guard to receive its 
input into the grant application process. One challenge is to 
consistently apply criteria for management selection in a more 
transparent way. IAIP faces the most challenges in this area, because 
once it makes recommendations about how to prioritize assets on a 
national scale, it is largely dependent on the actions of others to 
carry them out and, particularly for owners and operators of private 
infrastructure, is dependent to a large degree on persuasion, market 
forces, or the work of regulatory agencies that have authority over key 
infrastructure, to ensure that protective measures are in place. 

* Implementation and monitoring: Particularly for the Coast Guard and 
ODP, we have been able to identify instances in which the components 
have moved aggressively to improve their risk management approaches-- 
and to continue doing so. Particularly with its recent setback on its 
RAMCAP methodology, IAIP is considerably behind these two components in 
implementing any kind of risk management approach. To move forward, it 
must overcome more basic problems with assessing risks and 
alternatives. 

Observation Four: In the Long Term, Progress Rests Heavily on a Level 
of Coordination That Has Yet to Be Demonstrated: 

In the long term, progress will become increasingly dependent on how 
well the nation's homeland security risk management effort is 
coordinated. We have identified and reported on some notable 
improvements in coordination at the port level, through such mechanisms 
as intelligence fusion and coordination centers, local area maritime 
security committees, and interagency operational centers.[Footnote 67] 
Replicating such coordination among DHS agencies and with state, local, 
and industry stakeholders is key. 

Currently, various assessment approaches are being used, and in many 
ways, these approaches are neither consistent nor comparable. Our work 
at IAIP, the Coast Guard, and ODP showed examples of these 
inconsistencies. For example, IAIP's initial plans called for treating 
all threat scenarios as equally likely to happen, while the Coast Guard 
and ODP are attempting to integrate the likelihood of various threat 
scenarios into their analysis vulnerabilities. The danger in using 
different methods is that if agencies develop systems and methodologies 
without some overall coordination, they may end up with redundant or 
incompatible systems that have little or no ability to inform one 
another. Even more important, these systems may provide decision makers 
with unreliable or incomplete data on how to allocate resources and 
protect the American people in a cost-effective way. Absolute 
compatibility is likely impossible given the multiple stakeholders at 
the federal, state, local, and industry levels. For example, owners and 
operators of critical infrastructure may value and act on risks 
differently than the Coast Guard and IAIP. Having a common risk 
management framework is a key consideration for assuring that knowledge 
and data can be transferred to all stakeholders, while permitting 
stakeholders to value risks in different ways. Even if agencies and 
stakeholders were working in close cooperation, lack of coordination is 
likely only to exacerbate the problem. This is particularly true given 
the difficulty of the task and the limited availability of federal risk 
management guidance. 

Until now, having such inconsistencies may have seemed less important 
than just getting risk management efforts under way. To a degree, we 
found this was the case with the larger universe of homeland security 
actions: When we first began reviewing agency actions shortly after the 
September 11 attacks, we found agencies at work on many efforts, but 
signs of coordination problems were already apparent. Similarly, the 
risk management efforts that have been conducted to date appear fueled 
by a strong sense of the need to make some headway, with coordination 
and consistency a lesser concern. For example, IAIP, which is charged 
by statute with developing comprehensive assessments of the 
vulnerabilities of critical infrastructure and key resources, did not 
attempt to guide the Coast Guard's efforts in setting up a methodology 
for assessing port-specific risks. IAIP officials told us it was more 
important for the Coast Guard and other agencies to proceed with their 
risk assessment efforts than to delay starting, even though the 
officials recognized that these efforts might create approaches that 
would not mesh cleanly with the approach that IAIP would eventually 
develop. Given what has occurred to date, this course of action appears 
prudent, in that the Coast Guard has a considerable portion of a risk 
management system in place. If it had waited to begin until guidelines 
and policies had been set, it would still be waiting to start. 

Now, however, the need for coordination is looming larger. IAIP has a 
significant role to play in this regard through its responsibility for 
providing agencies with guidance about risk management, but it has made 
limited progress. IAIP has been challenged in establishing uniform 
policies, approaches, guidelines, and methodologies for integrating 
federal infrastructure protection and risk management activities within 
and across sectors, along with metrics and criteria for related 
programs and activities as called for by HSPD-7. While IAIP has 
coordinated its activities with entities such as ODP and the Coast 
Guard, it has yet to issue policies, guidelines, and methodologies as 
required by the directive. Making progress with regard to this 
challenge is key to an effective use of risk management resources, as 
the National Strategy for the Physical Protection of Critical 
Infrastructure and Key Assets recognizes. 

Guidance is also important when agencies integrate a concern for risk 
into the annual cycle of program and budget review. Doing so within an 
agency is a difficult task in that traditional ways of reviewing 
budgets and programs often rely on program data that call for 
continuing or expanding a program without examining the relative risks 
that are addressed with the funds that are expended. Shifting 
organizations toward this nexus of using risk-based data as part of 
annual management review cycles will take time, attention, and 
leadership. Even in agencies where much progress has been made in 
developing risk management techniques, integrating disparate systems 
such as risk management with budget and program management remains a 
long-term challenge. The Secretary of DHS has said that operations and 
budgets of its agencies will be reviewed through the prism of risk, but 
doing this is made difficult by the level of guidance and coordination 
that has been provided so far. 

DHS has recently reorganized, and the consideration of whether its new 
organization will effectively implement its risk management 
responsibilities is an important one. At the time we conducted our 
review, risk management was the responsibility of IAIP. IAIP's risk 
management efforts were focused mainly on assessing and reducing the 
vulnerabilities that exist in and around specific facilities or assets. 
But DHS's responsibility is broader than this: besides assessing and 
reducing vulnerabilities at specific facilities, it also includes 
preventing attacks from occurring (and in the process protecting people 
and critical infrastructure) and responding to and recovering from 
natural disasters and acts of terrorism. This initial focus on 
vulnerabilities at specific assets had the potential of limiting DHS's 
ability to achieve the broader goal of using risk-based data as a tool 
to inform management decisions on all aspects of its missions. The 
Secretary of DHS has now moved risk management to a new Preparedness 
Directorate. Although, it is unclear how such a move could affect DHS's 
ability to carry out its risk management responsibilities, the new 
focus on preparedness may result in an emphasis that may go too far the 
other way--that is an emphasis on protection of specific assets and 
response and recovery at the expense of prevention. As DHS goes 
forward, the office in which the risk management responsibility resides 
should have a broad perspective across the department's entire mission 
as well as the necessary authority to hold DHS component agencies 
responsible for carrying out risk management activities in a 
coordinated and consistent manner. 

Beyond DHS, integrating risk with existing systems for budget and 
program review is complicated by the fact that while IAIP has 
responsibility for coordinating this effort, IAIP and the Secretary of 
DHS are challenged because they must depend on others to follow risk 
management principles for programs and budgets at the other six major 
Departments or agencies that have been charged with assessing risks 
under HSPD-7. In regard to this situation, OMB has taken the position 
that this is what the Homeland Security Act and HSPD-7 call for and it 
does not play a role in this process. These conditions increase the 
uncertainty of implementing risk management across federal agencies in 
a way that informs program and budget review processes. Whether such 
practices will occur within the executive branch is unclear because of 
these organizational barriers. 

Recommendations for Executive Action: 

To strengthen individual agency efforts to implement a risk management 
approach to homeland security activities, we recommend that the 
Secretary of Homeland Security direct the Undersecretary for IAIP to 
undertake the following three actions: 

* As required by presidential directive, establish uniform policies, 
approaches, guidelines, and methodologies for integrating federal 
infrastructure protection and risk management activities within and 
across sectors, along with metrics and criteria for related programs 
and activities and develop a timetable for completing such guidance. 
Such policies and guidance should address the issue of integrating risk 
management systems into existing systems of program and budget review. 

* As DHS continues to review its organizational structure, work with 
the Secretary's office to determine which office is best suited to help 
ensure that the responsibility for risk management policy and 
implementation has a broad enough perspective on all elements of risk, 
including threats, as well as the necessary authority to coordinate 
with DHS component agencies and hold them accountable for risk 
management activities. 

* Work with the Office of Management and Budget to examine options for 
holding departments and agencies accountable for integrating risk 
management for homeland security programs and activities into the 
annual cycle of program and budget review. 

Agency Comments and Our Evaluation: 

In commenting on a draft of chapter 5, DHS, including IAIP, generally 
concurred with the recommendations. In regard to our observation that 
there is a long-term need for guidance and coordination, DHS noted that 
as part of the department's second-stage review, a six-point agenda has 
been created to ensure that policies, operations, and structures are 
best aligned to address potential threats. This agenda is a major step 
in the right direction, and as we observe in this chapter, much work 
remains to be done to translate this goal into actions. 

DHS agrees that the application of risk management to domestic 
terrorism has no precedent, and that the probabilities and consequences 
of terrorist acts are difficult to predict. DHS also concurred with our 
observation that the scope of establishing a risk management framework-
-a former IAIP Directorate responsibility--across the federal 
government is immense. DHS acknowledges that IAIP's progress has been 
limited in part because its risk assessment responsibilities span broad 
sectors of the nation's infrastructure rather than seaports alone. DHS 
also submitted written comments under separate cover and we revised the 
report where appropriate. Written comments from DHS are in appendix II. 

[End of section] 

Appendix I: A Risk Management Framework: 

This appendix describes how we developed the risk management framework 
and how we used it to evaluate activities related to homeland security 
and combating terrorism. The framework is intended to be a starting 
point for risk management activities and will likely evolve as 
processes mature and lessons are learned. A glossary is included at the 
end of this appendix. 

General Lack of Uniform Guidance on Risk Management: 

Although the Homeland Security Act and subsequent strategies advocate 
the use of risk management to protect the nation's critical 
infrastructure and key resources, they did not define how this was to 
be accomplished. Homeland Security Presidential Directive 7 (HSPD-7) 
directed the Secretary of the Department of Homeland Security (DHS) to 
establish uniform policies, approaches, guidelines, and methodologies 
integrating federal infrastructure protection and risk management 
activities. However, no further direction or guidance as to the course 
of action has been forthcoming. 

The ability to anticipate future happenings and to choose among 
alternatives lies at the heart of risk management and provides us with 
a guide, based on good management practices and supported by 
established internal controls that can enhance decision making. 
Although risk management has long been used for assessing risk in some 
sectors, such as environmental issues, health care, finance, and the 
insurance industry, the application of risk management principles to 
the homeland security area is relatively new. The many areas and 
activities under homeland security provide untested and difficult 
challenges because the source of the risk is an intelligent adversary 
with whom there exists little domestic experience. As a result, the 
probabilities and consequences of a terrorist attack are difficult to 
predict. In spite of this high degree of uncertainty and the knowledge 
that not all risk can be eliminated, enhancing protection from known or 
potential threats can help prevent or mitigate adverse events. 

Methodology for Developing a Risk Management Framework: 

Given that there is no established universally agreed upon set of 
requirements or processes for a risk management framework specifically 
related to homeland security and combating terrorism, we developed a 
framework that would be applicable by reviewing, analyzing, and 
synthesizing several sources of information. 

We began by reviewing current risk literature and previous GAO reports 
and testimonies.[Footnote 68] We consulted the Government Performance 
and Results Act (GPRA) of 1993; the Government Auditing Standards, 2003 
Revision, GAO's Standards for Internal Control in the Federal 
Government (November 1999); guidance from the Office of Management and 
Budget (OMB); the work of the President's Commission on Risk 
Management; consulting papers; and the enterprise risk management 
approach of the Committee of Sponsoring Organizations (COSO) of the 
Treadway Commission. In addition, we consulted with experts in the 
fields of risk management, risk modeling, and terrorism. We reviewed 
numerous frameworks from industry, government, and academic sources. 

We synthesized information from these numerous government, industry, 
and academic sources in developing our risk management framework. The 
framework was field-tested on several GAO reviews. The draft framework 
was then reviewed by three academic experts in risk management. No 
substantial changes to the draft framework were recommended. 

A Risk Management Framework: 

The framework should be considered to be a starting point in a field 
that is evolving, and the entire cycle of risk management activities 
should be viewed as a goal. The phases contained in the framework are: 

* strategic goals, objectives, and constraints; 

* risk assessment; 

* alternatives evaluation; 

* management selection; and: 

* implementation and monitoring. 

The framework has been developed so that individual phases of the 
approach, such as risk assessment, do not become ends in themselves, 
but provide a full cycle of related activities, from strategic planning 
through implementation and monitoring. The process is dynamic, and 
although the various phases appear linear, new information can be 
entered at any phase. The framework can be used to inform agency 
officials and decision makers of the basic components of a risk 
management system or can be used as a stand-alone guide. Figure 4 
illustrates our risk management framework and some sources of criteria, 
such as GAO best practices, Office of Management and Budget circulars, 
GAO guidance on internal controls, and the Government and Performance 
Act of 1993 and their link with management processes. While statistical 
methods and risk-ranking approaches frequently underlie risk assessment 
approaches, different application areas tend to develop their own 
terminologies and their own logical sequences for the cause of risk. 

Figure 4: Sources of Evaluation Criteria Associated with Risk 
Management Phases: 

[See PDF for image] 

[End of figure] 

The risk management framework is designed to be flexible, in that the 
approach may be applied at various organizational levels ranging from 
that of a department level of a multiagency organization down to that 
of a specific project or program. 

Because there is no one uniformly accepted approach to risk management, 
terms and activities may differ across applications. However, any 
approach that omits the substance of the steps shown in figure 4 is 
likely to have material weaknesses. Table 11 summarizes the phases of 
our risk management framework and provides examples of elements 
contained in those phases. 

Table 11: A Risk Management Framework: 

Phase: Strategic goals, objectives, and constraints; 
Description: Addresses what the strategic goals are attempting to 
achieve and the steps needed to attain those results; 
Example of elements: 
* Overall results desired, i.e., "end state"; 
* Hierarchy of strategic goals and subordinate objectives related to 
those goals; 
* Specific activities to achieve results; 
* Priorities, milestones, and outcome-related performance measures; 
* Limitations or constraints that affect outcomes. 

Phase: Risk assessment; 
Description: Addresses identification of key elements of potential 
risks so that countermeasures can be selected and implemented to 
prevent or mitigate their effects; 
Example of elements: 
* Analysis of threat gained from available sources (This threat 
information will be used to develop scenarios. See below); 
* Estimation of vulnerability of an asset based on standards, such as; 
* Availability/predictability; 
* Accessibility; 
* Countermeasures in place, and; 
* Target hardness; 
* Identification of consequence of a terrorist attack on a specific 
asset and criticality, or the relative importance, of the asset 
involved. 

Phase: Alternatives evaluation; 
Description: Addresses the evaluation of alternative countermeasures to 
reduce risk being considered with associated costs; 
Example of elements: 
* Specific countermeasure(s) to reduce risk; 
* Use of external sources to improve decision making such as 
consultation with experts and threat scenarios; 
* Cost-benefit analysis of countermeasure(s). 

Phase: Management selection; 
Description: Addresses where resources and investments will be made 
based on alternatives evaluation and other management criteria, such as 
availability of funds; 
Example of elements: 
* Management's preferences and value judgments associated with 
expenditure of countermeasures and funds, such as distribution of 
antiterrorism measures over assets; 
* Organizational risk tolerance; 
* Resource allocations; 
* Documentation of decisions, including rationale. 

Phase: Implementation and monitoring; 
Description: Addresses how countermeasures will be applied and 
mechanism to keep security measures updated; 
Example of elements: 
* Implementation of countermeasures according to strategy; 
* Periodic testing of countermeasures; 
* Linkages to other risk management strategies, state, local, or 
private entities (horizontal); 
* Linkages to other strategies, departmental and national (vertical); 
* Mechanisms for alterations in system based on current threat data; 
* Periodic evaluation for assessing efficiency and effectiveness of 
program. 

Source: GAO. 

[End of table] 

The following sections provide more detail on the five phases of our 
risk management framework. 

Strategic Goals, Objectives, and Constraints: 

This phase addresses what the strategic goals are attempting to achieve 
and the steps needed to attain those results, including milestones and 
performance measures to permit measurement of progress toward those 
goals. Ideally, management decisions should be made in the context of a 
strategic plan, with clearly articulated goals and objectives that flow 
from the plan. Strategic goals at the highest level could be considered 
an "end-state" followed by a logical hierarchy of major goals and 
subordinate objectives composed of clear, concise, measurable 
activities and timelines to achieve results and ensure accountability. 
An organization's program or plan and risk planning documents should 
address risk-related issues that are central to its mission. Our work 
related to the Government Performance and Results Act of 1993 has 
produced guidance that identifies risk for the congressional oversight 
of federal agencies' strategic plans.[Footnote 69] The consideration of 
risk in strategic planning may be incorporated into planning documents 
or into specific management strategies. 

Currently, it is difficult to translate plans and actions into a clear 
sense of how we are progressing in making our nation more secure. One 
reason is that homeland security efforts, in general, lack clear goals 
with corresponding performance measures to evaluate progress toward 
these goals.[Footnote 70] As others, such as the Gilmore Commission, 
have stated, a continuing problem for homeland security has been the 
lack of clear strategic guidance about the definition and objectives of 
preparedness.[Footnote 71] 

Risk management allows entities to operate more effectively in 
environments of uncertainty by providing the discipline and structure 
in which to address these issues, since risk management is not an end 
in itself, but an important component of an entity's management 
process. As such, risk management is interrelated with, among other 
things, an entity's governance, performance management, and internal 
controls. The process of risk management provides the rigor and 
structure necessary to identify and select among alternative risk 
responses whose cumulative effect is intended to reduce risk, and the 
methodologies and techniques for making selection decisions. This 
process enables entities to enhance their capability to identify 
potential adverse events, assess risks, and establish integrated 
responses. Further, this phase in the planning process would include 
support and buy-in from upper levels of management and stakeholders. 
Acceptance for concepts of the model from this group provides the 
groundwork for future discussions. 

Finally, various constraints may have an impact on risk management 
plans. Some constraints may be imposed by statute, higher-level policy, 
budget, or other factors beyond management's control and may vary with 
the scale of the application. Managers at different levels within an 
organization will have different degrees of flexibility to institute 
risk management countermeasures. An important constraint for federal 
agencies, such as DHS, is the role that Congress plays in authorizing 
and funding programs. For example, Congress may direct specific actions 
affecting how agencies allocate funding. 

Risk Assessment: 

This phase addresses the process of evaluating the threats and 
vulnerabilities of assets so that countermeasures might be instituted 
to prevent or mitigate risks. Threat, in the risk management model, 
concerns the probability that a specific type of attack will be 
initiated against a specific target. It includes any circumstance or 
event with the potential to cause loss or damage to the asset. Although 
agencies may not have enough information to identify and characterize 
all threats related to their assets, known or imagined adverse events 
would be characterized in some detail. Effective threat analysis is 
dependent on an understanding of an adversary's intention, motivation, 
historical data, and capability to damage. An additional crucial 
component of risk assessment is vulnerability, that is, any weakness 
that an adversary can exploit to harm or damage the asset. An asset may 
be highly vulnerable to one mode of attack but have a low level of 
vulnerability to another, depending on a variety of factors, such as 
countermeasures already in place. While consequence concerns the result 
of an adverse event on a particular asset, criticality is the asset's 
relative importance to the entity. A criticality assessment identifies 
and prioritizes assets and functions in terms of specific criteria, 
such as their importance to public safety and the economy, as a basis 
for identifying which structures or processes are relatively more 
important to protect from attack. Criticality assessments are important 
because they provide, in combination with the framework's threat and 
risk assessments, the basis for prioritizing which assets require 
greater protection relative to finite resources and provide information 
for later stages in the risk management process. Risk assessments 
should utilize the most appropriate subject matter experts in assessing 
the components of risk. When dealing with Bayesian probability 
estimates, this is critical.[Footnote 72] In addition, mathematical 
constructs must be chosen carefully, specifically tailoring approaches 
in the context of uncertainty and data quality. 

Alternative Evaluation: 

This phase addresses the evaluation of risk reduction methods by 
consideration of countermeasures or countermeasure systems and the 
costs and benefits associated with them. Ideally, a risk management 
framework would include an evaluation of a risk assessment as a valid 
decision support tool to establish and prioritize a risk management 
strategy. Furthermore, a strategy might include risk management 
consultants and decision-making tools, such as software that simulates 
a particular type of attack. Information developed in previous phases 
would inform decisions. 

Specific countermeasures would be considered and prioritized based on a 
number of factors, such as the degree of risk reduction they afford and 
the cost and difficulty to implement them. Risk assessment may give 
guidance to implementing countermeasures or countermeasure systems that 
may be used for more than one critical asset, or to prevent, mitigate, 
or respond to multiple adverse events occurring simultaneously. In 
addition, external risk consultants can be advantageous at this phase 
in terms of creating a variety of countermeasure options. While 
external reviewers cannot ensure the success of a plan, they can 
increase the probability of selecting the most effective 
countermeasures for the least cost because of their expertise in the 
area. Further, risk scenarios might also provide valuable information 
pertaining to an entity's ability to respond to a terrorist event, 
evaluate its coordination with other entities, identify problems, and 
institute corrective action. 

Finally, a risk management strategy should include a cost-benefit 
analysis of countermeasure options as they are a critical element of 
alternatives evaluation. Major regulatory actions or capital 
investments of federal expenditures generally require a cost-benefit or 
cost-effectiveness approach.[Footnote 73] This approach can be useful 
in evaluating alternatives, since it links the benefits from risk- 
reducing countermeasures to the costs associated with them. In the 
development of such analyses, quantitative impacts affecting both costs 
and benefits are, to the extent possible, identified in monetary terms. 

While the core OMB guidance for evaluating countermeasures for 
budgetary and regulatory purposes focuses on monetary cost-benefit 
evaluation, OMB is essentially silent when costs and benefits cannot be 
easily quantified or monetized. Costs that are not generally estimated 
or included in monetary terms include opportunity costs, that is, the 
value of opportunities forgone because resources are applied to 
antiterrorism countermeasures. These costs are most controversial when 
considered in areas such as public service programs or services 
curtailed or cancelled. Benefits are usually measured in terms of the 
risk reduction they provide. They are considered in terms of the 
overall effectiveness of the countermeasures with respect to the 
estimated vulnerabilities. 

Management Selection: 

This phase addresses such issues as determining where resources and 
investments will be made, the sources and types of resources needed, 
and where those resources would be targeted. Management's active 
participation in this phase is important as decisions are of necessity 
influenced by the preferences and value judgments of agency leadership. 
For example, some managers will prefer to concentrate countermeasures 
on a relatively few critical assets, while others may value 
distributional impacts, that is, to distribute resources over a wide 
variety of assets. Ideally, a risk management strategy would also 
identify appropriate mechanisms to allocate resources, such as grants 
based on identified needs. Furthermore, a key factor in the selection 
of risk reducing measures is risk tolerance, the level of comfort 
management has with various levels of risk. This tolerance may change 
over time, depending on new information, changes in financial 
constraints, and attitude toward risk. The risk management strategy, 
with stakeholder input, will identify what constitutes an acceptable 
level of risk for assets and how resources are delegated. 

The risk management strategy reflects consideration as to which risks 
should be managed immediately and to what extent, and which risks can 
be deferred and addressed at a later time. Milestones and timelines for 
implementation are important elements that allow evaluating the extent 
to which progress is being made toward achieving goals. It also 
illustrates the degree of protection that can be obtained and places 
security and costs in perspective. The documentation of management 
decisions, including the rationales that support the decisions, will 
provide valuable information for future adjustments. 

Implementation and Monitoring: 

This phase addresses the degree to which risk management strategies 
contain internal controls and performance measurement guidelines. In 
addition to implementing countermeasures, it may also include 
implementing new organizational policies and procedures, as well as 
human, physical, and technical controls. Countermeasures would be 
initiated in accordance with the timelines in the risk management 
schedule. 

Monitoring and evaluation include, when and where appropriate, external 
peer review, testing and validation of countermeasures, evaluating the 
effects of these actions on future operation, and identifying 
unintended consequences. GAO has also discussed the importance of these 
activities as they ensure actions are taken to address risks. 

Internal control monitoring should assess the quality of performance 
over time and ensure that the findings of audits and other reviews are 
promptly resolved.[Footnote 74] Internal controls help to ensure that 
procedures are documented and maintained. 

Difficulties Applying a Risk Management Framework: 

Risk management represents a unique set of challenges. Developing such 
a framework is designed to guide the actions of management to prepare 
for and respond to adverse events in an environment of uncertainty. As 
applied to homeland security, the ability to determine the likelihood 
of terrorism-related events occurring and quantifying the resulting 
outcomes is balanced against the benefit as protection (security) 
provided at an acceptable cost. 

Generally, a major difficulty in executing and implementing a risk 
management system occurs because of what is commonly called decision 
uncertainty, that is, the assumed goal of minimizing risk does not have 
a common meaning.[Footnote 75] For example, minimizing risk is based on 
values, the measure of risk, and the comparison of values and risk. 
Decision uncertainty arises when there is controversy or ambiguity 
concerning how to compare and weight social objectives. Three major 
sources of decision uncertainty are: 

* Risk measurement--although the selection of risk measurement is both 
an art and a science, it must be technically correct as well as both 
valid and meaningful. 

* The social cost of risk--in order to make different risks comparable, 
various risks often have to be quantified into comparable quantities 
and placing values on cost and benefits involves judgments. 

* The quantification of social values--uncertainty surrounds the level 
of risk that is acceptable or can be tolerated. That is, how much money 
is to be spent on protection meaning risk reduction and what is the 
cost in terms of opportunities forgone because of finite resources. 
This value is dependent upon determining society's risk attitude or 
tolerance and may change over time. 

Coordination activities suggest that for the results of a risk 
management system to be meaningful and useful, all related agencies 
should be using similar methods. If agencies' methods are not 
compatible, then comparisons between agencies become difficult and 
sector or national risk assessments becomes less reliable. In our 
earlier work, we concluded that a structured, systematic approach to 
risk management offers the best assurance that activities designed to 
protect the homeland and combat the effects of terrorism will produce 
the most effective and efficient results.[Footnote 76] Specific 
difficulties implementing risk management systems are contained in 
chapters 2, 3, and 4. 

Glossary of Risk Management Terms: 

For purposes of our risk management framework, we use the following 
definitions: 

* Asset---any person, facility, material, information, or activity that 
has a positive or symbolic value. An asset may have a value to an 
adversary as well as to its owner, although the nature and magnitude of 
these values may differ. Assets may be categorized or combined in many 
ways; examples are people, information, equipment, facilities, 
operations, and activities. 

* Benefit---net outcome, usually translated into monetary terms; a 
benefit may include both direct and indirect effects. 

* Consequence---the expected worse case or reasonable worse case impact 
of a successful attack. The consequence to a particular asset can be 
evaluated when threat and vulnerability are considered together. This 
loss or damage may be long-or short-term in nature. 

* Cost---input, both direct and indirect. 

* Cost-benefit analysis---part of the management decision-making 
process in which the costs and benefits of each countermeasure 
alternative are compared and the most appropriate alternative is 
selected. Costs include the price paid for tangible materials and the 
ongoing operational costs associated with implementing the 
countermeasures. Benefits are expressed in terms of the amount of risk 
reduction based on the overall effectiveness of the countermeasure with 
respect to the assessed vulnerabilities. 

* Countermeasure---any action taken or physical equipment used 
principally to reduce or eliminate one or more vulnerabilities. The 
cost of a countermeasure is usually expressed in monetary terms but may 
include nonmonetary costs such as reduced operational effectiveness, 
unfavorable working conditions, adverse publicity and political 
consequences. 

* Criticality assessment---identifies and evaluates an entity's assets 
or operations on the basis of a variety of factors, including the 
importance of an asset or function and the significance of a system in 
terms of national security, economic activity, and public safety. A 
criticality assessment provides the basis for determining which assets 
require greater or special protection relative to finite resources. 

* Impact---the amount of loss or damage that can be expected from a 
successful attack on an asset. Loss may be monetary, but may include 
loss of lives and destruction of a symbolic structure. 

* Monitoring and evaluation---is a continuous repetitive assessment 
process to keep a risk management process current and relevant. It 
includes, among other activities, external peer review, testing, and 
validation. 

* Opportunity cost--the value of opportunities forgone. 

* Risk--an event that has a potentially negative impact and the 
possibility that such an event will occur and adversely affect an 
entity's assets, activities, and operations. The principal classes of 
risk from terrorism are to the general public, targets of symbolic 
value, organizational, governmental, and societal infrastructure, cyber 
and physical infrastructure, and economic sectors and structures. 

* Risk assessment---the process of qualitatively or quantitatively 
determining the probability of an adverse event and the severity of its 
impact on an asset. It is a function of threat, vulnerability, and 
consequence. A risk assessment may include scenarios in which two or 
more risks interact to create a greater or lesser impact. A risk 
assessment provides the basis for the rank ordering of risks and for 
establishing priorities for applying countermeasures. 

* Risk management---a continuous process of managing--through a series 
of mitigating actions that permeate an entity's activities--the 
likelihood of an adverse event and its negative impact. Risk management 
addresses risk before mitigating action, as well as the risk that 
remains after countermeasures have been taken. 

* Scenario---the combination of weapon and attack mode on a specific 
target or critical asset (for example, the release of sarin gas in a 
subway train). 

* Threat---an indication of the likelihood that a specific type of 
attack will be initiated against a specific target or class of targets. 
It may include any indication, circumstance, or event with the 
potential to cause the loss of or damage to an asset. It can also be 
defined as an adversary's intention and capability to undertake actions 
that would be detrimental to a valued asset. 

* Threat assessment---the identification and evaluation of adverse 
events that can harm or damage an asset. A threat assessment includes 
the probability of an event and the extent of its lethality. Threats 
may be present at the global, national, or local level. 

* Vulnerability---the probability that a particular attempted attack 
will succeed against a particular target or class of targets. 

* Vulnerability assessment---the identification of weaknesses in 
physical structures, personal protection systems, processes or other 
areas that may be exploited. A vulnerability assessment identifies 
inherent states and the extent of their susceptibility to exploitation 
relative to the existence of any countermeasures. 

[End of section] 

Appendix II: Comments from the Department of Homeland Security: 

U.S. Department of Homeland Security: 
Washington, DC 20528: 

November 23, 2005: 

Ms. Margaret T. Wrightson: 
Director, Homeland Security and Justice: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Ms. Wrightson: 

RE: Draft Report GAO-06-91, Risk Management: Further Refinements Needed 
to Assess Risks and Prioritize Protective Measures at Ports and Other 
Critical Infrastructure (GAO Job Code 440378): 

The Department of Homeland Security (DHS) appreciates the opportunity 
to review and comment on the Government Accountability Office's (GAO) 
draft report. The report describes the challenges faced by the 
Department and its components, specifically the U.S. Coast Guard 
(USCG), Office of Domestic Preparedness (ODP), and the Information 
Analysis and Infrastructure Protection (IAIP) Directorate. The report 
focuses on these three components and was not a comprehensive review of 
the entire Department. Nevertheless, the report provides information 
and a perspective on the degree of progress being made within the 
Department. We note that as a result of a recent reorganization ODP 
(now Grants and Training) and the Infrastructure Protection (IP) part 
of the former IAIP Directorate are now components in the Preparedness 
Directorate. 

We appreciate the acknowledgement of the work performed to date, 
particularly at the USCG and ODP and the challenges faced by the three 
components and the Department in fulfilling our mission. The draft 
report correctly articulates that the application of risk management to 
domestic terrorism has no precedent, and that the probabilities and 
consequences of terrorist acts are difficult to predict. Indeed, as 
noted in the report, the scope of establishing a risk management 
framework--a former IAIP Directorate responsibility--across the federal 
government to protect the nation's critical infrastructure and key 
resources is immense. IAIP's progress has been limited in part because 
its risk assessment responsibilities span broad sectors of the nation's 
infrastructure, rather than seaports alone. 

We generally concur with the recommendations that are essentially 
directed to USCG, ODP, and IAIP. While no recommendations are directed 
specifically to the Department, the auditors observed that there is a 
long term need for more guidance and coordination from the Department 
level, both to help ensure that individual components are carrying out 
their roles effectively and to ensure that individual components work 
as effectively as possible with one another. As part of the 
Department's Second Stage Review, a six point agenda has been created 
to ensure that our policies, operations, and structures are best 
aligned to address potential threats. The review, initiated by the 
Secretary, examined nearly every element of the Department of Homeland 
Security in order to recommend ways that DHS could better: 

* Manage risk in terms of threat, vulnerability and consequence; 

* Prioritize policies and operational missions according to this risk- 
based approach; and: 

* Establish a series of preventive and protective steps that would 
increase security at multiple levels. 

USCG officials generally agree with the findings and recommendations. 
As noted in the draft, USCG has made progress in all five risk 
management phases and is taking action to address the challenges it 
faces in each phase. 

The recommendations addressed to the Executive Director for ODP 
relating to the Port Security Grant (PSG) program are reasonable if 
taken in context. Most of GAO's review took place prior to significant 
changes made to the program in FY 2005. We appreciate your effort to 
review some of the FY 2005 PSG programmatic materials and include some 
analysis of FY 2005 in the report after most of your work was 
completed. However, while there is obviously an understanding of at 
least the modifications made in the grant guidance, many of the 
examples and criticisms continue to be derived from analysis of the FY 
2004 program. Several of the comments and recommendations have already 
been addressed, including the comparison of risk across ports and 
clarification of the conditions under which greater leveraging of 
federal dollars should be included as a strategic goal. We anticipate 
that the remaining ODP related recommendations will be addressed in the 
FY 2006 PSG program at least to the extent possible given the 
limitations correctly noted by GAO. ODP officials encourage GAO to 
revisit the program in the spring of 2006. 

IP is taking the following steps as part of its effort to address 
recommendations made in the report: 

* The intelligence community task is being pursued in a collaborative 
pilot with the USCG, which will be evaluated, improved upon, and them 
more broadly applied within the intelligence community pending their 
acceptance. 

* The FY 2006 grants were prioritized with a risk analysis methodology 
that was applied considering a small set of assets (less than 50 types) 
across sectors, and applying the current state of threat assessments. 

* The National Infrastructure Protection Plan is out for comment. The 
issue of identifying protective measures that could address multiple 
threat scenarios is being addressed by IP, as opposed to Sector 
Specific Agency guidance. It is a cross sector issue that is being 
considered in the developing cost/benefit framework. 

Technical comments will be sent under separate cover. 

Sincerely, 

Signed by: 

Steven J. Pecinovsky: 
Director: 
Departmental GAO/OIG Liaison Office: 

[End of section] 

Appendix III: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

Margaret T. Wrightson (415) 904-2200: 

Stephen L. Caldwell (202) 512-9610: 

Staff Acknowledgments: 

In addition to the persons named above, David Alexander, Neil Asaba, 
Nancy A. Briggs, Christine Davis, Scott Farrow, Kevin Heinz, Emily S. 
Pickrell, Albert Schmidt, Stan Stenersen, April Thompson, and L. James 
Valverde made key contributions to this report. 

[End of section] 

Related GAO Products: 

Critical Infrastructure Protection: 

Critical Infrastructure Protection: Challenges in Addressing 
Cybersecurity. GAO-05-827T. Washington, D.C.: July 19, 2005: 

Critical Infrastructure Protection: Department of Homeland Security 
Faces Challenges in Fulfilling Cybersecurity Responsibilities. GAO-05- 
434. Washington, D.C.: May 26, 2005. 

Homeland Security: Agency Plans, Implementation, and Challenges 
Regarding the National Strategy for Homeland Security. GAO-05-33. 
Washington, D.C.: January 14, 2005. 

Homeland Security: Further Actions Needed to Coordinate Federal 
Agencies' Facility Protection Efforts and Promote Key Practices. GAO- 
05-49. Washington, D.C.: November 30, 2004. 

U.S. Postal Service: Physical Security Measures Have Increased at Some 
Core Facilities, but Security Problems Continue. GAO-05-48. Washington, 
D.C.: November 16, 2004. 

Drinking Water: Experts' Views on How Federal Funding Can Best Be Spent 
to Improve Security. GAO-04-1098T. Washington, D.C.: September 30, 
2004. 

Financial Market Preparedness: Improvements Made, but More Action 
Needed to Prepare for Wide-Scale Disasters. GAO-04-984. Washington, 
D.C.: September 27, 2004. 

Nuclear Regulatory Commission: Preliminary Observations on Efforts to 
Improve Security at Nuclear Power Plants. GAO-04-1064T. Washington, 
D.C.: September 14, 2004. 

U.S. Postal Service: Better Guidance Is Needed to Ensure an Appropriate 
Response to Anthrax Contamination. GAO-04-239. Washington, D.C.: 
September 9, 2004. 

Public Key Infrastructure: Examples of Risk and Internal Control 
Objectives Associated with Certification Authorities. GAO-04-1023R. 
Washington, D.C.: August 10, 2004. 

Combating Terrorism: DOD Efforts to Improve Installation Preparedness 
Can Be Enhanced with Clarified Responsibilities and Comprehensive 
Planning. GAO-04-855. Washington, D.C.: August 9, 2004. 

Homeland Security: Transformation Strategy Needed to Address Challenges 
Facing the Federal Protective Service. GAO-04-537. Washington, D.C.: 
July 14, 2004. 

Critical Infrastructure Protection: Improving Information Sharing with 
Infrastructure Sectors. GAO-04-780. Washington, D.C.: July 9, 2004. 

Information Security: Agencies Need to Implement Consistent Processes 
in Authorizing Systems for Operation. GAO-04-376. Washington, D.C.: 
June 28, 2004. 

National Nuclear Security Administration: Key Management Structure and 
Workforce Planning Issues Remain as NNSA Conducts Downsizing. GAO-04- 
545. Washington, D.C.: June 25, 2004. 

Nuclear Security: Several Issues Could Impede Ability of DOE's Office 
of Energy, Science, and Environment to Meet the May 2003 Design Basis 
Threat. GAO-04-894T. Washington, D.C.: June 22, 2004. 

Information Security: Information System Controls at the Federal 
Deposit Insurance Corporation. GAO-04-630. Washington, D.C.: May 28, 
2004. 

Posthearing Questions Related to Fragmentation and Overlap in the 
Federal Food Safety System. GAO-04-832R. Washington, D.C.: May 26, 
2004. 

Terrorism Insurance: Effects of Terrorism Risk Insurance Act of 2002. 
GAO-04-720T. Washington, D.C.: April 28, 2004. 

Nuclear Security: DOE Needs to Resolve Significant Issues before It 
Fully Meets the New Design Basis Threat. GAO-04-623. Washington, D.C.: 
April 27, 2004. 

Terrorism Insurance: Implementation of the Terrorism Risk Insurance Act 
of 2002. GAO-04-307. Washington, D.C.: April 23, 2004. 

Critical Infrastructure Protection: Establishing Effective Information 
Sharing with Infrastructure Sectors. GAO-04-699T. Washington, D.C.: 
April 21, 2004. 

Homeland Security: Federal Action Needed to Address Security Challenges 
at Chemical Facilities. GAO-04-482T. Washington, D.C.: February 23, 
2004. 

Posthearing Questions from the September 17, 2003, Hearing on 
"Implications of Power Blackouts for the Nation's Cybersecurity and 
Critical Infrastructure Protection: The Electric Grid, Critical 
Interdependencies, Vulnerabilities, and Readiness." GAO-04-300R. 
Washington, D.C.: December 8, 2003. 

Security: Counterfeit Identification Raises Homeland Security Concerns. 
GAO-04-133T. Washington, D.C.: October 1, 2003. 

Nuclear Regulatory Commission: Oversight of Security at Commercial 
Nuclear Power Plants Needs to Be Strengthened. GAO-03-752. Washington, 
D.C.: September 4, 2003. 

Nuclear Security: DOE Faces Security Challenges in the Post September 
11, 2001, Environment. GAO-03-896TNI. Washington, D.C.: June 24, 2003. 

Nuclear Security: NNSA Needs to Better Manage Its Safeguards and 
Security Program. GAO-03-471. Washington, D.C.: May 30, 2003. 

Information Security: Progress Made, but Challenges Remain to Protect 
Federal Systems and the Nation's Critical Infrastructures. GAO-03-564T. 
Washington, D.C.: April 8, 2003. 

Homeland Security: EPA's Management of Clean Air Act Chemical Facility 
Data. GAO-03-509R. Washington, D.C.: March 14, 2003. 

Homeland Security: Voluntary Initiatives Are Under Way at Chemical 
Facilities, but the Extent of Security Preparedness Is Unknown. GAO-03- 
439. Washington, D.C.: March 14, 2003. 

Potential Terrorist Attacks: Additional Actions Needed to Better 
Prepare Critical Financial Market Participants. GAO-03-414. Washington, 
D.C.: February 12, 2003. 

Potential Terrorist Attacks: More Actions Needed to Better Prepare 
Critical Financial Markets. GAO-03-468T. Washington, D.C.: February 12, 
2003. 

Potential Terrorist Attacks: Additional Actions Needed to Better 
Prepare Critical Financial Market Participants. GAO-03-251. Washington, 
D.C.: February 12, 2003. 

High-Risk Series: Protecting Information Systems Supporting the Federal 
Government and the Nation's Critical Infrastructures. GAO-03-121. 
Washington, D.C.: January 1, 2003. 

Combating Terrorism: Actions Needed to Guide Services' Antiterrorism 
Efforts at Installations. GAO-03-14. Washington, D.C.: November 1, 
2002. 

Homeland Security: Department of Justice's Response to Its 
Congressional Mandate to Assess and Report on Chemical Industry 
Vulnerabilities. GAO-03-24R. Washington, D.C.: October 10, 2002. 

Building Security: Interagency Security Committee Has Had Limited 
Success in Fulfilling Its Responsibilities. GAO-02-1004. Washington, 
D.C.: September 17, 2002. 

Chemical Safety: Emergency Response Community Views on the Adequacy of 
Federally Required Chemical Information. GAO-02-799. Washington, D.C.: 
July 31, 2002. 

Critical Infrastructure Protection: Significant Homeland Security 
Challenges Need to Be Addressed. GAO-02-918T. Washington, D.C.: July 9, 
2002. 

Information Security: Corps of Engineers Making Improvements, but 
Weaknesses Continue. GAO-02-589. Washington, D.C.: June 10, 2002. 

Security Breaches at Federal Buildings in Atlanta, Georgia. GAO-02- 
668T. Washington, D.C.: April 30, 2002. 

National Preparedness: Technologies to Secure Federal Buildings. GAO- 
02-687T. Washington, D.C.: April 25, 2002. 

Diffuse Security Threats: Technologies for Mail Sanitation Exist, but 
Challenges Remain. GAO-02-365. Washington, D.C.: April 23, 2002. 

Terrorism Insurance: Rising Uninsured Exposure to Attacks Heightens 
Potential Economic Vulnerabilities. GAO-02-472T. Washington, D.C.: 
February 27, 2002. 

Critical Infrastructure Protection: Significant Challenges in 
Safeguarding Government and Privately Controlled Systems from Computer- 
Based Attacks. GAO-01-1168T. Washington, D.C.: September 26, 2001. 

Combating Terrorism: Actions Needed to Improve DOD Antiterrorism 
Program Implementation and Management. GAO-01-909. Washington, D.C.: 
September 19, 2001. 

Critical Infrastructure Protection: Significant Challenges in 
Protecting Federal Systems and Developing Analysis and Warning 
Capabilities. GAO-01-1132T. Washington, D.C.: September 12, 2001. 

Maritime Security: 

Coast Guard: Progress Being Made on Addressing Deepwater Legacy Asset 
Condition Issues and Program Management, but Acquisition Challenges 
Remain. GAO-05-757. Washington, D.C.: July 22, 2005. 

Coast Guard--Electronic Certification Procedures. B-302789. Washington, 
D.C.: July 6, 2005: 

Coast Guard: Preliminary Observations on the Condition of Deepwater 
Legacy Assets and Acquisition Management Challenges. GAO-05-651T. 
Washington, D.C.: June 21, 2005. 

Maritime Security: Enhancements Made, but Implementation and 
Sustainability Remain Key Challenges. GAO-05-448T. Washington, D.C.: 
May 17, 2005. 

Coast Guard: Preliminary Observations on the Condition of Deepwater 
Legacy Assets and Acquisition Management Challenges. GAO-05-307T. 
Washington, D.C.: April 20, 2005. 

Maritime Security: New Structures Have Improved Information Sharing, 
but Security Clearance Processing Requires Further Attention. GAO-05- 
394. Washington, D.C.: April 15, 2005. 

Coast Guard: Observations on Agency Priorities in Fiscal Year 2006 
Budget Request. GAO-05-364T. Washington, D.C.: March 17, 2005. 

Homeland Security: Process for Reporting Lessons Learned from Seaport 
Exercises Needs Further Attention. GAO-05-17. Washington, D.C.: January 
14, 2005. 

Coast Guard: Station Readiness Improving, but Resources Challenges and 
Management Concerns Remain. GAO-05-161. Washington, D.C.: January 31, 
2005. 

Port Security: Planning Needed to Develop and Operate Maritime Worker 
Identification Card Program. GAO-05-106. Washington, D.C.: December 10, 
2004. 

Maritime Security: Better Planning Needed to Help Ensure an Effective 
Port Security Assessment Program. GAO-04-1062. Washington, D.C.: 
September 30, 2004. 

Maritime Security: Partnering Could Reduce Federal Costs and Facilitate 
Implementation of Automatic Vessel Identification System. GAO-04-868. 
Washington, D.C.: July 23, 2004. 

Maritime Security: Substantial Work Remains to Translate New Planning 
Requirements into Effective Port Security. GAO-04-838. Washington, 
D.C.: June 30, 2004. 

Coast Guard: Station Spending Requirements Met, but Better Processes 
Needed to Track Designated Funds. GAO-04-695. June 14, 2004. 

Coast Guard: Key Management and Budget Challenges for Fiscal Year 2005 
and Beyond. GAO-04-636T. Washington, D.C.: April 7, 2004. 

Homeland Security: Summary of Challenges Faced in Targeting Oceangoing 
Cargo Containers for Inspection. GAO-04-557T. Washington, D.C.: March 
31, 2004. 

Coast Guard Programs: Relationship between Resources Used and Results 
Achieved Needs to Be Clearer. GAO-04-432. Washington, D.C.: March 22, 
2004. 

Homeland Security: Preliminary Observations on Efforts to Target 
Security Inspections of Cargo Containers. GAO-04-325T. Washington, 
D.C.: December 16, 2003. 

Posthearing Questions Related to Aviation and Port Security. GAO-04- 
315R. Washington, D.C.: December 12, 2003. 

Maritime Security: Progress Made in Implementing Maritime 
Transportation Security Act, but Concerns Remain. GAO-03-1155T. 
Washington, D.C.: September 9, 2003. 

Container Security: Expansion of Key Customs Programs Will Require 
Greater Attention to Critical Success Factors. GAO-03-770. Washington, 
D.C.: July 25, 2003. 

Homeland Security: Challenges Facing the Department of Homeland 
Security in Balancing its Border Security and Trade Facilitation 
Missions. GAO-03-902T. Washington, D.C.: June 16, 2003. 

Coast Guard: Challenges during the Transition to the Department of 
Homeland Security. GAO-03-594T. Washington, D.C.: April 1, 2003. 

Transportation Security: Post-September 11th Initiatives and Long-Term 
Challenges. GAO-03-616T. Washington, D.C.: April 1, 2003. 

Coast Guard: Comprehensive Blueprint Needed to Balance and Monitor 
Resource Use and Measure Performance for All Missions. GAO-03-544T. 
Washington, D.C.: March 12, 2003. 

Homeland Security: Challenges Facing the Coast Guard as It Transitions 
to the New Department. GAO-03-467T. Washington, D.C.: February 12, 
2003. 

Coast Guard: Strategy Needed for Setting and Monitoring Levels of 
Effort for All Missions. GAO-03-155. Washington, D.C.: November 12, 
2002. 

Port Security: Nation Faces Formidable Challenges in Making New 
Initiatives Successful. GAO-02-993T. Washington, D.C.: August 5, 2002. 

Combating Terrorism: Preliminary Observations on Weaknesses in Force 
Protection for DOD Deployments through Domestic Seaports. GAO-02- 
955TNI. Washington, D.C.: July 23, 2002. 

Risk Management: 

Passenger Rail Security: Enhanced Federal Leadership Needed to 
Prioritize and Guide Security Efforts. GAO-05-851. September 9, 2005: 

Homeland Security: Actions Needed to Better Protect National Icons and 
Federal Office Buildings from Terrorism. GAO-05-790. June 24, 2005. 

Protection of Chemical and Water Infrastructure: Federal Requirements, 
Actions of Selected Facilities, and Remaining Challenges. GAO-05-327. 
Washington, D.C.: March 28, 2005. 

Transportation Security: Systematic Planning Needed to Optimize 
Resources. GAO-05-357T. February 15, 2005: 

Homeland Security: Agency Plans, Implementation, and Challenges 
Regarding the National Strategy for Homeland Security. GAO-05-33. 
January 14, 2005. 

Homeland Security: Further Actions Needed to Coordinate Federal 
Agencies' Facility Protection Efforts and Promote Key Practices. GAO- 
05-49. November 30, 2004: 

General Aviation Security: Increased Federal Oversight is Needed, but 
Continued Partnership with the Private Sector is Critical to Long-Term 
Success. GAO-05-144. November 10, 2004. 

Air Traffic Control: System Management Capabilities Improved, but More 
Can Be Done to Institutionalize Improvements. GAO-04-901. August 20, 
2004: 

Aviation Security: Challenges in Using Biometric Technologies. GAO-04- 
785T. May 19, 2004. 

Homeland Security: Summary of Challenges Faced in Targeting Oceangoing 
Cargo Containers for Inspection. GAO-04-557T. March 31, 2004: 

Rail Security: Some Actions Taken to Enhance Passenger and Freight Rail 
Security, but Significant Challenges Remain. GAO-04-598T. March 23, 
2004. 

Homeland Security: A Risk Management Approach Can Guide Preparedness 
Efforts. GAO-02-208T. Washington, D.C.: October 31, 2001. 

Homeland Security: Key Elements of a Risk Management Approach. GAO-02- 
150T. Washington, D.C.: October 12, 2001. 

FOOTNOTES 

[1] On November 14, 2005, DHS reorganized the department. ODP and the 
Infrastructure Protection part of the former IAIP Directorate are now 
components in the Preparedness Directorate. We recognize the recent 
organizational changes, but because ODP and IAIP carried out the work 
we reviewed, we have not changed the name or organizational posture of 
these DHS components in our report.

[2] The Homeland Security Act incorporates the definition of "critical 
infrastructure" used in the USA PATRIOT Act of 2001, meaning "systems 
and assets, whether physical or virtual, so vital to the United States 
that the incapacity or destruction of such systems and assets would 
have a debilitating impact on security, national economic security, 
national public health or safety, or any combination of those matters." 
The Homeland Security Act defines "key resources" as "publicly or 
privately controlled resources essential to the minimal operations of 
the economy and government." 6 U.S.C. § 101.

[3] In November 2005, DHS issued a revised Interim National 
Infrastructure Protection Plan for comment.

[4] On November 14, 2005, DHS reorganized the department. ODP and the 
Infrastructure Protection component of the former IAIP Directorate are 
now in the Preparedness Directorate. We recognize the recent 
organizational changes, but because ODP and IAIP carried out the work 
we reviewed, we have not changed the name or organizational posture of 
these DHS components in our report.

[5] A more precise description of risk management is that it involves a 
continuous process of managing--through a series of mitigating actions 
that permeate an entity's activities--the likelihood of an adverse 
event and its negative impact. Risk management addresses risk before 
mitigating action, as well as the risk that remains after 
countermeasures have been taken. A glossary of risk management terms is 
contained at the end of appendix I.

[6] GAO, Combating Terrorism: Threat and Risk Assessments Can Help 
Prioritize and Target Program Investments, GAO/NSIAD-98-74 (Washington 
D.C.: Apr. 9, 1998), and GAO, Combating Terrorism: Need for 
Comprehensive Threat and Risk Assessments of Chemical and Biological 
Attacks, GAO/NSIAD-99-163 (Washington, D.C.: Sept. 14, 1999).

[7] 6 U.S.C. 201(d)(1), (2).

[8] GAO, High-Risk Series: An Update, GAO-05-207, p.29 (Washington, 
D.C.: January 2005).

[9] GAO-05-207.

[10] For reasons of security, this identification may not be public 
knowledge.

[11] A countermeasure is any action taken or physical equipment used 
principally to reduce or eliminate one or more vulnerabilities.

[12] The Coast Guard Port Security Risk Assessment Tool is designed to 
be used by the Captains of the Ports when making risk-based analyses of 
assets in their area of responsibility.

[13] U.S. Coast Guard, Maritime Sentinel: Coast Guard Strategic Plan 
for Ports, Waterways, and Coastal Security (Washington D.C.: September 
2005). According to Homeland Security Presidential Directive 13 on 
Maritime Security Policy (Dec. 21, 2004), maritime domain means all 
areas and things of, on, under relating to, adjacent to, or bordering 
on a sea, ocean, or other navigable waterway, including all maritime 
related activities, infrastructure, people, cargo, and vessels and 
other conveyances.

[14] According to the Coast Guard, there are 30 sectors with command 
centers already in place.

[15] GAO, Passenger Rail Security: Enhanced Federal Leadership Needed 
to Prioritize and Guide Security Efforts, GAO-05-851 (Washington D.C.: 
Sept. 9, 2005).

[16] See 46 U.S.C. § 70103(b), (c); 33 C.F.R. §§ 103.400, 103.500, 
104.300, 104.400, 105.300, and 105.400.

[17] U.S. Coast Guard, Maritime Strategy for Homeland Security 
(Washington, D.C.: December 2002).

[18] Any goals that the Coast Guard establishes will need to be aligned 
with two other national efforts to protect critical infrastructure. 
First, Homeland Security Presidential Directive-7 establishes a 
national policy for federal departments and agencies to identify and 
prioritize critical infrastructure and key resources, including assets 
and resources in and around ports. Second, Homeland Security 
Presidential Directive-13 establishes policy and guidelines for 
implementing actions that enhance maritime security. The policy 
includes (1) preventing terrorist attacks and reducing vulnerabilities 
to attacks in the maritime domain; (2) enhancing security of ports, 
critical infrastructure, and coastal approaches; (3) enhancing 
international relationships; and (4) ensuring coordinated 
implementation of authorities and responsibilities among federal 
departments and agencies.

[19] We previously reported on two of these efforts--the area maritime 
security assessments and the Port Security Assessment Program. See GAO, 
Maritime Security: Substantial Work Remains to Translate New Planning 
Requirements into Effective Port Security, GAO-04-838 (Washington D.C.: 
June 30, 2004), and Maritime Security: Better Planning Needed to Help 
Ensure an Effective Port Security Assessment Program, GAO-04-1062 
(Washington D.C.: Sept. 30, 2004).

[20] In addition to these efforts, the Coast Guard was involved in an 
interagency effort called the National Maritime Transportation Security 
Plan. This effort included the results of the National Maritime 
Security Profile. According to the Coast Guard, the results of the plan 
will contribute to the Coast Guard's outcome measures for its PWCS 
mission.

[21] The threat scenarios examined by the Coast Guard are contained in 
documents that are considered sensitive and for official use only. 
Accordingly, we do not provide detailed examples of various threat 
scenarios evaluated by the Coast Guard.

[22] Coast Guard officials said that in addition to making improvements 
in threat, vulnerability, and consequence data, they plan to align some 
of their efforts more closely with Department of Homeland Security risk 
assessment databases and to conduct sensitivity analyses of various 
mitigation strategies.

[23] IAIP officials said that the Coast Guard's efforts are a pilot 
project, and that if sufficiently mature, the effort will be more 
broadly implemented by the intelligence community.

[24] Similarly, area maritime security plans are not comparable with 
one another, in part because PS-RAT results underpin the plans. In 
addition to the limitations we describe above, the scores for each port 
are relative to that port and scores from one port cannot be compared 
with the scores from another port because, in some cases, local Coast 
Guard staff developed scores for asset types, such as bridges or 
warehouses, and at other port locations, the local staff assigned 
scores for individual types of assets. The Port Security Risk 
Assessment Program conducted studies at 55 ports at a cost of about $35 
million and these studies are also not comparable with one another.

[25] See Y. Y. Haimes, S. Kaplan, and J. H. Lambert, "Risk Filtering, 
Ranking, and Management Framework Using Hierarchical Holographic 
Modeling," Risk Analysis, Vol. 22, No. 2, 2002, and M. Leung, J. H. 
Lambert, and A. Mosenthal, "A Risk-Based Approach to Setting Priorities 
in Protecting Bridges Against Terrorist Attacks," Risk Analysis, Vol. 
4, No. 4, 2004.

[26] The legislation enabling the port security grant program was the 
Department of Defense and Emergency Supplemental Appropriations for 
Recovery from and Response to Terrorist Attacks on the United States, 
2002, Pub. L. No. 107-117, 115 Stat. 2230, 2327 (2002).

[27] Department of Homeland Security Appropriations Act, 2005, Pub. L. 
No. 108-334, 118 Stat. 1298, 1309 (2004).

[28] Federal agency and industry partners include the United States 
Coast Guard, the Information Analysis and Infrastructure Protection 
Directorate, the Border and Transportation Security Directorate, and 
the Transportation Security Administration within DHS; the Maritime 
Administration within the Department of Transportation; and the 
American Association of Port Authorities.

[29] Department of Homeland Security Appropriations Act, 2006, Pub. L. 
No. 109-90, 119 Stat. 2064, 2075 (2005).

[30] Department of Homeland Security, Office of Inspector General, 
Review of the Port Security Grant Program, OIG-05-10 (January 2005). In 
some years, more than one round of grants has been awarded. In all, 
five rounds of grants were awarded between 2002 and 2005.

[31] The Department of Defense and Emergency Supplemental 
Appropriations for Recovery from and Response to Terrorist Attacks on 
the United States, 2002, Pub. L. No. 107-117, 115 Stat. 2230, 2327 
(2002).

[32] Homeland Security Presidential Directive 13 (Dec. 21, 2004).

[33] Office for Domestic Preparedness, U.S. Department of Homeland 
Security, Fiscal Year 2005 Port Security Grant Program: Program 
Guidelines and Application Kit (Washington, D.C.: 2005). The specific 
national priorities cited are (1) chemical, biological, radiological, 
nuclear, and explosive detection and response capabilities and (2) 
National Infrastructure Protection Plan implementation.

[34] U.S. Coast Guard: Maritime Strategy for Homeland Security, 
(Washington, D.C.: December 2002).

[35] In some ways, owners and operators of facilities and vessels have 
set their own standards for what an acceptable level of risk is by the 
security plans they have developed under the Maritime Transportation 
Security Act of 2002. As required by the act, 46 U.S.C. 70103(c), 
facility and vessel owners developed over 12,000 plans to reduce 
vulnerabilities around their port areas. The Coast Guard required 
facility and vessel owners to implement them by July 1, 2004.

[36] ODP estimates that 90 percent of facilities and vessels are owned 
by industry.

[37] GAO, Federal Grants: Design Improvements Could Help Federal 
Resources Go Further, GAO-AIMD-97-7 (Washington, D.C.: Dec. 18, 1996). 
We include this figure here for illustrative purposes only and are not 
stating that this degree of substitution would occur in the port 
security grant program. For discussion of this issue in other 
public/private arenas, such as federal funding for highway investments 
and freight mobility projects, see GAO, Freight Transportation: 
Strategies Needed to Address Planning and Financing Limitations, GAO-04-
165 (Washington, D.C.: Dec. 19, 2003); Highway and Transit Investments: 
Options for Improving Information on Projects' Benefits and Costs and 
Increasing Accountability for Results, GAO-05-172 (Washington, D.C.: 
Jan. 24, 2005); and Freight Transportation: Short Sea Shipping Option 
Shows Importance of Systematic Approach to Public Investment Decisions, 
GAO-05-768 (Washington, D.C.: July 29, 2005).

[38] Department of Homeland Security Appropriations Act, 2006, Pub. L. 
No. 109-90, 119 Stat. 2064, 2075 (2005). The appropriation incorporates 
by reference the federal matching requirement of another federal grant 
program contained at 46 U.S.C. 70107(c). The match provision allows 
federal funding up to 75 percent of the total cost of the project, 
unless the DHS Secretary determines that a proposed project merits 
support and cannot be undertaken without a higher rate of federal 
support.

[39] Fortune magazine ranks the nation's largest companies on the basis 
of revenue.

[40] Management for the fiscal year 2004 program was transferred from 
the Transportation Security Agency to ODP during the middle of the 
process. ODP assumed full responsibility for the program in fiscal year 
2005.

[41] For the 2005 program, the evaluation at the port level is managed 
by the Coast Guard Captain of the Port and state officials when 
feasible. Applications are reviewed against established criteria and 
ranked on relative risk, with the highest rank going to applications 
that support the national port security priorities.

[42] ODP defines risk as credible threats and incidents (from the 
intelligence community), less credible threats and incidents 
(operational indicators), and vessels of interest. It defines 
vulnerability as distance from open water, number of port calls, and 
presence of tankers, and it defines consequence as people, economic, 
national security, and port-specific special considerations (such as 
hazardous materials or oil).

[43] Although ODP has primary responsibility for administering the 
grant program, the Coast Guard plays a key role in the program. Local 
Captains of the Port use the PS-RAT in evaluating grant applications at 
the local level, and ODP officials consult frequently with Coast Guard 
officials at the national level.

[44] ODP relied on threat-related information provided by IAIP for the 
fiscal year 2005 grant program.

[45] ODP worked with the Coast Guard and IAIP in developing the 
consequence measures.

[46] When the port security grant program was established, according to 
a program official, it was one of the first programs to use the PS-RAT 
in order to identify risks. The PS-RAT has been very useful on the 
local level as a means for the Coast Guard to look at relative risk to 
assets within a port zone in a systematic way. In addition to being a 
primary tool for the port security grant program, the PS-RAT has also 
been used by members of the Area Transportation Security Committees to 
interact with one another and to plan their work.

[47] ODP worked with TSA staff during the assessment process.

[48] MTSA required owners and operators of facilities and vessels to 
develop and implement security plans by July 1, 2004. In all, about 
12,300 plans were developed. The Coast Guard is charged with approving 
the plans and ensuring owners and operators are complying with the 
actions called for and determining whether all vulnerabilities have 
been identified. Coast Guard compliance inspections were scheduled for 
completion by July 2005. Funds requested by owners and operators may 
reflect additional security measures taken that go beyond the minimum 
requirements called for in the security plans or they could involve 
additional technology improvements.

[49] GAO, Standards for Internal Control in the Federal Government, 
GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999).

[50] The Homeland Security Act incorporates the definition of "critical 
infrastructure" used in the USA PATRIOT Act of 2001, meaning "systems 
and assets, whether physical or virtual, so vital to the United States 
that the incapacity or destruction of such systems and assets would 
have a debilitating impact on security, national economic security, 
national public health or safety, or any combination of those matters." 
The Homeland Security Act defines "key resources" as "publicly or 
privately controlled resources essential to the minimal operations of 
the economy and government." 6 U.S.C. § 101.

[51] See generally 6 U.S.C. 121(d).

[52] Id. 121(d)(1), (2), (3), (6).

[53] The National Strategy for the Physical Protection of Critical 
Infrastructures and Key Assets identifies five key assets. The key 
assets and lead federal agencies are as follows: national monuments and 
icons (Department of the Interior); dams, locks, and levees (Department 
of Homeland Security); government facilities (Department of Homeland 
Security); commercial and community assets (Department of Homeland 
Security); and nuclear reactors, materials, and spent fuel (Department 
of Homeland Security, working with the Nuclear Regulatory Commission 
and, as appropriate, the Department of Energy). When we use the term 
"critical infrastructure" in this chapter, we are referring both to the 
13 sectors and to the five key assets.

[54] The National Strategy for the Physical Protection of Critical 
Infrastructures and Key Assets is complemented by the National Strategy 
to Secure Cyberspace. Also issued in February 2003, the National 
Strategy to Secure Cyberspace provides direction to the federal 
government departments and agencies that have roles in cyberspace 
security and identifies steps that state and local governments, private 
companies and organizations, and individuals can take to improve 
collective cybersecurity.

[55] In November 2005, DHS issued a Draft National Infrastructure 
Protection Plan for comment. We did not evaluate the draft plan because 
our field work had already been completed.

[56] GAO, Critical Infrastructure Protection: Challenges in Addressing 
Cybersecurity, GAO-05-827T (Washington, D.C.: July 19, 2005).

[57] GAO did not analyze the data in the database and cannot comment on 
the database's reliability. The Congressional Research Service has 
raised a number of issues concerning the data integrity of the IAIP 
asset database. See Congressional Research Service, Risk Management and 
Critical Infrastructure Protection: Assessing, Integrating, and 
Managing Threats, Vulnerabilities, and Consequences (Washington, D.C.: 
February 2005).

[58] See 6 U.S.C.121(d)(2). IAIP considers the combination of the 
threat (relative likelihood of the attack type) and vulnerability 
(likelihood of the adversary's success) to represent the relative 
likelihood of a successful attack occurring.

[59] While the intelligence community does rank threats, IAIP does not 
find such information useful in developing data on threats. 
Intelligence components of DHS and other intelligence agencies 
routinely generate strategic threat assessment matrices and reports 
detailing the most likely targets and modes of terrorist attack. These 
assessments break down infrastructure and assets into categories and 
rank them, numerically or otherwise, according to the relative 
likelihood of attack within and across categories. However, IAIP 
officials said that the information was not specific to individual 
assets.

[60] After grouping all components of critical infrastructure by 
sector, IAIP plans to ask the owners or operators to complete a 
questionnaire--called a "top screen"--that provides an assessment of 
consequential losses arising from a worst-case scenario that assumes 
total loss of the asset. Once this information is gathered and 
assessed, assets considered to be of sufficient consequence are 
evaluated on vulnerability, using three elements: potential method of 
attack, probability of success, and consequences of the attack 
(including secondary and tertiary effects). By applying this 
information to the selected subset of assets, IAIP seeks to identify 
the assets of greatest vulnerability and identify strategies that hold 
the greatest potential benefits. Much like the consequence assessment, 
the vulnerability assessment is carried out by eliciting the opinions 
of experts, usually the owner/operator, for a specific piece of 
infrastructure.

[61] IAIP has plans to address the risks involved in protecting 
additional critical infrastructures, such as stadiums, though a tool 
called the Vulnerability, Identification, and Self-Assessment Tool. 
This tool will allow owners and operators to assess vulnerabilities in 
and around their facilities.

[62] IAIP and the private sector agreed that the goal of the self- 
assessments was to satisfy the information needs of DHS.

[63] The eight categories of assets are commercial nuclear power 
plants, commercial spent nuclear fuel facilities, chemical plants, 
petroleum refineries, liquefied natural gas storage facilities, subway 
systems (including bridges and tunnels), railroad systems (including 
bridges and tunnels), and highway systems (including bridges and 
tunnels). The vulnerability assessments for the chemical sector and 
nuclear power plants were pilot projects.

[64] GAO-05-851.

[65] OMB Circulars A-11 and A-94.

[66] GAO, Critical Infrastructure Protection: Challenges for Selected 
Agencies and Industry Sectors, GAO-03-233 (Washington D.C.: Feb. 28, 
2003); and Homeland Security: Information Sharing Responsibilities, 
Challenges, and Key Management Issues, GAO-03-715T (Washington D.C.: 
May 8, 2003).

[67] GAO, Maritime Security: New Structures Have Improved Information 
Sharing, but Security Clearance Processing Requires Further Attention, 
GAO-05-394 (Washington, D.C.: Apr. 15, 2005).

[68] GAO, Combating Terrorism: Threat and Risk Assessments Can Help 
Prioritize and Target Program Investments, GAO/NSIAD-98-74 (Washington, 
D.C.: April 1998); Combating Terrorism: Evaluation of Selected 
Characteristics in National Strategies Related to Terrorism, GAO-04-
408T (Washington, D.C.: Feb. 3, 2004); Rail Security: Some Actions 
Taken to Enhance Passenger and Freight Rail Security, but Significant 
Challenges Remain, GAO-04-598T (Washington, D.C.: Mar. 23, 2004); and 
Homeland Security: Summary of Challenges Faced in Targeting Oceangoing 
Cargo Containers for Inspection, GAO-04-557T (Washington, D.C.: Mar. 
31, 2004).

[69] GAO, Agencies' Strategic Plans under GPRA: Key Questions to 
Facilitate Congressional Review, GAO/GGD-10.1.16 (Washington, D.C.: May 
1, 1997).

[70] GAO, Homeland Security: Key Elements to Unify Efforts Are Underway 
but Uncertainty Remains, GAO-02-610 (Washington, D.C.: June 7, 2002).

[71] The Gilmore Commission (also known as the Advisory Panel to Assess 
Domestic Response Capabilities for Terrorism Involving Weapons of Mass 
Destruction), Forging America's New Normalcy (Arlington, Virginia: Dec. 
15, 2003).

[72] See Yacov Y. Haimes, Risk Modeling, Assessment, and Management, 
2nd ed. (Hoboken, New Jersey: Wiley and Sons, 2004).

[73] Executive Order 12866 and circulars A-4 and A-94 apply to 
regulatory actions, and circulars A-11 (sect. 7), A-94, and A-130 apply 
to capital investments.

[74] GAO, Standards for Internal Control in the Federal Government, 
GAO/AIMD-00-21.3.1, (Washington, D.C.: November 1999).

[75] For a more detailed discussion on this topic, see Yacov Y Haimes, 
Risk Modeling, Assessment, and Management, 2nd ed. (Hoboken, New 
Jersey: Wiley and Sons, 2004).

[76] See for example, GAO, Homeland Security: Summary of Challenges 
Faced in Targeting Oceangoing Cargo Containers for Inspection, GAO-04-
557T (Washington, D.C.: Mar. 31, 2004); Homeland Security: A Risk 
Management Approach Can Guide Preparedness Efforts, GAO-02-208T 
(Washington, D.C.: Oct. 31, 2001); and Key Elements of a Risk 
Management Approach, GAO-02-150T (Washington, D.C.: Oct. 12, 2001).

GAO's Mission: 

The Government Accountability Office, the investigative arm of 
Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site (www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading. 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548: 

To order by Phone: 

Voice: (202) 512-6000: 

TDD: (202) 512-2537: 

Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470: 

Public Affairs: 

Jeff Nelligan, managing director, 

NelliganJ@gao.gov 

(202) 512-4800 

U.S. Government Accountability Office, 

441 G Street NW, Room 7149 

Washington, D.C. 20548: