Risk Management:

Further Refinements Needed to Assess Risks and Prioritize Protective Measures at Ports and Other Critical Infrastructure

GAO-06-91: Published: Dec 15, 2005. Publicly Released: Jan 17, 2006.

Additional Materials:

Contact:

Stephen L. Caldwell
(415) 904-2200
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Congress and the President have called for various homeland security efforts to be based on risk management--a systematic process for assessing threats and taking appropriate steps to deal with them. GAO examined how three Department of Homeland Security (DHS) components were carrying out this charge: the Coast Guard, which has overall responsibility for security in the nation's ports; the Office for Domestic Preparedness (ODP), which awards grants for port security projects; and the Information Analysis and Infrastructure Protection Directorate (IAIP), which has responsibility for developing ways to assess risks across all types of critical infrastructure. GAO's work focused on identifying the progress each DHS component has made on risk management and the challenges each faces in moving further.

The three DHS components GAO studied varied considerably in their progress in developing a sound risk management framework for homeland security responsibilities. The varied progress reflects, among other things, each component's organizational maturity and the complexity of its task. The Coast Guard, which is furthest along, is the component of longest standing, being created in 1915, while IAIP came into being with the creation of the Department of Homeland Security in 2003. IAIP, which has made the least progress, is not only a new component but also has the most complex task--addressing not just ports but all types of infrastructure. The Coast Guard and ODP have a relatively robust methodology in place for assessing risks at ports; IAIP is still developing its methodology and has had several setbacks in completing the task. All three components, however, have much left to do. In particular, each component is limited in its ability to compare and prioritize risks. The Coast Guard and ODP can do so within a port but not between ports; IAIP has not demonstrated that it can do so either within or between all infrastructure sectors. Each component faces many challenges in making further progress. Success will depend partly on continuing to improve various technical and management processes that are part of risk management. For example, obtaining better quality data from intelligence agencies would help DHS components estimate the relative likelihood of various types of threats--a key element of assessing risks. In the longer term, progress will depend increasingly on how well risk management is coordinated across agencies, because current approaches in many ways are neither consistent nor comparable. Also, weaving risk-based data into the annual budget cycle of program review will be important. Supplying the necessary guidance and coordination is what the Department of Homeland Security was set up to do and, as the Secretary of Homeland Security has stated, what it now needs increasingly to address. This is a key issue for the department as it seeks to identify relative risks and take appropriate actions related to the nation's homeland security activities.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: The Secretary of Homeland Security should direct the Commandant of the Coast Guard to take action in the area of risk assessment by developing plans to establish a stronger linkage between local and national risk assessment efforts. This effort could involve strengthening the ties between local assessment efforts, such as area maritime security plans, and national risk assessment activities.

    Agency Affected: Department of Homeland Security: United States Coast Guard

    Status: Closed - Implemented

    Comments: In fiscal year 2006, we analyzed how multiple Department of Homeland Security (DHS) Components, including the United States Coast Guard, were basing their homeland security efforts on risk management--a systematic process for assessing threats and taking appropriate steps to deal with them. We reported, among other things, that the Coast Guard had developed the ability to compare and prioritize risks at individual ports but it could not yet compare and prioritize relative risks of various infrastructure across ports. In 2006, the Coast Guard transitioned its risk assessment model from the Port Security Risk Assessment Tool to the Maritime Security Risk Analysis Model (MSRAM); a tool based on the risk management framework proposed in GAO-06-91. MSRAM is a security risk analysis tool that assists in the prioritizing of relative risks associated with critical infrastructure across ports. It is designed to capture the security risk facing different types of targets spanning every industry sector, allowing comparison between different targets and geographic areas at the local, regional, and national levels. It does this by assessing the risk posed by different scenarios in terms of threat, vulnerability, and consequence. Coast Guard officials said that MSRAM continues to evolve and that it will be the risk management tool used by the Coast Guard moving forward. In prior years the decision to use MSRAM was communicated using Navigation and Vessel Inspection Circulars and message traffic. However, officials noted a contractor was hired to provide support to MSRAM stakeholders and will help develop a Commandant's Instruction addressing the use of MSRAM. As such, the Coast Guard has developed and is using a tool that enables it to establish a stronger linkage between local and national risk assessment efforts.

    Recommendation: The Secretary of Homeland Security should direct the Commandant of the Coast Guard to take action in the area of alternatives evaluation and management selection by ensuring that procedures for these two processes consider the most efficient use of resources. For example, one approach involves refining the degree to which risk management information is integrated into the annual cycle of program and budget review.

    Agency Affected: Department of Homeland Security: United States Coast Guard

    Status: Closed - Implemented

    Comments: In 2006 we reported that just as the Coast Guard's ability to assess risk is stronger at the individual port level than across ports, its ability to evaluate various alternatives for addressing these risks is greater at the port level as well. Part of this limitation is due to the Port Security Risk Assessment Tool (PS-RAT), which was designed to allow ports to prioritize resource allocations within, not between, ports to address risk most efficiently. We said data from PS-RAT help identify vulnerabilities within a port and can be used in improving security measures related to the area maritime security plans. PS-RAT is not designed to work, however, above the port level. At the national level, the Coast Guard (CG) had conducted qualitative evaluations of the potential benefits of various alternatives for reducing risk levels, such as improved information sharing through the use of interagency operational centers, waterborne patrols, and escorting ships. Since that time the Coast Guard has transitioned its risk assessment model from the PS-RAT to the Maritime Security Risk Analysis Model (MSRAM); a tool based on the risk management framework proposed in GAO-06-91. MSRAM is a security risk analysis tool that assists in the prioritizing of relative risks associated with critical infrastructure across ports. It is designed to capture the security risk facing different types of targets spanning every industry sector, allowing comparison between different targets and geographic areas at the local, regional, and national levels. The 2011 Congressional Budget Justification shows that the CG uses risk or relative risk to direct resources to mitigation of the highest risk. For example, the FY 2011 Coast Guard budget request preserves basic Search and Rescue requirements (i.e., minimum required asset readiness) through asset reallocation and risk management. The FY 2011 budget also proposes decommissioning five of the Coast Guard's twelve (42%) Maritime Safety and Security Teams (MSSTs). The request states that reducing MSST capacity is a risk-based decision to optimally allocate resources within current fiscal constraints. More specific to port security, the Ports, Waterways and Coastal Security (PWCS) program has a performance goal to manage terror-related risk in the U.S. Maritime Domain to an acceptable level. The Coast Guard uses the PWCS Program Efficiency (Outcome Performance/Program Cost) measure to direct resources to port security programs. This measure is the program's annual percent risk reduction outcome performance divided by the program's annual cost. Efficiency is expressed as the annual percent risk reduction per billion. (This measure was baselined in FY 2005). Thus, risk management drives resource allocation across Coast Guard missions.

    Recommendation: To strengthen ODP efforts to implement a risk management approach to its port security grant program, the Secretary of Homeland Security should direct the Executive Director for ODP to clarify, in its grant guidance, the conditions under which greater leveraging of federal dollars should be included as a strategic goal for the port security grant program.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Office for Domestic Preparedness

    Status: Closed - Implemented

    Comments: In 2006 we found that the Office of Domestic Preparedness (ODP) (now within FEMA) had made progress setting goals, the first phase of GAO's risk management framework for the port security grant program. Congress and the Administration had laid out broad policy goals for maritime security and for the grant program. Congress's stated purpose in establishing the program was to finance the costs of enhancing facility and operational security at critical national seaports. We also reported a challenge DHS faced involved determining an appropriate way to ensure that grants address key needs while at the same time ensuring that they make the most efficient use of federal dollars. We reported in many federal grant programs, the desired outcome is that federal grants supplement what other stakeholders are willing to spend. If a grant program is not designed to encourage supplementation, or grant sharing, the danger is that other stakeholders will rely solely on the federal funds and choose to use their own funds for other purposes. Since that time, we found that 2008 Port Security Grant Program (PSGP) guidelines define the level of cost-sharing required for grant award. As stated in the 2008 PSGP guidelines, the objective is to leverage federal resources to the greatest extent possible. For example, public service applicants for grants must provide proof that 25% of the cost will be provided from other sources while private-sector applicants must produce proof that matching funds from private sources support 50% of the total project costs. This cost-sharing approach meets the objective of leveraging federal dollars for port security.

    Recommendation: To strengthen ODP efforts to implement a risk management approach to its port security grant program, the Secretary of Homeland Security should direct the Executive Director for ODP to develop measurable objectives for managing the grant program's progress toward achieving strategic goals and use these measures to gauge progress and make adjustments to the program.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Office for Domestic Preparedness

    Status: Closed - Implemented

    Comments: In 2006 we reported that the evaluation of alternatives in risk management is an area that the Office for Domestic Preparedness (ODP) now within FEMA) recognizes as being an important part of awarding port security grants. We reported that one change that was instituted for the fiscal year 2005 grant process involved additional steps to consider benefits and costs. We said when ODP asked local Coast Guard Captains of the Port to review applications, one criterion it asked them to apply is to determine which projects offer the highest potential for risk reduction for the least cost. We said ODP's ability to assess proposed security improvements, like the Coast Guard's, is influenced by the program goals and performance measures that the component sets and the reliability and completeness of the risk assessments that it carries out. However, when measurable objectives are missing, the degree to which security gaps remain and the extent to which progress has been made remain unclear. Similarly, while the Port Security Risk Assessment Tool (PS-RAT) provided a starting point for evaluating the proposed measures and the extent to which the measure narrows security gaps within a port, it was not designed to compare and prioritize relative risks from one port to relative risks in a different port. 2008 PSGP guidelines instruct the Director to direct grant awards to the proposals that address risk to the greatest degree. The MSRAM risk calculation, which calculates risk by individual ports and allows for comparison across ports, are updated annually and used in the PSGP award process. For each port, a total score is computed; with all proposals received from each port being ranked from highest to lowest in terms of their contributions to regional risk reduction and cost effectiveness. Furthermore, the PSGP guidance states that the Department of Homeland Security will focus the bulk of its available port grant dollars on the highest-risk port systems (known as groups 1 and II). For other ports, no more than 20% of the total award amount may be used in the development of the Port Area-Wide Risk Management/Mitigation Plan and optional Business Continuity/Resumption of Trade Plans. Remaining funds (80% of the total) will then be used to implement prioritized projects that provide the greatest risk reduction benefit for the port area as a whole. The use of risk calculation to identify ports for grant priority and potential risk reduction for further assessing grant eligibility addresses the strategic goal to reduce the risk to ports and provides a means to measure progress in addressing that goal.

    Recommendation: To strengthen ODP efforts to implement a risk management approach to its port security grant program, the Secretary of Homeland Security should direct the Executive Director for ODP to coordinate efforts with the Coast Guard and IAIP to use more reliable risk assessment data as they become available. At a minimum, such data should include (1) the relative likelihood of various threat scenarios, (2) consequences and vulnerabilities that are linked to terrorist scenarios, and (3) a comparison of risks across ports.

    Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Office for Domestic Preparedness

    Status: Closed - Implemented

    Comments: In 2006, we reported on ODP's (now within FEMA) adjustments to its fiscal year 2005 Port Security Grant Program (PSGP) procedures at the national level, and that it had made a concerted effort to narrow the program to ports of greatest concern, and to use threat, vulnerability, and consequence data to rank and prioritize both ports and applications. Our review of ODP's risk assessment approach and our discussions with ODP and Coast Guard personnel identified several challenges related to limitations regarding the existing data on threats, vulnerabilities, and consequences. We also noted there was a key methodological limitation at the time that affected one goal of risk assessments: informing decision makers on relative risks across port locations. As noted at the time, the Coast Guard used the Port Security Risk Assessment Tool (PS-RAT) which provided information on vulnerability and consequence, but could not be used to compare the risk at one port with that of another. Since then the Coast Guard transitioned its risk assessment model from the PS-RAT to the Maritime Security Risk Analysis Model (MSRAM); a tool based on the risk management framework proposed in GAO-06-91. MSRAM is a security risk analysis tool that assists in the prioritization of relative risks associated with critical infrastructure across ports. It is designed to capture the security risk facing different types of targets spanning every industry sector, allowing comparison between different targets and geographic areas at the local, regional, and national levels. It does this by assessing the risk posed by different scenarios in terms of threat, vulnerability, and consequence. Coast Guard have officials said that MSRAM continues to evolve and that it will be the risk management tool used by the Coast Guard moving forward. According to the 2008 PSGP guidelines, the DHS risk assessment methodology for the PSGP includes multiple data sets regarding length of port channel; military mission variables; adjacent critical asset inventories; Coast Guard MSRAM data; and international cargo value and measures of cargo throughput (container, break bulk, international and domestic).The use of MSRAM in this determination allows grant award determination to be based on criteria that address the relative likelihood of various threat scenarios; consequences and vulnerabilities that are linked to terrorist scenarios; and a comparison of risks across ports, meeting the objective of our recommendation.

    Recommendation: To help ensure the development of risk management approaches to homeland security activities, the Secretary of Homeland Security should direct the Undersecretary for IAIP to work with the intelligence community to develop ways to better assess terrorist threats and use available information and expert judgment to develop a relative probability for various terrorist scenarios and provide this information to sector-specific agencies.

    Agency Affected: Department of Homeland Security: Directorate of Information Analysis and Infrastructure Protection

    Status: Closed - Implemented

    Comments: In fiscal year 2006, we analyzed how multiple Department of Homeland Security (DHS) components-including its Information Analysis and Infrastructure Protection component(now known as the Office of Infrastructure Protection and the Information Analysis Directorate) were basing their homeland security efforts on risk management: a systematic process for assessing threats and taking appropriate steps to deal with them. We reported, among other things, that these groups face challenges in developing data on the relative likelihood of various threat scenarios--a key part of the assessments it must conduct under the Homeland Security Act of 2002--because the information produced by the intelligence community was of limited use for risk assessment purposes. At the time our report was published, DHS officials said that they planned to develop such data by coordinating more closely with the intelligence community. Subsequently, DHS developed and implemented the State Homeland Infrastructure Risk Assessment (SHIRA) process. This process assesses a set of attack methods on a sector-by-sector basis-using information from multiple sources including the intelligence community-to determine the threat each attack method poses. DHS then assesses each sector for vulnerabilities to each type of attack and estimates the consequences if each type of attack were successful. DHS combines the assessments of threat, vulnerability and consequences into an overarching assessment of risk to each sector for each attack method as well as a national level risk profile. As a result, DHS has demonstrated that it has developed a process to assess terrorist threats and compute the relative probability those threats will manifest and is distributing that information to the sector-specific agencies through the annual Strategic Homeland Infrastructure Risk Assessment Report.

    Recommendation: To help ensure the development of risk management approaches to homeland security activities, the Secretary of Homeland Security should direct the Undersecretary for IAIP to, as tasked by presidential directive, develop a methodology for comparing and prioritizing risks of assets within and across infrastructure sectors by including data on the relative probability of various threat scenarios.

    Agency Affected: Department of Homeland Security: Directorate of Information Analysis and Infrastructure Protection

    Status: Closed - Implemented

    Comments: In fiscal year 2006, we analyzed how multiple Department of Homeland Security (DHS) Components-including its Information Analysis and Infrastructure Protection component(now known as the Office of Infrastructure Protection and the Information Analysis Directorate) were basing their homeland security efforts on risk management; a systematic process for assessing threats and taking appropriate steps to deal with them. We reported, among other things, that these groups face challenges in developing data on the relative likelihood of various threat scenarios--a key part of the assessments it must conduct under the Homeland Security Act of 2002--because the information produced by the intelligence community was of limited use for risk assessment purposes. At the time our report was published DHS officials said that they planned to develop such data by coordinating more closely with the intelligence community. Subsequently, DHS developed and implemented the State Homeland Infrastructure Risk Assessment (SHIRA) process. This process assesses a set of attack methods on a sector-by-sector basis-using information from multiple sources including the intelligence community-to determine the threat each attack method poses. DHS then assesses each sector for vulnerabilities to each type of attack and estimates the consequences if each type of attack were successful. DHS combines the assessments of threat, vulnerability and consequences into an overarching assessment of risk to each sector for each attack method as well as a national level risk profile. As a result, DHS has demonstrated that it has developed a process to assess terrorist threats and compute the relative probability those threats will manifest and is distributing that information to the sector-specific agencies through the annual Strategic Homeland Infrastructure Risk Assessment Report.

    Recommendation: To help ensure the development of risk management approaches to homeland security activities, the Secretary of Homeland Security should direct the Undersecretary for IAIP to, in completing the National Infrastructure Protection Plan, include target dates for completing sector-specific plans, developing performance measures, and identifying protective measures that could address multiple threat scenarios.

    Agency Affected: Department of Homeland Security: Directorate of Information Analysis and Infrastructure Protection

    Status: Closed - Implemented

    Comments: In fiscal year 2006, we analyzed how multiple Department of Homeland Security (DHS) Components-including its Information Analysis and Infrastructure Protection (IAIP) Directorate(now known as the Office of Infrastructure Protection and the Information Analysis Directorate) were basing their homeland security efforts on risk management; a systematic process for assessing threats and taking appropriate steps to deal with them. We reported, among other things, that IAIP's progress in all five phases of risk management has been limited. Specifically, despite issuing an Interim National Infrastructure Protection Plan (NIPP) in February 2005 IAIP faced developing performance measures to evaluate progress and establishing milestones and timeframes for processing and prioritizing assets across the infrastructure sectors. DHS issued the NIPP in 2006 obtaining letters of agreement from multiple federal agencies which agreed to provide DHS with annual reports on their efforts to identify, prioritize, and coordinate CI/KR protection in their respective sectors and coordinate the development of Sector-Specific Plans (SSPs). All 17 of the sectors published their sector plans in December 2006 including narratives about the protective programs--some of which could address multiple types of threats--being developed and implemented in each sector and reporting information on core metrics designed to measure NIPP implementation. However, the NIPP requires each sector is required to develop sector specific metrics. As of November and December of 2008 not all of the 18 sectors (an additional sector was added to the NIPP framework in 2008) had developed their sector-specific metrics. However, DHS officials told us that all 18 sectors are expected to develop the sector specific performance measures during the Spring of 2009 and submit data on those measures in 2010.

    Recommendation: To strengthen individual agency efforts to implement a risk management approach to homeland security activities, the Secretary of Homeland Security direct the Undersecretary for IAIP to, as required by presidential directive, establish uniform policies, approaches, guidelines, and methodologies for integrating federal infrastructure protection and risk management activities within and across sectors, along with metrics and criteria for related programs and activities and develop a timetable for completing such guidance. Such policies and guidance should address the issue of integrating risk management systems into existing systems of program and budget review.

    Agency Affected: Department of Homeland Security: Directorate of Information Analysis and Infrastructure Protection

    Status: Closed - Implemented

    Comments: In fiscal year 2006, we reported, among other things, that Information Analysis and Infrastructure Protection IAIP) has been challenged in establishing uniform policies, approaches, guidelines, and methodologies for integrating federal infrastructure protection and risk management activities within and across sectors, along with metrics and criteria for related programs and activities as called for by the Homeland Security Presidential Directive-7 (HSPD-7). Since 2006, DHS has implemented and updated the National Infrastructure Protection Plan (NIPP), its Risk Lexicon, and the NIPP implementation guidance thereby establishing uniform policies, approaches, guidelines and methodologies for integrating federal infrastructure protection and risk management activities within and across sectors. The NIPP also addresses the issue of integrating risk management systems into existing systems of program and budget review. The 2009 version of the NIPP identifies DHS as being responsible for informing the Federal Budget process on risk management. The NIPP also delineates roles and responsibilities for the sector specific agencies (SSA) such as establishing an annual budget process for outlining sector specific critical infrastructure and key resources (CIKR) protection requirements and related budget projections, to the extent possible, as a component of SSA annual budget submissions. According to the 2009 NIPP, maximizing the efficient use of resources for CIKR protection includes a coordinated and integrated annual process for program implementation that informs the annual Federal process regarding planning, programming, and budgeting for national-level CIKR protection. To further demonstrate that SSAs are held accountable for integrating risk management and agency budget review, the National CIKR Protection Annual Report is submitted along with the DHS budget submission to the Executive Office of the President on or before September 1 as part of the annual Federal budget process. The SSAs submit the CIKR protection priorities and requirements to DHS in their sector annual reports. The SSAs work within their respective department or agency budget process to determine the CIKR protection-related aspects of their department's budget submission. Furthermore, the 2009 NIPP states the use of performance metrics is a critical step in the NIPP risk management process to enable DHS and its partners to objectively and quantitatively assess improvements in CIKR protection and resiliency at the sector and national levels. To this end DHS issued NIPP Metrics Guidance in February 2009 to assist sectors in the development of performance measures to measure sector progress in CIKR protection. The guidance stressed the importance of metrics development to the budget process when it stated that NIPP metrics inform current and prospective allocation of resources in light of previously implemented protective actions or other factors. The DHS guidance states DHS will work with each sector to develop metrics that focus on outcomes the sector are trying to achieve, understanding that this is an ongoing process. The 2009 National CIKR Protection Annual Report identifies hundreds of risk mitigation activities (RMA)or programs designed to reduce the risk to CIKR sectors. A review of selected 2009 sector annual reports indicates that the reports contain risk mitigation activities that include progress indicators that may contain outcome metrics that provide an indicator of changes to the level of risk in a specific sector due to the implementation of the RMAs contained in the reports.

    Recommendation: To strengthen individual agency efforts to implement a risk management approach to homeland security activities, the Secretary of Homeland Security direct the Undersecretary for IAIP to, as DHS continues to review its organizational structure, work with the Secretary's office to determine which office is best suited to help ensure that the responsibility for risk management policy and implementation has a broad enough perspective on all elements of risk, including threats, as well as the necessary authority to coordinate with DHS component agencies and hold them accountable for risk management activities.

    Agency Affected: Department of Homeland Security: Directorate of Information Analysis and Infrastructure Protection

    Status: Closed - Implemented

    Comments: In fiscal year 2006, we reported, among other things, Information Analysis and Infrastructure Protection's (IAIP) (now National Protection and Programs Directorate) risk management efforts were focused mainly on assessing and reducing vulnerabilities which had the potential of limiting DHS's ability to achieve the broader goal of using risk-based data as a tool to inform management decisions. In 2007, the Secretary for the DHS issued Delegation Number 17001 which delegated authority to the Under Secretary for the National Protection and Programs Directorate (NPPD) for managing risk to the nation's critical infrastructure and key resources (CIKR). The Under Secretary, in collaboration with the Office of Risk Management and Analysis (RMA), implemented a Department-wide Risk Steering Committee to serve as DHS's risk management governance structure to further hold DHS accountable for risk management activities, thereby enabling the sharing and integration of risk management efforts. The RMA and Risk Steering Committees were both implemented in April 2007. The Risk Steering Committee is a cooperative body formed to ensure that risk management is carried out consistently and comparably throughout the Department. Chaired by the Under Secretary for National Protection and Programs and comprising component heads and other identified personnel, the committee assists in the framing of processes and procedures for the Department's risk-management architecture, enabling collaboration and Department-wide agreement on risk-management efforts. According to the Delegation directive the Under Secretary also has the authority to coordinate with other DHS components to synchronize risk management programs.

    Recommendation: To strengthen individual agency efforts to implement a risk management approach to homeland security activities, the Secretary of Homeland Security direct the Undersecretary for IAIP to work with the Office of Management and Budget to examine options for holding departments and agencies accountable for integrating risk management for homeland security programs and activities into the annual cycle of program and budget review.

    Agency Affected:

    Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Jul 31, 2014

    Jul 29, 2014

    Jul 24, 2014

    Jul 16, 2014

    Jun 27, 2014

    Jun 24, 2014

    Jun 23, 2014

    Jun 18, 2014

    Looking for more? Browse all our products here