Risk Management: Further Refinements Needed to Assess Risks and Prioritize Protective Measures at Ports and Other Critical Infrastructure

In fiscal year 2006, we analyzed how multiple Department of Homeland Security (DHS) Components, including the United States Coast Guard, were basing their homeland security efforts on risk management--a systematic process for assessing threats and taking appropriate steps to deal with them. We reported, among other things, that the Coast Guard had developed the ability to compare and prioritize risks at individual ports but it could not yet compare and prioritize relative risks of various infrastructure across ports. In 2006, the Coast Guard transitioned its risk assessment model from the Port Security Risk Assessment Tool to the Maritime Security Risk Analysis Model (MSRAM); a tool based on the risk management framework proposed in GAO-06-91. MSRAM is a security risk analysis tool that assists in the prioritizing of relative risks associated with critical infrastructure across ports. It is designed to capture the security risk facing different types of targets spanning every industry sector, allowing comparison between different targets and geographic areas at the local, regional, and national levels. It does this by assessing the risk posed by different scenarios in terms of threat, vulnerability, and consequence. Coast Guard officials said that MSRAM continues to evolve and that it will be the risk management tool used by the Coast Guard moving forward. In prior years the decision to use MSRAM was communicated using Navigation and Vessel Inspection Circulars and message traffic. However, officials noted a contractor was hired to provide support to MSRAM stakeholders and will help develop a Commandant's Instruction addressing the use of MSRAM. As such, the Coast Guard has developed and is using a tool that enables it to establish a stronger linkage between local and national risk assessment efforts.

In 2006 we reported that just as the Coast Guard's ability to assess risk is stronger at the individual port level than across ports, its ability to evaluate various alternatives for addressing these risks is greater at the port level as well. Part of this limitation is due to the Port Security Risk Assessment Tool (PS-RAT), which was designed to allow ports to prioritize resource allocations within, not between, ports to address risk most efficiently. We said data from PS-RAT help identify vulnerabilities within a port and can be used in improving security measures related to the area maritime security plans. PS-RAT is not designed to work, however, above the port level. At the national level, the Coast Guard (CG) had conducted qualitative evaluations of the potential benefits of various alternatives for reducing risk levels, such as improved information sharing through the use of interagency operational centers, waterborne patrols, and escorting ships. Since that time the Coast Guard has transitioned its risk assessment model from the PS-RAT to the Maritime Security Risk Analysis Model (MSRAM); a tool based on the risk management framework proposed in GAO-06-91. MSRAM is a security risk analysis tool that assists in the prioritizing of relative risks associated with critical infrastructure across ports. It is designed to capture the security risk facing different types of targets spanning every industry sector, allowing comparison between different targets and geographic areas at the local, regional, and national levels. The 2011 Congressional Budget Justification shows that the CG uses risk or relative risk to direct resources to mitigation of the highest risk. For example, the FY 2011 Coast Guard budget request preserves basic Search and Rescue requirements (i.e., minimum required asset readiness) through asset reallocation and risk management. The FY 2011 budget also proposes decommissioning five of the Coast Guard's twelve (42%) Maritime Safety and Security Teams (MSSTs). The request states that reducing MSST capacity is a risk-based decision to optimally allocate resources within current fiscal constraints. More specific to port security, the Ports, Waterways and Coastal Security (PWCS) program has a performance goal to manage terror-related risk in the U.S. Maritime Domain to an acceptable level. The Coast Guard uses the PWCS Program Efficiency (Outcome Performance/Program Cost) measure to direct resources to port security programs. This measure is the program's annual percent risk reduction outcome performance divided by the program's annual cost. Efficiency is expressed as the annual percent risk reduction per billion. (This measure was baselined in FY 2005). Thus, risk management drives resource allocation across Coast Guard missions.

In 2006 we found that the Office of Domestic Preparedness (ODP) (now within FEMA) had made progress setting goals, the first phase of GAO's risk management framework for the port security grant program. Congress and the Administration had laid out broad policy goals for maritime security and for the grant program. Congress's stated purpose in establishing the program was to finance the costs of enhancing facility and operational security at critical national seaports. We also reported a challenge DHS faced involved determining an appropriate way to ensure that grants address key needs while at the same time ensuring that they make the most efficient use of federal dollars. We reported in many federal grant programs, the desired outcome is that federal grants supplement what other stakeholders are willing to spend. If a grant program is not designed to encourage supplementation, or grant sharing, the danger is that other stakeholders will rely solely on the federal funds and choose to use their own funds for other purposes. Since that time, we found that 2008 Port Security Grant Program (PSGP) guidelines define the level of cost-sharing required for grant award. As stated in the 2008 PSGP guidelines, the objective is to leverage federal resources to the greatest extent possible. For example, public service applicants for grants must provide proof that 25% of the cost will be provided from other sources while private-sector applicants must produce proof that matching funds from private sources support 50% of the total project costs. This cost-sharing approach meets the objective of leveraging federal dollars for port security.

In 2006 we reported that the evaluation of alternatives in risk management is an area that the Office for Domestic Preparedness (ODP) now within FEMA) recognizes as being an important part of awarding port security grants. We reported that one change that was instituted for the fiscal year 2005 grant process involved additional steps to consider benefits and costs. We said when ODP asked local Coast Guard Captains of the Port to review applications, one criterion it asked them to apply is to determine which projects offer the highest potential for risk reduction for the least cost. We said ODP's ability to assess proposed security improvements, like the Coast Guard's, is influenced by the program goals and performance measures that the component sets and the reliability and completeness of the risk assessments that it carries out. However, when measurable objectives are missing, the degree to which security gaps remain and the extent to which progress has been made remain unclear. Similarly, while the Port Security Risk Assessment Tool (PS-RAT) provided a starting point for evaluating the proposed measures and the extent to which the measure narrows security gaps within a port, it was not designed to compare and prioritize relative risks from one port to relative risks in a different port. 2008 PSGP guidelines instruct the Director to direct grant awards to the proposals that address risk to the greatest degree. The MSRAM risk calculation, which calculates risk by individual ports and allows for comparison across ports, are updated annually and used in the PSGP award process. For each port, a total score is computed; with all proposals received from each port being ranked from highest to lowest in terms of their contributions to regional risk reduction and cost effectiveness. Furthermore, the PSGP guidance states that the Department of Homeland Security will focus the bulk of its available port grant dollars on the highest-risk port systems (known as groups 1 and II). For other ports, no more than 20% of the total award amount may be used in the development of the Port Area-Wide Risk Management/Mitigation Plan and optional Business Continuity/Resumption of Trade Plans. Remaining funds (80% of the total) will then be used to implement prioritized projects that provide the greatest risk reduction benefit for the port area as a whole. The use of risk calculation to identify ports for grant priority and potential risk reduction for further assessing grant eligibility addresses the strategic goal to reduce the risk to ports and provides a means to measure progress in addressing that goal.

In 2006, we reported on ODP's (now within FEMA) adjustments to its fiscal year 2005 Port Security Grant Program (PSGP) procedures at the national level, and that it had made a concerted effort to narrow the program to ports of greatest concern, and to use threat, vulnerability, and consequence data to rank and prioritize both ports and applications. Our review of ODP's risk assessment approach and our discussions with ODP and Coast Guard personnel identified several challenges related to limitations regarding the existing data on threats, vulnerabilities, and consequences. We also noted there was a key methodological limitation at the time that affected one goal of risk assessments: informing decision makers on relative risks across port locations. As noted at the time, the Coast Guard used the Port Security Risk Assessment Tool (PS-RAT) which provided information on vulnerability and consequence, but could not be used to compare the risk at one port with that of another. Since then the Coast Guard transitioned its risk assessment model from the PS-RAT to the Maritime Security Risk Analysis Model (MSRAM); a tool based on the risk management framework proposed in GAO-06-91. MSRAM is a security risk analysis tool that assists in the prioritization of relative risks associated with critical infrastructure across ports. It is designed to capture the security risk facing different types of targets spanning every industry sector, allowing comparison between different targets and geographic areas at the local, regional, and national levels. It does this by assessing the risk posed by different scenarios in terms of threat, vulnerability, and consequence. Coast Guard have officials said that MSRAM continues to evolve and that it will be the risk management tool used by the Coast Guard moving forward. According to the 2008 PSGP guidelines, the DHS risk assessment methodology for the PSGP includes multiple data sets regarding length of port channel; military mission variables; adjacent critical asset inventories; Coast Guard MSRAM data; and international cargo value and measures of cargo throughput (container, break bulk, international and domestic).The use of MSRAM in this determination allows grant award determination to be based on criteria that address the relative likelihood of various threat scenarios; consequences and vulnerabilities that are linked to terrorist scenarios; and a comparison of risks across ports, meeting the objective of our recommendation.

In fiscal year 2006, we analyzed how multiple Department of Homeland Security (DHS) components-including its Information Analysis and Infrastructure Protection component(now known as the Office of Infrastructure Protection and the Information Analysis Directorate) were basing their homeland security efforts on risk management: a systematic process for assessing threats and taking appropriate steps to deal with them. We reported, among other things, that these groups face challenges in developing data on the relative likelihood of various threat scenarios--a key part of the assessments it must conduct under the Homeland Security Act of 2002--because the information produced by the intelligence community was of limited use for risk assessment purposes. At the time our report was published, DHS officials said that they planned to develop such data by coordinating more closely with the intelligence community. Subsequently, DHS developed and implemented the State Homeland Infrastructure Risk Assessment (SHIRA) process. This process assesses a set of attack methods on a sector-by-sector basis-using information from multiple sources including the intelligence community-to determine the threat each attack method poses. DHS then assesses each sector for vulnerabilities to each type of attack and estimates the consequences if each type of attack were successful. DHS combines the assessments of threat, vulnerability and consequences into an overarching assessment of risk to each sector for each attack method as well as a national level risk profile. As a result, DHS has demonstrated that it has developed a process to assess terrorist threats and compute the relative probability those threats will manifest and is distributing that information to the sector-specific agencies through the annual Strategic Homeland Infrastructure Risk Assessment Report.

In fiscal year 2006, we analyzed how multiple Department of Homeland Security (DHS) Components-including its Information Analysis and Infrastructure Protection component(now known as the Office of Infrastructure Protection and the Information Analysis Directorate) were basing their homeland security efforts on risk management; a systematic process for assessing threats and taking appropriate steps to deal with them. We reported, among other things, that these groups face challenges in developing data on the relative likelihood of various threat scenarios--a key part of the assessments it must conduct under the Homeland Security Act of 2002--because the information produced by the intelligence community was of limited use for risk assessment purposes. At the time our report was published DHS officials said that they planned to develop such data by coordinating more closely with the intelligence community. Subsequently, DHS developed and implemented the State Homeland Infrastructure Risk Assessment (SHIRA) process. This process assesses a set of attack methods on a sector-by-sector basis-using information from multiple sources including the intelligence community-to determine the threat each attack method poses. DHS then assesses each sector for vulnerabilities to each type of attack and estimates the consequences if each type of attack were successful. DHS combines the assessments of threat, vulnerability and consequences into an overarching assessment of risk to each sector for each attack method as well as a national level risk profile. As a result, DHS has demonstrated that it has developed a process to assess terrorist threats and compute the relative probability those threats will manifest and is distributing that information to the sector-specific agencies through the annual Strategic Homeland Infrastructure Risk Assessment Report.

In fiscal year 2006, we analyzed how multiple Department of Homeland Security (DHS) Components-including its Information Analysis and Infrastructure Protection (IAIP) Directorate(now known as the Office of Infrastructure Protection and the Information Analysis Directorate) were basing their homeland security efforts on risk management; a systematic process for assessing threats and taking appropriate steps to deal with them. We reported, among other things, that IAIP's progress in all five phases of risk management has been limited. Specifically, despite issuing an Interim National Infrastructure Protection Plan (NIPP) in February 2005 IAIP faced developing performance measures to evaluate progress and establishing milestones and timeframes for processing and prioritizing assets across the infrastructure sectors. DHS issued the NIPP in 2006 obtaining letters of agreement from multiple federal agencies which agreed to provide DHS with annual reports on their efforts to identify, prioritize, and coordinate CI/KR protection in their respective sectors and coordinate the development of Sector-Specific Plans (SSPs). All 17 of the sectors published their sector plans in December 2006 including narratives about the protective programs--some of which could address multiple types of threats--being developed and implemented in each sector and reporting information on core metrics designed to measure NIPP implementation. However, the NIPP requires each sector is required to develop sector specific metrics. As of November and December of 2008 not all of the 18 sectors (an additional sector was added to the NIPP framework in 2008) had developed their sector-specific metrics. However, DHS officials told us that all 18 sectors are expected to develop the sector specific performance measures during the Spring of 2009 and submit data on those measures in 2010.

In fiscal year 2006, we reported, among other things, that Information Analysis and Infrastructure Protection IAIP) has been challenged in establishing uniform policies, approaches, guidelines, and methodologies for integrating federal infrastructure protection and risk management activities within and across sectors, along with metrics and criteria for related programs and activities as called for by the Homeland Security Presidential Directive-7 (HSPD-7). Since 2006, DHS has implemented and updated the National Infrastructure Protection Plan (NIPP), its Risk Lexicon, and the NIPP implementation guidance thereby establishing uniform policies, approaches, guidelines and methodologies for integrating federal infrastructure protection and risk management activities within and across sectors. The NIPP also addresses the issue of integrating risk management systems into existing systems of program and budget review. The 2009 version of the NIPP identifies DHS as being responsible for informing the Federal Budget process on risk management. The NIPP also delineates roles and responsibilities for the sector specific agencies (SSA) such as establishing an annual budget process for outlining sector specific critical infrastructure and key resources (CIKR) protection requirements and related budget projections, to the extent possible, as a component of SSA annual budget submissions. According to the 2009 NIPP, maximizing the efficient use of resources for CIKR protection includes a coordinated and integrated annual process for program implementation that informs the annual Federal process regarding planning, programming, and budgeting for national-level CIKR protection. To further demonstrate that SSAs are held accountable for integrating risk management and agency budget review, the National CIKR Protection Annual Report is submitted along with the DHS budget submission to the Executive Office of the President on or before September 1 as part of the annual Federal budget process. The SSAs submit the CIKR protection priorities and requirements to DHS in their sector annual reports. The SSAs work within their respective department or agency budget process to determine the CIKR protection-related aspects of their department's budget submission. Furthermore, the 2009 NIPP states the use of performance metrics is a critical step in the NIPP risk management process to enable DHS and its partners to objectively and quantitatively assess improvements in CIKR protection and resiliency at the sector and national levels. To this end DHS issued NIPP Metrics Guidance in February 2009 to assist sectors in the development of performance measures to measure sector progress in CIKR protection. The guidance stressed the importance of metrics development to the budget process when it stated that NIPP metrics inform current and prospective allocation of resources in light of previously implemented protective actions or other factors. The DHS guidance states DHS will work with each sector to develop metrics that focus on outcomes the sector are trying to achieve, understanding that this is an ongoing process. The 2009 National CIKR Protection Annual Report identifies hundreds of risk mitigation activities (RMA)or programs designed to reduce the risk to CIKR sectors. A review of selected 2009 sector annual reports indicates that the reports contain risk mitigation activities that include progress indicators that may contain outcome metrics that provide an indicator of changes to the level of risk in a specific sector due to the implementation of the RMAs contained in the reports.

In fiscal year 2006, we reported, among other things, Information Analysis and Infrastructure Protection's (IAIP) (now National Protection and Programs Directorate) risk management efforts were focused mainly on assessing and reducing vulnerabilities which had the potential of limiting DHS's ability to achieve the broader goal of using risk-based data as a tool to inform management decisions. In 2007, the Secretary for the DHS issued Delegation Number 17001 which delegated authority to the Under Secretary for the National Protection and Programs Directorate (NPPD) for managing risk to the nation's critical infrastructure and key resources (CIKR). The Under Secretary, in collaboration with the Office of Risk Management and Analysis (RMA), implemented a Department-wide Risk Steering Committee to serve as DHS's risk management governance structure to further hold DHS accountable for risk management activities, thereby enabling the sharing and integration of risk management efforts. The RMA and Risk Steering Committees were both implemented in April 2007. The Risk Steering Committee is a cooperative body formed to ensure that risk management is carried out consistently and comparably throughout the Department. Chaired by the Under Secretary for National Protection and Programs and comprising component heads and other identified personnel, the committee assists in the framing of processes and procedures for the Department's risk-management architecture, enabling collaboration and Department-wide agreement on risk-management efforts. According to the Delegation directive the Under Secretary also has the authority to coordinate with other DHS components to synchronize risk management programs.

In fiscal year 2006, we analyzed how multiple Department of Homeland Security (DHS) components--including the Information Analysis and Infrastructure Protection (IAIP) Directorate (now known as the National Programs and Protection Directorate) were basing their homeland security efforts on risk management: a systematic process for assessing threats and taking appropriate steps to deal with them. We reported, among other things, that beyond DHS, integrating risk with existing systems for budget and program review is complicated by the fact that IAIP and DHS must depend on others to follow risk management principles for programs and budgets at the other six major Departments or agencies charged with assessing risks under Homeland Security Presidential Directive-7 (HSPD-7), which creates uncertainty in implementing risk management across federal agencies in a way that informs program and budget review processes. In June 2006, the Office of Management and Budget (OMB) issued a proposed Risk Assessment Bulletin to enhance the technical quality and objectivity of risk assessments performed by federal agencies. In January 2009, an updated National Infrastructure Protection Plan (NIPP) was issued by NPPD. The NIPP states in several places that CIKR protection plans and programs for DHS and agencies acting as sector specific agencies (SSAs) should be based on the use of a risk management framework and integrated into the applicable federal department's budget submission. The NIPP states that the federal budget process directly affects the sector specific agencies other than DHS that have responsibility for CIKR protection. To further hold SSA's accountable for integrating risk management within the budget process, the NIPP states the SSA's established annual budget process is the primary mechanism for outlining sector-specific CIKR protection requirements and related budget projections, and should be included as a component of their annual budget submissions to the Office of Management and Budget (OMB). According to the 2009 NIPP, maximizing the efficient use of resources for CIKR protection includes a coordinated and integrated annual process for program implementation that informs the annual Federal process regarding planning, programming, and budgeting for national-level CIKR protection. It also considers State, local, tribal, and territorial government and private sector issues related to planning, programming, and budgeting. The NIPP states that DHS's CIKR planning helps inform the annual federal budget process based on CIKR risk and the potential for reducing risk and need, in coordination with SSAs, Government Coordinating Council, and other partners. To further demonstrate that SSAs are held accountable for integrating risk management and agency budget review, the National CIKR Protection Annual Report is submitted along with the DHS budget submission to the Executive Office of the President on or before Sept. 1 as part of the annual budget process. The SSAs submit CIKR protection priorities and requirements to DHS in their sector annual reports. The SSAs work within their respective department or agency budget process to determine CIKR protection-related aspects of their department's budget submission. SSA annual reports are submitted to DHS on or before June 1. Resource information contained in the SSA annual reports is based on appropriated funding, as well as the President's most recent budget. According to the NIPP, the risk management initiatives that result in the greatest risk mitigation for the investment proposed are accorded the highest priority in program design, resource allocation, budgeting, and implementation, a consideration that applies to agency research and development (R&D) investment as well as CIKR programs. According to the 2009 NIPP, the NIPP R&D strategic goals are used to guide federal R&D investment decisions and also to provide a coordinated approach to the overall Federal research program.