Critical Infrastructure Protection:

Sector-Specific Agencies Need to Better Measure Cybersecurity Progress

GAO-16-79: Published: Nov 19, 2015. Publicly Released: Nov 19, 2015.

Multimedia:

Additional Materials:

Contact:

Gregory C. Wilshusen
(202) 512-6244
wilshuseng@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Sector-specific agencies (SSA) determined the significance of cyber risk to networks and industrial control systems for all 15 of the sectors in the scope of GAO's review. Specifically, they determined that cyber risk was significant for 11 of 15 sectors. Although the SSAs for the remaining four sectors had not determined cyber risks to be significant during their 2010 sector-specific planning process, they subsequently reconsidered the significance of cyber risks to the sector. For example, commercial facilities sector–specific agency officials stated that they recognized cyber risk as a high-priority concern for the sector as part of the updated sector planning process. SSAs and their sector partners are to include an overview of current and emerging cyber risks in their updated sector-specific plans for 2015.

SSAs generally took actions to mitigate cyber risks and vulnerabilities for their respective sectors. SSAs developed, implemented, or supported efforts to enhance cybersecurity and mitigate cyber risk with activities that aligned with a majority of actions called for by the National Infrastructure Protection Plan (NIPP). SSAs for 12 of the 15 sectors had not identified incentives to promote cybersecurity in their sectors as proposed in the NIPP; however, the SSAs are participating in a working group to identify appropriate incentives. In addition, SSAs for 3 of 15 sectors had not yet made significant progress in advancing cyber-based research and development within their sectors because it had not been an area of focus for their sector. Department of Homeland Security guidance for updating the sector-specific plans directs the SSAs to incorporate the NIPP's actions to guide their cyber risk mitigation activities, including cybersecurity-related actions to identify incentives and promote research and development.

All SSAs that GAO reviewed used multiple public-private and cross-sector collaboration mechanisms to facilitate the sharing of cybersecurity-related information. For example, the SSAs used councils of federal and nonfederal stakeholders, including coordinating councils and cybersecurity and industrial control system working groups, to coordinate with each other. In addition, SSAs participated in the National Cybersecurity and Communications Integration Center, a national center at the Department of Homeland Security, to receive and disseminate cyber-related information for public and private sector partners.

The Departments of Defense, Energy, and Health and Human Services established performance metrics for their three sectors. However, the SSAs for the other 12 sectors had not developed metrics to measure and report on the effectiveness of all of their cyber risk mitigation activities or their sectors' cybersecurity posture. This was because, among other reasons, the SSAs rely on their private sector partners to voluntarily share information needed to measure efforts. The NIPP directs SSAs and their sector partners to identify high-level outcomes to facilitate progress towards national goals and priorities. Until SSAs develop performance metrics and collect data to report on the progress of their efforts to enhance the sectors' cybersecurity posture, they may be unable to adequately monitor the effectiveness of their cyber risk mitigation activities and document the resulting sector-wide cybersecurity progress.

Why GAO Did This Study

U. S. critical infrastructures, such as financial institutions, commercial buildings, and energy production and transmission facilities, are systems and assets, whether physical or virtual, vital to the nation's security, economy, and public health and safety. To secure these systems and assets, federal policy and the NIPP establish responsibilities for federal agencies designated as SSAs, including leading, facilitating, or supporting the security and resilience programs and associated activities of their designated critical infrastructure sectors.

GAO's objectives were to determine the extent to which SSAs have (1) identified the significance of cyber risks to their respective sectors' networks and industrial control systems, (2) taken actions to mitigate cyber risks within their respective sectors, (3) collaborated across sectors to improve cybersecurity, and (4) established performance metrics to monitor improvements in their respective sectors. To conduct the review, GAO analyzed policy, plans, and other documentation and interviewed public and private sector officials for 8 of 9 SSAs with responsibility for 15 of 16 sectors.

What GAO Recommends

GAO recommends that certain SSAs collaborate with sector partners to develop performance metrics and determine how to overcome challenges to reporting the results of their cyber risk mitigation activities. Four of these agencies concurred with GAO's recommendation, while two agencies did not comment on the recommendations.

For more information, contact Gregory C. Wilshusen at (202) 512-6244 or wilshuseng@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: DHS has released updated sector-specific plans for the chemical, commercial facilities, communications, critical manufacturing, dams, emergency services, information technology, and nuclear reactors sectors. The plans include a section on measuring effectiveness based on the plan development guidance. The plans provide expected metrics to track the progress of sector activities and state that the outcomes will be reported through the National Annual Reporting process as well as through the quadrennial plan update. Because the metrics are new and annual reporting has not yet occurred, DHS has not provided evidence of metrics data collected and reported to address the challenges.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretary of Homeland Security should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the chemical, commercial facilities, communications, critical manufacturing, dams, emergency services, information technology, and nuclear sectors' cybersecurity progress.

    Agency Affected: Department of Homeland Security

  2. Status: Open

    Comments: The 2015 sector-specific plan for the financial services sector includes a section on measuring the effectiveness of sector activities; however, the plan does not include specific metrics. The plan refers to working groups and meetings of sector stakeholders as mechanisms to track sector progress. No specific metrics and associated reports of outcomes have been provided to address overcoming the challenges of monitoring the sector's cybersecurity progress.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretary of the Treasury should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the financial services sector's cybersecurity progress.

    Agency Affected: Department of the Treasury

  3. Status: Open

    Comments: The Departments of Agriculture and Health and Human Services released an update to the food and agriculture sector-specific plan for 2015. The plan states the sector's lack of an overarching mechanism to measure and evaluate risk mitigation activities and the challenge of obtaining performance measurement data from non-federal partners. However, the plan notes a goal of evaluating the progress of individual protective programs and strategies. No metrics or reports of outcomes have been provided to address the challenge of monitoring the sector's cybersecurity progress.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Agriculture and Health and Human Services (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the food and agriculture sector's cybersecurity progress.

    Agency Affected: Department of Agriculture

  4. Status: Open

    Comments: The Departments of Agriculture and Health and Human Services released an update to the food and agriculture sector-specific plan for 2015. The plan states the sector's lack of an overarching mechanism to measure and evaluate risk mitigation activities and the challenge of obtaining performance measurement data from non-federal partners. However, the plan notes a goal of evaluating the progress of individual protective programs and strategies. No metrics or reports of outcomes have been provided to address the challenge of monitoring the sector's cybersecurity progress.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Agriculture and Health and Human Services (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the food and agriculture sector's cybersecurity progress.

    Agency Affected: Department of Health and Human Services

  5. Status: Open

    Comments: As co-Sector-Specific Agencies (SSAs) for the Transportation Systems Sector, DHS (TSA and Coast Guard) and the Department of Transportation released the 2015 update to the sector-specific plan. The plan includes a section on measuring effectiveness that identifies qualitative and quantitative approaches to measuring the effectiveness of sector activities. According to SSA officials, they are developing a plan to collaborate with transportation industry owners and operators to collect cybersecurity performance metrics reflecting the voluntary adoption of the NIST Cybersecurity Framework. Their estimated completion date for this effort is December 31, 2016. No metrics or reports of outcomes have been provided to address the challenge of monitoring the sector's cybersecurity progress.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Homeland Security and Transportation (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the transportation systems sector's cybersecurity progress.

    Agency Affected: Department of Homeland Security

  6. Status: Open

    Comments: As co-Sector-Specific Agencies (SSAs) for the Transportation Systems Sector, DHS (TSA and Coast Guard) and the Department of Transportation released the 2015 update to the sector-specific plan. The plan includes a section on measuring effectiveness that identifies qualitative and quantitative approaches to measuring the effectiveness of sector activities. According to SSA officials, they are developing a plan to collaborate with transportation industry owners and operators to collect cybersecurity performance metrics reflecting the voluntary adoption of the NIST Cybersecurity Framework. Their estimated completion date for this effort is December 31, 2016. No metrics or reports of outcomes have been provided to address the challenge of monitoring the sector's cybersecurity progress.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Homeland Security and Transportation (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the transportation systems sector's cybersecurity progress.

    Agency Affected: Department of Transportation

  7. Status: Open

    Comments: The 2015 water and wastewater sector-specific plan includes a segment on measuring the effectiveness of sector activities that describes the overall principles for collecting data and using the National Annual Report data calls as a tool for assessing performance and reporting on progress within the sector. However, the plan does not state specific measures and the agency acknowledged in its response to our report that it does not collect performance metrics on the effectiveness of its cybersecurity programs for the sector. According to agency officials, the development of performance metrics in collaboration with sector partners is underway.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Administrator of the Environmental Protection Agency should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the water and wastewater systems sector's cybersecurity progress.

    Agency Affected: Environmental Protection Agency

 

Explore the full database of GAO's Open Recommendations »

Dec 6, 2016

Nov 30, 2016

Nov 16, 2016

Sep 29, 2016

Sep 26, 2016

Sep 15, 2016

Sep 14, 2016

Sep 8, 2016

Jun 29, 2016

Jun 22, 2016

Looking for more? Browse all our products here