Critical Infrastructure Protection:

Sector-Specific Agencies Need to Better Measure Cybersecurity Progress

GAO-16-79: Published: Nov 19, 2015. Publicly Released: Nov 19, 2015.

Multimedia:

Additional Materials:

Contact:

Gregory C. Wilshusen
(202) 512-6244
wilshuseng@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

Sector-specific agencies (SSA) determined the significance of cyber risk to networks and industrial control systems for all 15 of the sectors in the scope of GAO's review. Specifically, they determined that cyber risk was significant for 11 of 15 sectors. Although the SSAs for the remaining four sectors had not determined cyber risks to be significant during their 2010 sector-specific planning process, they subsequently reconsidered the significance of cyber risks to the sector. For example, commercial facilities sector–specific agency officials stated that they recognized cyber risk as a high-priority concern for the sector as part of the updated sector planning process. SSAs and their sector partners are to include an overview of current and emerging cyber risks in their updated sector-specific plans for 2015.

SSAs generally took actions to mitigate cyber risks and vulnerabilities for their respective sectors. SSAs developed, implemented, or supported efforts to enhance cybersecurity and mitigate cyber risk with activities that aligned with a majority of actions called for by the National Infrastructure Protection Plan (NIPP). SSAs for 12 of the 15 sectors had not identified incentives to promote cybersecurity in their sectors as proposed in the NIPP; however, the SSAs are participating in a working group to identify appropriate incentives. In addition, SSAs for 3 of 15 sectors had not yet made significant progress in advancing cyber-based research and development within their sectors because it had not been an area of focus for their sector. Department of Homeland Security guidance for updating the sector-specific plans directs the SSAs to incorporate the NIPP's actions to guide their cyber risk mitigation activities, including cybersecurity-related actions to identify incentives and promote research and development.

All SSAs that GAO reviewed used multiple public-private and cross-sector collaboration mechanisms to facilitate the sharing of cybersecurity-related information. For example, the SSAs used councils of federal and nonfederal stakeholders, including coordinating councils and cybersecurity and industrial control system working groups, to coordinate with each other. In addition, SSAs participated in the National Cybersecurity and Communications Integration Center, a national center at the Department of Homeland Security, to receive and disseminate cyber-related information for public and private sector partners.

The Departments of Defense, Energy, and Health and Human Services established performance metrics for their three sectors. However, the SSAs for the other 12 sectors had not developed metrics to measure and report on the effectiveness of all of their cyber risk mitigation activities or their sectors' cybersecurity posture. This was because, among other reasons, the SSAs rely on their private sector partners to voluntarily share information needed to measure efforts. The NIPP directs SSAs and their sector partners to identify high-level outcomes to facilitate progress towards national goals and priorities. Until SSAs develop performance metrics and collect data to report on the progress of their efforts to enhance the sectors' cybersecurity posture, they may be unable to adequately monitor the effectiveness of their cyber risk mitigation activities and document the resulting sector-wide cybersecurity progress.

Why GAO Did This Study

U. S. critical infrastructures, such as financial institutions, commercial buildings, and energy production and transmission facilities, are systems and assets, whether physical or virtual, vital to the nation's security, economy, and public health and safety. To secure these systems and assets, federal policy and the NIPP establish responsibilities for federal agencies designated as SSAs, including leading, facilitating, or supporting the security and resilience programs and associated activities of their designated critical infrastructure sectors.

GAO's objectives were to determine the extent to which SSAs have (1) identified the significance of cyber risks to their respective sectors' networks and industrial control systems, (2) taken actions to mitigate cyber risks within their respective sectors, (3) collaborated across sectors to improve cybersecurity, and (4) established performance metrics to monitor improvements in their respective sectors. To conduct the review, GAO analyzed policy, plans, and other documentation and interviewed public and private sector officials for 8 of 9 SSAs with responsibility for 15 of 16 sectors.

What GAO Recommends

GAO recommends that certain SSAs collaborate with sector partners to develop performance metrics and determine how to overcome challenges to reporting the results of their cyber risk mitigation activities. Four of these agencies concurred with GAO's recommendation, while two agencies did not comment on the recommendations.

For more information, contact Gregory C. Wilshusen at (202) 512-6244 or wilshuseng@gao.gov.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: The Department of Homeland Security (DHS)'s Cybersecurity and Infrastructure Security Agency (CISA), as the sector-specific agency for the chemical, commercial facilities, communications, critical manufacturing, dams, emergency services, information technology, and nuclear reactors sectors, has implemented measurement approaches to capture the results of specific security-related activities, which meet the intent of the recommendation. For example, CISA's Cybersecurity Advisor (CSA) Program issues a post-assessment questionnaire to individual stakeholders that participate in CSA-led cybersecurity assessments. CISA compiles survey results quarterly, identifying which organizations have planned, scheduled, or implemented options for consideration as a result of the CSA-led assessment. CISA collects data via the questionnaire in order to guide process improvements and communicate the effectiveness of the program's effectiveness which meets the intent of the recommendation.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretary of Homeland Security should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the chemical, commercial facilities, communications, critical manufacturing, dams, emergency services, information technology, and nuclear sectors' cybersecurity progress.

    Agency Affected: Department of Homeland Security

  2. Status: Open

    Comments: The Department of the Treasury, as the sector-specific agency for the financial services sector, continues to develop initiatives intended to enhance the sector's cybersecurity. In 2016, Treasury developed and promulgated a set of seven fundamental elements or critical building blocks for sector stakeholders' cybersecurity, disseminated a template for financial sector cyber exercises, and promoted the NIST Cybersecurity Framework throughout the sector. However, they have not provided evidence of metrics implemented, and the 2015 sector-specific plan does not include specific metrics to track and report on their effectiveness. We will continue to monitor Treasury's efforts to create specific metrics and related reports on the sector's cybersecurity progress.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretary of the Treasury should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the financial services sector's cybersecurity progress.

    Agency Affected: Department of the Treasury

  3. Status: Open

    Comments: The Department of Agriculture (USDA), as the co-sector specific agency for the food and agriculture sector, with the Department of Health and Human Services (HHS) continues to implement cybersecurity-related activities for the sector. In particular, USDA, through the sector coordination council, routinely shares best practices and informational bulletins from the Department of Homeland Security on cybersecurity with sector stakeholders via the Homeland Security Information Network. In addition, at semi-annual council meetings, USDA has hosted roundtable discussions of cybersecurity challenges and best practices. No evidence of performance metrics to track and report on the SSAs' activities or the sector's cybersecurity progress has been provided. As USDA and HHS continue to carry out their sector-specific agency role, we will continue to monitor their efforts and associated performance metrics to be developed to demonstrate the effectiveness of these activities

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Agriculture and Health and Human Services (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the food and agriculture sector's cybersecurity progress.

    Agency Affected: Department of Agriculture

  4. Status: Open

    Comments: The Department of Health and Human Services (HHS), as the co-sector specific agency for the food and agriculture sector, with the Department of Agriculture (USDA) continues to implement cybersecurity-related activities for the sector. In particular, through the sector coordination council, they routinely share best practices and informational bulletins from the Department of Homeland Security on cybersecurity with sector stakeholders via the Homeland Security Information Network. In addition, at semi-annual council meetings, they have hosted roundtable discussions of cybersecurity challenges and best practices. No evidence of performance metrics to track and report on the SSAs' activities or the sector's cybersecurity progress has been provided. As HHS and USDA continue to carry out their sector-specific agency role, we will continue to monitor their efforts and associated performance metrics to be developed to demonstrate the effectiveness of these activities

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Agriculture and Health and Human Services (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the food and agriculture sector's cybersecurity progress.

    Agency Affected: Department of Health and Human Services

  5. Status: Closed - Implemented

    Comments: DHS (Transportation Security Administration and Coast Guard) and the Department of Transportation, as the co-Sector-Specific Agencies (SSAs) for the transportation systems sector, implemented measurement approaches to capture the results of specific security-related activities, which meets the intent of the recommendation. For example, in 2017, participants in a federal exercise program focused on security in the nation's transportation sector were surveyed to measure the change in their level of knowledge of five nontechnical cybersecurity actions: familiarity with the National Institute of Standards and Technology's Cybersecurity Framework; unique password change policy, latest phishing and spam trends; role-based access controls, and cybersecurity incident reporting. The participants were also surveyed to measure the likelihood that they would implement the subject cybersecurity actions. The outcomes from the responses were reported via bar charts showing the percentage change in the participants' pre- and post-knowledge and the likelihood of implementation. Although the measures do not indicate how they capture outcomes across the entire transportation systems sector and do not relate to any other cybersecurity-related activities the SSAs have instituted, they do give insight into the effectiveness of the training and exercise program based on participant feedback, which meets the intent of the recommendation.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Homeland Security and Transportation (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the transportation systems sector's cybersecurity progress.

    Agency Affected: Department of Homeland Security

  6. Status: Closed - Implemented

    Comments: DHS (Transportation Security Administration and Coast Guard) and the Department of Transportation, as the co-Sector-Specific Agencies (SSAs) for the transportation systems sector, implemented measurement approaches to capture the results of specific security-related activities, which meets the intent of the recommendation. For example, in 2017, participants in a federal exercise program focused on security in the nation's transportation sector were surveyed to measure the change in their level of knowledge of five nontechnical cybersecurity actions: familiarity with the National Institute of Standards and Technology's Cybersecurity Framework; unique password change policy, latest phishing and spam trends; role-based access controls, and cybersecurity incident reporting. The participants were also surveyed to measure the likelihood that they would implement the subject cybersecurity actions. The outcomes from the responses were reported via bar charts showing the percentage change in the participants' pre- and post-knowledge and the likelihood of implementation. Although the measures do not indicate how they capture outcomes across the entire transportation systems sector and do not relate to any other cybersecurity-related activities the SSAs have instituted, they do give insight into the effectiveness of the training and exercise program based on participant feedback.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Homeland Security and Transportation (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the transportation systems sector's cybersecurity progress.

    Agency Affected: Department of Transportation

  7. Status: Open

    Comments: The Environmental Protection Agency (EPA) continues to develop and implement activities in support of the water and wastewater sector's cybersecurity such as a cyber-attack risk assessment tool and cybersecurity training for sector partners. The 2015 water and wastewater sector-specific plan calls for assessing performance and reporting on sector cybersecurity progress; however, the plan does not state specific measures. In 2017, agency officials stated that the development of performance metrics in collaboration with sector partners was underway; however, EPA has not provided evidence of the metrics or any tracking effort. As EPA continues to carry out its sector-specific agency role, we will continue to monitor its efforts and associated performance metrics to be developed to demonstrate the effectiveness of these activities.

    Recommendation: To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Administrator of the Environmental Protection Agency should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the water and wastewater systems sector's cybersecurity progress.

    Agency Affected: Environmental Protection Agency

 

Explore the full database of GAO's Open Recommendations »

Jun 1, 2020

Mar 5, 2020

Feb 20, 2020

  • it icon, source: PhotoDisc

    Science & Tech Spotlight:

    Deepfakes
    GAO-20-379SP: Published: Feb 20, 2020. Publicly Released: Feb 20, 2020.

Dec 12, 2019

Dec 11, 2019

Oct 30, 2019

Sep 10, 2019

Jun 26, 2019

Jun 11, 2019

Looking for more? Browse all our products here