Skip to main content

Critical Infrastructure Protection: Sector-Specific Agencies Need to Better Measure Cybersecurity Progress

GAO-16-79 Published: Nov 19, 2015. Publicly Released: Nov 19, 2015.
Jump To:
Skip to Highlights

Highlights

What GAO Found

Sector-specific agencies (SSA) determined the significance of cyber risk to networks and industrial control systems for all 15 of the sectors in the scope of GAO's review. Specifically, they determined that cyber risk was significant for 11 of 15 sectors. Although the SSAs for the remaining four sectors had not determined cyber risks to be significant during their 2010 sector-specific planning process, they subsequently reconsidered the significance of cyber risks to the sector. For example, commercial facilities sector–specific agency officials stated that they recognized cyber risk as a high-priority concern for the sector as part of the updated sector planning process. SSAs and their sector partners are to include an overview of current and emerging cyber risks in their updated sector-specific plans for 2015.

SSAs generally took actions to mitigate cyber risks and vulnerabilities for their respective sectors. SSAs developed, implemented, or supported efforts to enhance cybersecurity and mitigate cyber risk with activities that aligned with a majority of actions called for by the National Infrastructure Protection Plan (NIPP). SSAs for 12 of the 15 sectors had not identified incentives to promote cybersecurity in their sectors as proposed in the NIPP; however, the SSAs are participating in a working group to identify appropriate incentives. In addition, SSAs for 3 of 15 sectors had not yet made significant progress in advancing cyber-based research and development within their sectors because it had not been an area of focus for their sector. Department of Homeland Security guidance for updating the sector-specific plans directs the SSAs to incorporate the NIPP's actions to guide their cyber risk mitigation activities, including cybersecurity-related actions to identify incentives and promote research and development.

All SSAs that GAO reviewed used multiple public-private and cross-sector collaboration mechanisms to facilitate the sharing of cybersecurity-related information. For example, the SSAs used councils of federal and nonfederal stakeholders, including coordinating councils and cybersecurity and industrial control system working groups, to coordinate with each other. In addition, SSAs participated in the National Cybersecurity and Communications Integration Center, a national center at the Department of Homeland Security, to receive and disseminate cyber-related information for public and private sector partners.

The Departments of Defense, Energy, and Health and Human Services established performance metrics for their three sectors. However, the SSAs for the other 12 sectors had not developed metrics to measure and report on the effectiveness of all of their cyber risk mitigation activities or their sectors' cybersecurity posture. This was because, among other reasons, the SSAs rely on their private sector partners to voluntarily share information needed to measure efforts. The NIPP directs SSAs and their sector partners to identify high-level outcomes to facilitate progress towards national goals and priorities. Until SSAs develop performance metrics and collect data to report on the progress of their efforts to enhance the sectors' cybersecurity posture, they may be unable to adequately monitor the effectiveness of their cyber risk mitigation activities and document the resulting sector-wide cybersecurity progress.

Why GAO Did This Study

U. S. critical infrastructures, such as financial institutions, commercial buildings, and energy production and transmission facilities, are systems and assets, whether physical or virtual, vital to the nation's security, economy, and public health and safety. To secure these systems and assets, federal policy and the NIPP establish responsibilities for federal agencies designated as SSAs, including leading, facilitating, or supporting the security and resilience programs and associated activities of their designated critical infrastructure sectors.

GAO's objectives were to determine the extent to which SSAs have (1) identified the significance of cyber risks to their respective sectors' networks and industrial control systems, (2) taken actions to mitigate cyber risks within their respective sectors, (3) collaborated across sectors to improve cybersecurity, and (4) established performance metrics to monitor improvements in their respective sectors. To conduct the review, GAO analyzed policy, plans, and other documentation and interviewed public and private sector officials for 8 of 9 SSAs with responsibility for 15 of 16 sectors.

Recommendations

GAO recommends that certain SSAs collaborate with sector partners to develop performance metrics and determine how to overcome challenges to reporting the results of their cyber risk mitigation activities. Four of these agencies concurred with GAO's recommendation, while two agencies did not comment on the recommendations.

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Homeland Security To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretary of Homeland Security should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the chemical, commercial facilities, communications, critical manufacturing, dams, emergency services, information technology, and nuclear sectors' cybersecurity progress.
Closed – Implemented
The Department of Homeland Security (DHS)'s Cybersecurity and Infrastructure Security Agency (CISA), as the sector-specific agency for the chemical, commercial facilities, communications, critical manufacturing, dams, emergency services, information technology, and nuclear reactors sectors, has implemented measurement approaches to capture the results of specific security-related activities, which meet the intent of the recommendation. For example, CISA's Cybersecurity Advisor (CSA) Program issues a post-assessment questionnaire to individual stakeholders that participate in CSA-led cybersecurity assessments. CISA compiles survey results quarterly, identifying which organizations have planned, scheduled, or implemented options for consideration as a result of the CSA-led assessment. CISA collects data via the questionnaire in order to guide process improvements and communicate the effectiveness of the program's effectiveness which meets the intent of the recommendation.
Department of the Treasury To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretary of the Treasury should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the financial services sector's cybersecurity progress.
Closed – Not Implemented
The Department of the Treasury, as the sector-specific agency for the financial services sector, continues to take steps to reduce risks and bolster the sector's efforts to improve its cybersecurity. However, in September 2020, we reported that Treasury had not fully implemented our recommendation to establish metrics related to the financial services sector's cybersecurity progress (see GAO-20-631). In that report, we expanded on our original recommendation with a new recommendation that Treasury, in coordination with the other federal and nonfederal sector partners, update the financial services sector-specific plan to include specific metrics for measuring the progress of risk mitigation efforts. We are closing the earlier recommendation from GAO-16-79 because the recommendation in GAO-20-631 supersedes it, and calls for the agency to take more definite action to measure the sector's progress in mitigating cybersecurity risks. We will continue to monitor Treasury's progress in addressing the newer recommendation.
Department of Agriculture To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Agriculture and Health and Human Services (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the food and agriculture sector's cybersecurity progress.
Closed – Not Implemented
The Department of Agriculture (USDA), as the co-sector specific agency for the food and agriculture sector with the Department of Health and Human Services (HHS), has not developed performance metrics to monitor the food and agriculture sector's cybersecurity progress. According to USDA and HHS officials, the co-sector-specific agencies continue to implement cybersecurity-related activities to help sector partners mitigate against and respond to cyber events including sharing tools, resources, and information. However, USDA and HHS officials explained that they have no plans to develop performance metrics to track the sector's cybersecurity progress because of the voluntary nature of the relationship with their non-federal sector partners, the lack of required feedback from them, and the minimal feedback available.
Department of Health and Human Services To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Agriculture and Health and Human Services (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the food and agriculture sector's cybersecurity progress.
Closed – Not Implemented
The Department of Health and Human Services (HHS), as the co-sector specific agency for the food and agriculture sector with the Department of Agriculture (USDA), has not developed performance metrics to monitor the food and agriculture sector's cybersecurity progress. According to HHS and USDA officials, the co-sector-specific agencies continue to implement cybersecurity-related activities to help sector partners mitigate against and respond to cyber events including sharing tools, resources, and information. However, HHS and USDA officials explained that they have no plans to develop performance metrics to track the sector's cybersecurity progress because of the voluntary nature of the relationship with their non-federal sector partners, the lack of required feedback from them, and the minimal feedback available.
Department of Homeland Security To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Homeland Security and Transportation (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the transportation systems sector's cybersecurity progress.
Closed – Implemented
DHS (Transportation Security Administration and Coast Guard) and the Department of Transportation, as the co-Sector-Specific Agencies (SSAs) for the transportation systems sector, implemented measurement approaches to capture the results of specific security-related activities, which meets the intent of the recommendation. For example, in 2017, participants in a federal exercise program focused on security in the nation's transportation sector were surveyed to measure the change in their level of knowledge of five nontechnical cybersecurity actions: familiarity with the National Institute of Standards and Technology's Cybersecurity Framework; unique password change policy, latest phishing and spam trends; role-based access controls, and cybersecurity incident reporting. The participants were also surveyed to measure the likelihood that they would implement the subject cybersecurity actions. The outcomes from the responses were reported via bar charts showing the percentage change in the participants' pre- and post-knowledge and the likelihood of implementation. Although the measures do not indicate how they capture outcomes across the entire transportation systems sector and do not relate to any other cybersecurity-related activities the SSAs have instituted, they do give insight into the effectiveness of the training and exercise program based on participant feedback, which meets the intent of the recommendation.
Department of Transportation To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Secretaries of Homeland Security and Transportation (as co-SSAs) should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the transportation systems sector's cybersecurity progress.
Closed – Implemented
DHS (Transportation Security Administration and Coast Guard) and the Department of Transportation, as the co-Sector-Specific Agencies (SSAs) for the transportation systems sector, implemented measurement approaches to capture the results of specific security-related activities, which meets the intent of the recommendation. For example, in 2017, participants in a federal exercise program focused on security in the nation's transportation sector were surveyed to measure the change in their level of knowledge of five nontechnical cybersecurity actions: familiarity with the National Institute of Standards and Technology's Cybersecurity Framework; unique password change policy, latest phishing and spam trends; role-based access controls, and cybersecurity incident reporting. The participants were also surveyed to measure the likelihood that they would implement the subject cybersecurity actions. The outcomes from the responses were reported via bar charts showing the percentage change in the participants' pre- and post-knowledge and the likelihood of implementation. Although the measures do not indicate how they capture outcomes across the entire transportation systems sector and do not relate to any other cybersecurity-related activities the SSAs have instituted, they do give insight into the effectiveness of the training and exercise program based on participant feedback.
Environmental Protection Agency To better monitor and provide a basis for improving the effectiveness of cybersecurity risk mitigation activities, informed by the sectors' updated plans and in collaboration with sector stakeholders, the Administrator of the Environmental Protection Agency should direct responsible officials to develop performance metrics to provide data and determine how to overcome challenges to monitoring the water and wastewater systems sector's cybersecurity progress.
Open
The Environmental Protection Agency (EPA) continues to develop and implement activities in support of the water and wastewater sector's cybersecurity such as a cyber-attack risk assessment tool and cybersecurity training for sector partners. The 2015 water and wastewater sector-specific plan calls for assessing performance and reporting on sector cybersecurity progress; however, the plan does not state specific measures. EPA officials recognize the challenge of developing consensus-based performance metrics for the sector. In June 2022, officials stated that they had confirmed the water sector's continued opposition to developing and collecting metrics data. In addition, officials stated that EPA had not been able to use results from self-assessment products or other cybersecurity tools to collect and report cybersecurity improvements from the water sector's use of the NIST Framework. At present, water sector facilities are not required to report this information and water utility owners and operators have expressed a reluctance to disclose this information voluntarily. Officials stated that EPA expects to gain further information from the regulatory approach under development. While its efforts are important, EPA officials did not provide evidence of specific sector-related performance metrics.

Full Report

GAO Contacts

Office of Public Affairs

Topics

Computer securityCritical infrastructureCritical infrastructure protectionCyber securityDefense industryGovernment information disseminationInternal controlsRisk assessmentRisk managementCybersecurity