Disaster Risk: Improvements Needed to Enhance FEMA’s National Risk Index
Fast Facts
Natural disasters are more frequent and costlier. The National Risk Index is a tool communities can use to know their disaster risk level, which can help inform their resilience planning. The Federal Emergency Management Agency oversees the tool.
While FEMA followed some risk assessment leading practices related to the tool, it hasn't fully implemented others. For example, the tool doesn't help users, like emergency managers, apply its risk information. FEMA also doesn’t collect feedback from users to know if the tool is useful or valuable for their resilience planning.
Our recommendations address these issues and more.
Cars in a flooded road with storm clouds in the background and a tornado.
Highlights
What GAO Found
GAO found that the Federal Emergency Management Agency (FEMA) —a component within the Department of Homeland Security (DHS)— partially followed leading practices for risk assessment and information quality during the development, operation, and maintenance of the National Risk Index (NRI). The figure below lists these practices.
FEMA took steps to define and disseminate information about the purpose and scope of the NRI and provided additional information on the methodologies and data used to inform the index. However, FEMA did not consider what information users (e.g., emergency managers) would need to tangibly apply NRI results when making decisions about how to reduce risk. Moreover, FEMA did not explain how users should apply NRI information in conjunction with other FEMA risk tools to generate effective risk insights. Lack of information about how to apply results undermines the NRI’s perceived utility.
FEMA took steps to use diverse sources to design the NRI, including consulting relevant sources and subject matter experts. FEMA also followed an established process to verify and validate hazard calculations and risk scores. However, FEMA did not consistently use sensitivity analyses to verify the model’s outputs or inform significant changes to data sources and methodologies. Conducting such analyses would enhance understanding of factors that could affect NRI results, such as methodological choices that could lead to over- or under-estimation of specific aspects of hazard risk. Disclosing results of any such analyses conducted would also enhance user understanding of how those factors could affect risk scoring in their specific circumstances.
FEMA established a process to communicate and engage with subject matter experts on future updates to NRI data. However, FEMA does not have an established mechanism to systematically collect ongoing user feedback and use it for continuous NRI improvement. Without such a mechanism, FEMA does not have the information it needs to assess whether the tool provides valuable information to its users to address their hazard mitigation planning needs.
Why GAO Did This Study
Natural disasters have become more costly and frequent. Jurisdictions can use the NRI to help address disaster risk. It is the only national-level index that combines disaster loss with social vulnerability and community resilience measures to score risk, according to FEMA officials.
GAO was asked to review the extent to which FEMA has applied leading practices to the NRI. This report examines leading practices for (1) defining the purpose of risk information; (2) designing a sound model and ensuring information quality; and (3) engaging with relevant stakeholders.
GAO consulted federal sources and international standards to develop leading practices for assessing risk. GAO then analyzed FEMA’s actions against these practices. GAO analyzed technical documentation and interviewed FEMA officials to assess practices. GAO also interviewed emergency management stakeholders from select states for user perspectives.
Recommendations
GAO is making three recommendations, including that FEMA provide information to users on how to apply NRI results effectively and systematically solicit, collect, and incorporate user feedback to inform NRI improvements. DHS concurred with two recommendations and did not concur with one recommendation—that FEMA conduct sensitivity analyses and disclose the results when making significant changes to NRI data sources or methodologies. The department stated that the lack of planned NRI data updates removes the need for such analysis. GAO maintains its recommendation would ensure the NRI meets federal information quality standards.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Federal Emergency Management Agency | The FEMA Administrator should provide additional information for users—such as state and local hazard mitigation and resilience planners—based on common use cases, to help them understand how to apply NRI results effectively when deciding how to reduce risk. (Recommendation 1) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|
| Federal Emergency Management Agency | The FEMA Administrator should ensure the NRI program consistently uses and discloses the outcomes of sensitivity analyses to verify the NRI's results and consider any significant changes to the index's data sources or methodologies. (Recommendation 2) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|
| Federal Emergency Management Agency | The FEMA administrator should create a means—such as a process, strategy, or user group—to systematically solicit, collect, and incorporate user feedback while making ongoing NRI improvements and other updates. (Recommendation 3) |
When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
|