FEMA Flood Maps:
Some Standards and Processes in Place to Promote Map Accuracy and Outreach, but Opportunities Exist to Address Implementation Challenges
GAO-11-17, Dec 2, 2010
Additional Materials:
- Highlights Page:
- Accessible Text:
Contact:
(202) 512-3000
contact@gao.gov
Office of Public Affairs
(202) 512-4800
youngc1@gao.gov
The Federal Emergency Management Agency (FEMA), a component of the Department of Homeland Security (DHS), maps flood hazard areas across the country and makes flood insurance available to more than 20,100 communities through the National Flood Insurance Program. From 2003 through 2008, FEMA spent $1.2 billion in a comprehensive effort to update the nation's flood insurance maps. In 2009, FEMA began an annual review of 20 percent of the nation's flood maps, for which Congress allocated $440 million in 2009 and 2010. As requested, GAO reviewed the actions FEMA has taken to enhance the accuracy of updated flood maps, and FEMA's outreach efforts in conducting flood mapping activities. GAO analyzed FEMA's mapping standards and information systems, tested quality assurance processes, and interviewed FEMA officials and contractors.
FEMA has taken a number of steps to enhance the accuracy of flood maps, but challenges related to implementing standards to ensure map accuracy remain. Steps FEMA has taken include adopting a risk-based method to prioritize mapping projects, implementing mapping standards and guidance, establishing risk-based standards for topographic detail to ensure that the highest risk areas have the most accurate topographic data, and implementing quality control processes for ensuring engineering data is collected and used in accordance with standards. However, FEMA's mapping standards could be improved. For example, FEMA has standards for determining the extent to which new and updated flood mapping data are sufficiently current to promote map accuracy, yet FEMA has not developed uniform guidance for the validation of existing mapping data. Doing so could help FEMA both track and report the accuracy of maps at the national and regional levels and better assess mapping data needs. FEMA's quality control process for ensuring the accuracy of flood maps could also be improved. Audits of FEMA's mapping contractors' efforts have been conducted since 2006 by an independent verification contractor; however, FEMA officials said they planned to transfer responsibility for the verification audits, part of its independent verification and validation process, to its program management contractor by the end of this year, who will then monitor FEMA's mapping contractors. The transfer of these responsibilities creates a potential conflict of interest because the program management contractor is to monitor the results of its program management efforts. According to industry best practices, verification and validation efforts should be independent and reported directly to senior management to provide added assurance that reported results on the project's status are unbiased. The performance of the verification and validation function by an entity that is technically, managerially, and financially independent of the organization in charge of what it is assessing could better position FEMA to help ensure the independence of the verification and validation function, both in appearance and in fact. FEMA has taken a variety of steps to conduct outreach to state and local officials, including developing a national outreach strategy, but could enhance its efforts to improve public awareness and promote map acceptance. For example, FEMA has not developed performance goals or measures, or identified the resources needed for its flood mapping outreach efforts, which could help FEMA better determine whether its outreach efforts are achieving their intended results. In addition, FEMA could better quantify, allocate, and leverage resources needed to support national outreach efforts. For example, by tracking spending and using risk in its decisions for allocating outreach resources, FEMA could better allocate resources for flood mapping outreach efforts. In addition, FEMA could enhance its outreach efforts by leveraging existing flood insurance marketing resources and expertise during the mapping process to increase public acceptance of flood maps. Among other things, GAO recommends that FEMA establish guidance for validating data, transfer responsibility for verification audits to an independent entity, and establish goals and measures for promoting public acceptance of mapping. FEMA concurred with 10 of the 11 recommendations in this report, but disagreed with transferring verification audit duties to an independent entity because it believes its program management contractor is sufficiently independent. GAO believes this recommendation remains valid as stated in this report.
Status Legend:
- Review Pending
- Open
- Closed - implemented
- Closed - not implemented
Recommendations for Executive Action
Recommendation: To address challenges in ensuring the accuracy of flood maps, the Administrator of the Federal Emergency Management Agency should establish separate measures and collect data needed to assess compliance with the Floodplain Boundary Standard for detailed and approximate flood studies.
Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Open
Comments: In its 60-day letter dated March 1, 2011, FEMA concurred with this recommendation to establish tracking and reporting that will allow FEMA to report the level of precision by flood source. On December 15, 2010, FEMA initiated the implementation of the Coordinated Needs Management Strategy (CNMS) and provided a memorandum implementing CNMS. According to FEMA, the CNMS system will provide significantly improved national tracking and reporting on the flood hazard data inventory; flooding sources with new and updated flood hazards must comply with the floodplain boundary standard and CNMS will track the study method for each flooding source. This recommendation remains open pending completion of this assessment.
Recommendation: To address challenges in ensuring the accuracy of flood maps, the Administrator of the Federal Emergency Management Agency should establish uniform guidance for the validation of existing engineering data to help FEMA fully implement the NVUE standard and provide a basis for mapping partners to validate flood hazard data.
Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Closed - Implemented
Comments: In its 60-day letter dated March 1, 2011, FEMA concurred with this recommendation. On December 15, 2010, FEMA issued guidance for validating existing engineering data and procedures to implement a full validation assessment of the flood map inventory utilizing the Coordinated Needs Management Strategy CNMS. FEMA will utilize these procedures to verify the validity of existing engineering data prior to the initiation of a new or revised engineering flood study.
Recommendation: To enhance the independent verification and validation (IV&V) audit process, the Administrator of the Federal Emergency Management Agency should implement probability sampling during the IV&V audit process to the extent that the benefits outweigh the costs, to ensure that the results are generalizable for decisionmaking.
Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Closed - Implemented
Comments: In its 60-day letter dated March 1, 2011, FEMA concurred with this recommendation and said the agency is assessing the costs, benefits, and feasibility of implementing probability sampling as a component of the Risk MAP IV& V function. In August 2012, FEMA issued an IV&V Implementation Plan that established an approach to for project selection and a review methodology based on, among other things, requirements for statistical representation with respect to the distribution of the project population by FEMA region and mapping services provider. As a result, FEMA's IV&V process will better ensure that the results of nation-wide audits are generalizable and help FEMA management use the information more strategically.
Recommendation: To enhance the independent verification and validation (IV&V) audit process, the Administrator of the Federal Emergency Management Agency should transfer IV&V duties back to an independent entity to help ensure impartiality.
Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Closed - Implemented
Comments: Although FEMA nonconcured with this recommendation agency officials provided documentation that the Program Manager (PM) contractor is technically, managerially, and financially independent of the flood hazard development process and that the IV&V activities were contractually segregated from map production and technical services elements of the PM contract. Because of these steps to help ensure impartiality in IV&V activities, FEMA's efforts should help prevent a conflict of interest. In October 2012, FEMA provided documentation of its contractual approach to ensure impartiality in the conduct of the IV&V process including prohibitions specifying that subcontractors for the Program Management Contractor may not pursue any FEMA map production contracts and that the incentives are for the IV&V function are not linked to mapping production and quality services. As a result of these steps taken to specifically highlight and resolve issues of potential conflict, FEMA's ability to help ensure impartiality in the IV&V process will be enhanced and, thereby, the quality of the audit results.
Recommendation: To enhance the independent verification and validation (IV&V) audit process, the Administrator of the Federal Emergency Management Agency should adopt a systematic approach to IV&V data collection, so FEMA can better track map quality issues, more easily analyze the data, and adopt a corrective action plan.
Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Closed - Implemented
Comments: In its 60-day letter dated March 1, 2011, FEMA concurred with this recommendation. FEMA will work with the contractor performing audits to redesign the reporting of map quality issues. FEMA is assessing the feasibility of implementing additional data analysis capabilities as a component of the Risk MAP IV&V function. In August 2012, FEMA provided a report on its mapping IV&V results that described the program goal to identify systematic program quality gaps and issues to inform continuous improvement activities by conducting quality compliance reviews based on established evaluation criteria. As a result of its more systematic data collection, FEMA's ability to establish corrective action plans to resolve issues identified through the IV&V process will be enhanced and FEMA management can use the information from national audits more strategically.
Recommendation: To address challenges in improving community outreach, the Administrator of the Federal Emergency Management Agency should establish a mechanism to better ensure compliance with the documentation requirements of public notification regulations.
Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Closed - Implemented
Comments: In its 60-day letter dated March 1, 2011, FEMA concurred with this recommendation. FEMA is finalizing a procedure memorandum to reinforce the existing contract requirements to compile and archive this data. FEMA will also transition to fully digital storage of these data on the Mapping Information Process (MIP) and FEMA will incorporate a review of the required deliverables into the quality management process. FEMA issued guidance for Flood Hazard Mapping Partners on May 25, 2011 establishing FEDD file review protocols for mapping projects, emphasizing the importance of timely and complete submission of Flood Insurance Study support data, and clarifying mapping partner responsibilities to ensure that all FEDD Files are accurate and complete by having mapping partners use a checklist and certify that flood mapping project documentation is complete. As a result of these efforts, FEMA will be better able to ensure that the agency is notifying the public as required and improve its community outreach efforts.
Recommendation: To address challenges in improving community outreach, the Administrator of the Federal Emergency Management Agency should collect and analyze data on appeals and protests, including those on ineligible appeals, to the extent that the benefits outweigh the costs.
Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Closed - Implemented
Comments: FEMA concurred with this recommendation and said the agency is coordinating with Regional Offices, Mapping Partners, and other stakeholders in assessing the feasibility and methodology for collecting, tracking, and analyzing appeals and protests. In September 2012, FEMA issued "Guidance for Tracking and Reporting the Instances and Root Causes of Appeals and Revised Preliminaries in the FEMA Mapping Production Lifecycle" that will provide the information needed to assess causal factors for appeals and protests and take effective corrective actions, as needed. As a result, FEMA's IV&V process will better ensure that the results of nation-wide audits are generalizable and help FEMA management use the information more strategically.
Recommendation: To address challenges in improving community outreach, the Administrator of the Federal Emergency Management Agency should issue guidance to mapping stakeholders to standardize the process for analyzing appeals and protests and submitting this data to FEMA.
Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Closed - Implemented
Comments: In its 60-day letter dated March 1, 2011, FEMA concurred with this recommendation and said the agency is coordinating with Regional Offices, Mapping Partners, and other stakeholder in assessing the feasibility and methodology for collecting, tracking, and analyzing appeals and protests. In September 2012, FEMA issued guidance that outlines a process to track the instances and root causes of community appeals to enhance visibility of systematic quality issues, provide transparency to stakeholders on the nature and causes of when revisions are necessary, and analyze any trends within the floodmapping production process when appeals are received. As a result, FEMA will be better able utilize data on community appeals and protests to inform its decisions about where to focus outreach efforts, and standardize the process by which mapping partners analyze appeals.
Recommendation: To address challenges in improving community outreach, the Administrator of the Federal Emergency Management Agency should establish performance goals and measures for promoting public acceptance of flood maps.
Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Closed - Implemented
Comments: In its 60-day letter dated March 1, 2011, FEMA concurred with this recommendation. Through the Risk MAP program, FEMA's goals are to develop flood maps that will build awareness and influence communities to take action to reduce their flood risk. While quality flood hazard data is the base component of this goal, community acceptance of the maps is critical to achieve the goal of increasing flood risk awareness. FEMA currently measures the community adoption rates for maps. FEMA is initiating the development and use of performance measures to track quality data, the level of community awareness, and where Risk MAP has been deployed. In September 2012, FEMA issued guidance that outlines a process to track the instances and root causes of community appeals. The guidance includes specific criteria for how data on appeals is to be categorized, including how to define and input root causes so that the data on community appeals can be collected and analyzed. As a result, FEMA will be better able utilize data on community appeals and protests to inform its decisions about where to focus outreach efforts, and standardize the process by which mapping partners analyze appeals.
Recommendation: To address challenges in improving community outreach, the Administrator of the Federal Emergency Management Agency should develop a reporting structure for regions to use to identify resources needed to conduct flood mapping outreach activities, and implement a risk-based approach to allocate outreach resources.
Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Closed - Implemented
Comments: In its 60-day letter dated March 1, 2011, FEMA concurred with this recommendation and said that Risk MAP formalizes outreach and communications, as well as certain products as part of the standard Risk MAP project workflow to be tracked and reported for earned value and other program management purposes. FEMA has convened a work group to develop and recommend an interim solution for project-level tracking and reporting of communications and outreach activities and new products. Once the interim solution is in place, FEMA will begin to explore and evaluate a longer-term and more permanent solution. FEMA is beginning to explore ways to analyze other outreach efforts. FEMA is looking at ways to quantify and track outreach activities that are integrated into map production activities that have historically been difficult to separate from production costs. Finally, FEMA indicated that it would define a process for allocating resources to Risk MAP outreach based on project risk, but that the agency anticipated the risk factors for outreach may differ from risk factors used to allocate engineering resources. In September 2012, FEMA issued guidance that outlines a process to track the instances and root causes of community appeals. The guidance includes specific criteria for how data on appeals is to be categorized, including how to define and input root causes so that the data on community appeals can be collected and analyzed. As a result, FEMA will be better able utilize data on community appeals and protests to inform its decisions about where to focus outreach efforts, and standardize the process by which mapping partners analyze appeals.
Recommendation: To address challenges in improving community outreach, the Administrator of the Federal Emergency Management Agency should leverage, as appropriate, existing FloodSmart marketing resources and expertise to help increase public acceptance of flood maps.
Agency Affected: Department of Homeland Security: Directorate of Emergency Preparedness and Response: Federal Emergency Management Agency
Status: Closed - Implemented
Comments: In its 60-day letter dated March 1, 2011, FEMA concurred with this recommendation. FEMA said its Risk MAP Outreach Team has established a regular bi-weekly meeting with FloodSmart leadership and providers to identify opportunities to leverage FloodSmart expertise and resources within Risk MAP's community engagement and outreach efforts. For example, during a recent meeting, FEMA engaged FloodSmart leadership in a review of the Risk MAP project timeline to identify when and how best to leverage FloodSmart resources to disseminate flood risk and insurance information within a given project area. Agency officials believe these meetings will be effective and productive in other ways by: Providing a regular forum for updating and sharing of Risk MAP and FloodSmart activities, lessons learned, best practices, or other insights; Obtaining recommendations and comment on our annual flood risk awareness survey design; Providing FloodSmart insight and support for engaging insurance, lender, real estate agent, and developer industries into the development of Risk MAP project meeting agendas and guidelines; Learning about current and upcoming FloodSmart initiatives and gathering feedback on public response to recent FloodSmart marketing efforts, either Regional or national in scope; and providing Risk MAP input into FloodSmart toolkits and other materials, such as the FloodSmart mapping toolkit, to ensure that Risk MAP information and messaging is presented consistently. In addition to these bi-weekly meetings, FEMA said the agency has begun coordinating stakeholder outreach efforts, including conference support, to present a unified message regarding flood mapping and flood insurance. In February 2012, FEMA provided an example of a September 2011 fact sheet reflecting recent RiskMap efforts to communicate flood risk, insurance and mitigation information for the public. In addition, FEMA's FloodSmart Website now includes a variety of resources available to the public that are designed to communicate the importance of flood mapping efforts.