Skip to main content

Environmental Regulation: EPA Should Improve Adherence to Guidance for Selected Elements of Regulatory Impact Analyses

GAO-14-519 Published: Jul 18, 2014. Publicly Released: Aug 11, 2014.
Jump To:
Skip to Highlights

Highlights

What GAO Found

The Environmental Protection Agency (EPA) used the seven Regulatory Impact Analyses (RIA) GAO reviewed to inform decision making, and its adherence to relevant Office of Management and Budget (OMB) guidance varied. According to senior EPA officials, the agency used these RIAs to facilitate communication with management throughout the rulemaking process and communicate information that supported its regulatory decisions to Congress and the public. However, it generally did not use them as the primary basis for final regulatory decisions.

EPA generally adhered to many aspects of OMB's Circular A-4 guidance for analyzing the economic effects of regulations including, for example, considering regulatory alternatives and analyzing uncertainties underlying its RIAs. However, EPA did not always adhere to other aspects. Specifically, the information EPA included and presented in the RIAs was not always clear. According to OMB guidance, RIAs should communicate information supporting regulatory decisions and enable a third party to understand how the agency arrives at its conclusions. In addition, EPA's review process does not ensure that the information about selected elements that should appear in the analyses—such as descriptions of baselines and alternatives considered—is transparent or clear, within and across its RIAs. As a result, EPA cannot ensure that its RIAs adhere to OMB's guidance to provide the public with a clear understanding of its decision making.

In addition to using Circular A-4 (issued in 2003) to analyze the effects of regulations, EPA used more recent guidance developed by an interagency working group co-led by OMB and another White House office in 2010 for valuing carbon dioxide emissions. Applying this guidance while using Circular A-4 to estimate other benefits and costs yielded inconsistencies in some of EPA's estimates and has raised questions about whether its approach was consistent with Circular A-4. Circular A-4 does not reference the new guidance and the new guidance does not include an overall statement explaining its relationship to Circular A-4. Without increased clarity about the relationship, questions about the agencies' adherence to OMB guidance will likely persist.

In assessing EPA's adherence to OMB guidance, GAO identified two other areas in which EPA faced challenges that limited the usefulness of some of its estimates. First, EPA did not monetize certain benefits and costs related to the primary purposes or key impacts of the rules GAO reviewed, such as reducing hazardous air pollutants and water quality effects. EPA officials said resource and data limitations constrained the agency's ability to monetize these effects. OMB guidance acknowledges that monetizing effects is not always possible. However, without doing so, the public may face challenges understanding the trade-offs associated with regulatory alternatives. Second, EPA estimated effects of its regulations on employment, in part, using a study that, according to EPA officials, represented the best reasonably obtainable data when they conducted their analyses. However, the study was based on data that were more than 20 years old and may not have represented the regulated entities addressed in the RIAs. EPA officials said they are exploring new approaches for analyzing these effects but were uncertain about when such results would be available. Without improvements in its estimates, EPA's RIAs may be limited in their usefulness for helping decision makers and the public understand these important effects.

Why GAO Did This Study

Federal regulations, especially those addressing health, safety, and the environment, can generate hundreds of billions of dollars in benefits and costs to society annually. Various statutes, executive orders, and OMB guidance direct federal agencies to analyze the benefits and costs of proposed regulations. These analyses—known as RIAs—can also provide affected entities, agencies, Congress, and the public with important information about the potential effects of new regulations.

According to OMB, EPA regulations account for the majority of the estimated benefits and costs of major federal regulations. GAO was asked to review EPA's RIAs for recent regulations. This report examines how EPA has used RIAs during the rulemaking process and the extent to which EPA adhered to OMB guidance on selected elements of RIAs for recent rules. GAO reviewed RIAs from a nonprobability sample of seven recent air, water, and other environmental regulations, assessed them against relevant OMB guidance, and interviewed agency officials.

Recommendations

GAO recommends that EPA improve adherence to OMB guidance and enhance the usefulness of its RIAs, and that OMB clarify the application of guidance for estimating the benefits of reducing greenhouse gas emissions. In commenting on a draft of this report, EPA stated that it generally agreed with GAO's recommendations. On behalf of OMB, in oral comments OMB staff said that they neither agreed nor disagreed with the recommendations but saw some merit in them.

Recommendations for Executive Action

Agency Affected Recommendation Status
Environmental Protection Agency To improve future adherence to OMB guidance for conducting RIAs, the EPA Administrator should enhance the agency's review process for RIAs to ensure the transparency and clarity of information presented for selected elements in and across RIAs.
Closed – Implemented
In May 2017, EPA reported that the National Center for Environmental Economics (NCEE) in the Office of Policy would review RIAs for economically significant rules and provide a written review prior to their submission to the Office of Management and Budget (OMB). In October 2019, EPA reported that it was taking additional steps to ensure the transparency and clarity of information presented for selected elements in and across RIAs. Specifically, EPA reported that the NCEE is revising the agency's Guidelines for Conducting Economic Analyses to include OMB's checklist for RIAs, which according to EPA, has criteria designed to reinforce the use of sound economic theory and methods, consistent with the provisions in Executive Order 12866 and Circular A-4. In addition, EPA reported that the agency is engaging its Economics Forum Steering Committee--comprised of the most senior economists in each of the program offices--to advance the quality, clarity, and consistency within and across RIAs by discussing analytic treatments and challenges related to issues such as employment impacts analysis, economywide modeling, health benefits and lead, and calculation of costs and cost savings. Finally, EPA reported that the NCEE established an internal workbook to operationalize accounting standards and establish consistency across the agency in cost and cost savings calculations. This effort was also recognized by EPA's Office of the Inspector General as a noteworthy achievement. Based on these actions, GAO believes EPA has taken steps to ensure the transparency and clarity of information presented for selected elements in and across RIAs, and we consider this recommendation closed.
Environmental Protection Agency To improve future adherence to OMB guidance for conducting RIAs, the EPA Administrator should improve the accuracy, transparency, and clarity of the information included in the executive summaries of each RIA.
Closed – Implemented
In November 2014, EPA's Office of Policy issued a memo to all program offices requesting that the agency pay particular attention to transparency and clarity as it drafts its RIAs. The memo included references to useful resources from OMB's and EPA's Economic Guidelines. In addition, the memo noted that the Office of Policy's National Center for Environmental Economics staff are available to assist on improving transparency, clarity, and other aspects of EPA's RIAs. Based on the executive summaries of three regulatory impact assessments EPA has issued since November 2014, GAO believes the agency has taken steps to improve the transparency and clarity of the information included in the executive summaries. For example, in one executive summary, EPA included a detailed two-and-a-half page table identifying the unquantified, quantified, and monetized benefits it considered. In another summary, EPA included an explicit statement of the baseline, cross-referenced technical support document where readers could obtain additional detail, and stated what the agency could not monetize. In addition, EPA is providing greater emphasis on transparency and clarity in the economic analysis component of its Action Development Training. Based on these examples and the expectation that the trend toward increased transparency and clarity will continue, GAO is closing this recommendation as implemented.
Office of Management and Budget To clarify the relationship between OMB Circular A-4 and an Interagency Working Group's Technical Support document for estimating the effects of changes in carbon dioxide emissions, and the approach agencies should use when informing decision makers and the public of their findings, the Director of OMB should consider continuing monitoring the economic literature and working with agencies to identify approaches for presenting social cost of carbon estimates with other analytical results that have been discounted at different rates to help agencies more transparently communicate about the circumstances unique to assessing the long-term effects of changes in carbon dioxide emissions.
Closed – Not Implemented
In March 2017, Executive Order 13783 called for the Interagency Working Group to be disbanded and for the withdrawal of six specified Interagency Working Group technical documents as no longer representative of governmental policy. Because of this change, we believe future action on this recommendation is unlikely, and we are closing the recommendation as not implemented.
Environmental Protection Agency In addition, to enhance the usefulness of EPA's RIAs, the EPA Administrator should identify and prioritize for research key categories of benefits and costs that the agency cannot currently monetize that, once monetized, would most enhance the agency's ability to consider economic trade-offs associated with different regulatory alternatives.
Closed – Implemented
EPA has reported taking multiple steps in response to GAO's recommendation. EPA reported that, beginning in 2016, the agency has identified priority research categories of benefits and costs for its program offices. To do so, it inventoried existing ongoing research, which served as a baseline for identifying research gaps and priorities. In addition, the National Center for Environmental Economics (NCEE) engaged program offices as it reviewed RIAs and identified gaps in valuation of benefits and costs that are research priorities for future RIAs. The agency also reported that its Economics Forum's discussions of cross-office economic issues helps inform research priorities. Among several priorities identified, EPA reported that, beginning in 2017, the agency had identified two research needs that cut across regulatory programs-health benefits and economy-wide modeling of cost impacts-and initiated several research efforts to study these issues. As part of this effort, EPA convened a Science Advisory Board panel to consider the use of economy-wide models to evaluate regulatory impacts at a broader scope than EPA's typically more narrowly focused attention on the costs born in the directly regulated sector. In response to the SAB recommendations, EPA tasked NCEE with leading the agency's efforts to develop economy-wide modeling capabilities. In 2018, NCEE demonstrated progress on this effort, highlighted by published working papers, including documentation of a new, EPA-developed CGE model and an application of the model to study when the CGE modeling approach might add value in regulatory analysis. Based on these actions, GAO believes EPA has taken steps to identify and prioritize research categories that would enhance the agency's RIAs, and we consider this recommendation closed.
Office of Management and Budget To clarify the relationship between OMB Circular A-4 and an Interagency Working Group's Technical Support document for estimating the effects of changes in carbon dioxide emissions, and the approach agencies should use when informing decision makers and the public of their findings, the Director of OMB should consider clarifying the relationship between OMB Circular A-4 and the Technical Support Document by increasing the visibility of relevant language in the Technical Support Document.
Closed – Not Implemented
In March 2017, Executive Order 13783 called for the Interagency Working Group to be disbanded and for the withdrawal of six specified Interagency Working Group technical documents as no longer representative of governmental policy. Because of this change, we believe future action on this recommendation is unlikely, and we are closing the recommendation as not implemented.
Environmental Protection Agency In addition, to enhance the usefulness of EPA's RIAs, the EPA Administrator should continue efforts to update and improve the agency's approach to estimating employment effects.
Closed – Implemented
In April 2018, EPA reported that it had updated its approach to estimating employment effects in RIAs for economically significant rules. Specifically, EPA stated that it now emphasizes describing a rule's impact on gross changes in employment, rather than limiting concern to net changes, and according to EPA, analysts now discuss job declines and where and to whom they might occur regardless of potential offsetting job increases that might occur elsewhere. EPA also stated that it now emphasizes highlighting the importance of impacts on specific groups of workers, and according to EPA, analysts should examine changes in employment by industry, differences in workers' attributes, and geographic location where unemployment rates are high, to the extent possible. EPA reported that this updated approach was demonstrated in EPA's October 2017 RIA for the proposed review of the Clean Power Plan. We reviewed EPA's approach and found that it met the intent of our recommendation.

Full Report

Office of Public Affairs

Topics

Environmental regulationsEnvironmental lawEnvironmental monitoringFederal regulationsInteragency relationsRegulatory agenciesBenefit-cost trackingTransparencyEconomic analysisEmissions