Federal Reserve:

Additional Actions Could Help Ensure the Achievement of Stress Test Goals

GAO-17-48: Published: Nov 15, 2016. Publicly Released: Nov 15, 2016.

Additional Materials:

Contact:

Lawrance L. Evans, Jr
(202) 512-8678
EvansL@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

The Board of Governors of the Federal Reserve System (Federal Reserve) has two related supervisory programs that involve stress testing but serve different purposes. Stress tests are hypothetical exercises that assess the potential impact of economic, financial, or other scenarios on the financial performance of a company. Stress tests of banking institutions typically evaluate if the institutions have sufficient capital to remain solvent under stressful economic conditions.

The Dodd-Frank Wall Street Reform and Consumer Protection Act (Dodd-Frank Act) implements statutory stress test requirements, known as the Dodd-Frank Act Stress Tests (DFAST) for Federal Reserve-supervised banking institutions with more than $10 billion in total consolidated assets. DFAST projects how banking institutions' capital levels would fare in hypothetical stressful economic and financial scenarios. It applies to a broad range of banking institutions and consists of supervisory- and company-run stress tests that produce capital adequacy information for firms' internal use and for public disclosure.

The Federal Reserve also conducts a Comprehensive Capital Analysis and Review (CCAR), which uses DFAST information to assess the capital adequacy (a quantitative assessment) and capital planning processes (a qualitative assessment) for bank holding companies with total consolidated assets of $50 billion or more. CCAR generally does not require additional stress tests and uses the same data, models, and projections used for DFAST.

While the primary purpose of DFAST is to produce and disclose comparable information on the financial condition of banking institutions (the stress test results), the Federal Reserve uses CCAR to make supervisory assessments and decisions about the capital adequacy plans (including proposed capital actions such as dividend payments) of large bank holding companies. For example, the Federal Reserve may object to a company's plan if stress test results show the company's post-stress capital ratios (regulatory measures that indicate how much capital is available to cover unexpected losses) falling below required minimum levels or if the Federal Reserve's qualitative assessment deems the firm's capital planning and related processes inadequate. An objection can result in restrictions on a firm's capital distributions. Several of the companies GAO interviewed that are subject to Federal Reserve stress tests identified benefits from the tests (such as overall improvements in risk management and capital planning) and also identified costs (including for staff resources and other expenditures necessary to conduct the tests and meet the Federal Reserve's supervisory expectations).

GAO found limitations in the Federal Reserve's stress test programs that could hinder their effectiveness.

Qualitative assessment disclosure and communication. The Federal Reserve uses a framework with multiple levels of review to assess qualitative CCAR submissions that helps ensure consistency, but it has not disclosed information needed to fully understand its assessment approach or the reasons for decisions to object to a company's capital plan. Transparency is a key feature of accountability and such incomplete disclosure may limit understanding of the CCAR assessments and hinder public and market confidence in the program and the extent to which the Federal Reserve can be held accountable for its decisions. Federal internal control standards state the importance of relevant and timely communications with external stakeholders. The Federal Reserve has not regularly updated guidance to firms about supervisory expectations and peer practices related to the qualitative assessment. For example, it has not published observations of leading capital planning practices used in CCAR since 2014. The limited communication can pose challenges to companies that must meet these expectations annually and could hinder the achievement of CCAR goals.

Scenario design. The Federal Reserve has a framework for designing stress test scenarios but its analysis of some key design decisions has been limited. For example, the Federal Reserve has not conducted analyses to determine whether the single severe scenario it uses for the supervisory stress test is sufficient to accomplish DFAST and CCAR goals. While there are advantages to using one scenario—including simplicity and transparency—many different types of financial crises are possible, and the single selected scenario does not reflect a fuller range of possible outcomes. Without additional analysis, the Federal Reserve cannot be reasonably assured that banks are resilient against a range of future risks. The Federal Reserve also has not explicitly analyzed how to balance the choice of severity—and its influence on the resiliency of the banking system—with any impact on the cost and availability of credit, which could limit its ability to avoid undesired economic effects from scenario design choices.

Model risk management. Federal Reserve supervisory guidance for banking institutions states that risk from individual models and also from the aggregate system of models should be managed. The Federal Reserve makes supervisory decisions based on the results of its own stress test models, but its management of model risk—the potential for adverse consequences from decisions based on incorrect or misused model outputs—has not focused on its system of models that produce stress test results. To estimate the effect of stress test scenarios on companies' ability to maintain capital, the Federal Reserve has developed individual component models that predict a company's financial performance in the scenarios. The results of these component models are combined with assumed or planned capital actions of companies and form the system of models used by the Federal Reserve. The Federal Reserve has an oversight structure for developing and using models in the supervisory stress tests but its own risk-management efforts have not targeted the system of models. For example, it has not conducted sensitivity and uncertainty analyses—important elements in the Federal Reserve's model risk management guidance—of how its modeling decisions affected overall results. Without such a focus, the Federal Reserve's ability to effectively evaluate and manage model risk and uncertainty from the entire system of stress test models will be limited. Understanding and communicating this uncertainty is critical because the outcome of the CCAR assessment can have significant implications for a company, including limiting its capital actions (such as dividend payments and share repurchases).

Why GAO Did This Study

The Federal Reserve has two stress test programs for certain banking institutions it supervises. DFAST encompasses stress tests required by the Dodd-Frank Act. CCAR comprises a qualitative assessment of firms' capital planning processes and a quantitative assessment of firms' ability to maintain sufficient capital to continue operations under stress. Questions have been raised about the effectiveness and burden of requiring two stress test programs. GAO was asked to review these programs and their effectiveness. This report examines how the stress test programs compare, the CCAR qualitative assessment, and the design of the stress test scenarios and models.

GAO analyzed Federal Reserve documents including rules, guidance, and internal policies and procedures on DFAST and CCAR implementation and assessed practices against federal internal control standards and other criteria. GAO also interviewed Federal Reserve staff and officials of 19 banking institutions selected based on characteristics such as their size, prior stress test participation, and history of CCAR results.

What GAO Recommends

GAO is making 15 recommendations to help improve the effectiveness of the Federal Reserve's stress test programs, such as improving disclosures and communications to firms, considering the potential consequences of its scenario design choices, and expanding model risk management to include the entire system of models. The Federal Reserve generally agreed with the recommendations and highlighted select ongoing and future efforts.

For more information, contact Lawrance L. Evans, Jr. at (202) 512-8678 or EvansL@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To help improve the consistency of federal banking regulators' stress test requirements and help ensure that institutions overseen by different regulators receive consistent regulatory treatment, the heads of the Federal Reserve, the Federal Deposit Insurance Corporation, and the Office of the Comptroller of the Currency should harmonize their agencies' approach to granting extensions and exemptions from stress test requirements.

    Agency Affected: Department of the Treasury: Office of the Comptroller of the Currency

  2. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To help improve the consistency of federal banking regulators' stress test requirements and help ensure that institutions overseen by different regulators receive consistent regulatory treatment, the heads of the Federal Reserve, the Federal Deposit Insurance Corporation, and the Office of the Comptroller of the Currency should harmonize their agencies' approach to granting extensions and exemptions from stress test requirements.

    Agency Affected: Federal Deposit Insurance Corporation

  3. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To help improve the consistency of federal banking regulators' stress test requirements and help ensure that institutions overseen by different regulators receive consistent regulatory treatment, the heads of the Federal Reserve, the Federal Deposit Insurance Corporation, and the Office of the Comptroller of the Currency should harmonize their agencies' approach to granting extensions and exemptions from stress test requirements.

    Agency Affected: Federal Reserve System

  4. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To help provide stronger incentives for companies to perform company-run stress tests in a manner consistent with Federal Reserve goals, the Federal Reserve should remove company-run stress tests from the CCAR quantitative assessment.

    Agency Affected: Federal Reserve System

  5. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To increase transparency and improve CCAR effectiveness, the Federal Reserve should publicly disclose additional information that would allow for a better understanding of the methodology for completing qualitative assessments, such as the role of ratings and rankings and the extent to which they affect final determination decisions.

    Agency Affected: Federal Reserve System

  6. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To increase transparency and improve CCAR effectiveness, the Federal Reserve should, for future determinations to object or conditionally not object to a company's capital plan on qualitative grounds, disclose additional information about the reasons for the determinations.

    Agency Affected: Federal Reserve System

  7. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To increase transparency and improve CCAR effectiveness, the Federal Reserve should publicly disclose, on a periodic basis, information on capital planning practices observed during CCAR qualitative assessments, including practices the Federal Reserve considers stronger or leading practices.

    Agency Affected: Federal Reserve System

  8. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To increase transparency and improve CCAR effectiveness, the Federal Reserve should improve policies for official responses to CCAR companies by establishing procedures for notifying companies about time frames relating to Federal Reserve responses to company inquiries.

    Agency Affected: Federal Reserve System

  9. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To strengthen the scenario design process, the Federal Reserve should assess--and adjust as necessary--the overall level of severity of its severely adverse scenario by establishing a process to facilitate proactive consideration of levels of severity that may fall outside U.S. postwar historical experience.

    Agency Affected: Federal Reserve System

  10. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To strengthen the scenario design process, the Federal Reserve should assess--and adjust as necessary--the overall level of severity of its severely adverse scenario by expanding consideration of the trade-offs associated with different degrees of severity.

    Agency Affected: Federal Reserve System

  11. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To improve understanding of the range of potential crises against which the banking system would be resilient and the outcomes that might result from different scenarios, the Federal Reserve should assess whether a single severe supervisory scenario is sufficient to inform CCAR decisions and promote the resilience of the banking system. Such an assessment could include conducting sensitivity analysis involving multiple severe supervisory scenarios--potentially using CCAR data for a cycle that is already complete, to avoid concerns about tailoring the scenario to achieve a particular outcome.

    Agency Affected: Federal Reserve System

  12. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To help ensure that Federal Reserve stress tests do not amplify future economic cycles, the Federal Reserve should develop a process to test its proposed severely adverse scenario for procyclicality annually before finalizing and publicly releasing the supervisory scenarios.

    Agency Affected: Federal Reserve System

  13. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To improve the Federal Reserve's ability to manage model risk and ensure that decisions based on supervisory stress test results are informed by an understanding of model risk, the Federal Reserve should apply its model development principles to the combined system of models used in the supervisory stress tests.

    Agency Affected: Federal Reserve System

  14. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To improve the Federal Reserve's ability to manage model risk and ensure that decisions based on supervisory stress test results are informed by an understanding of model risk, the Federal Reserve should create an appropriate set of system-level model documentation, including an overview of how component models interact and key assumptions made in the design of model interactions.

    Agency Affected: Federal Reserve System

  15. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To improve the Federal Reserve's ability to manage model risk and ensure that decisions based on supervisory stress test results are informed by an understanding of model risk, the Federal Reserve should design and implement a process to test and document the sensitivity and uncertainty of the model system's output--the post-stress capital ratios used to make CCAR quantitative assessment determinations--including, at a minimum, the cumulative uncertainty surrounding the capital ratios and their sensitivity to key model parameters, specifications, and assumptions from across the system of models.

    Agency Affected: Federal Reserve System

  16. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To improve the Federal Reserve's ability to manage model risk and ensure that decisions based on supervisory stress test results are informed by an understanding of model risk, the Federal Reserve should design and implement a process to communicate information about the range and sources of uncertainty surrounding the post-stress capital ratio estimates to the Board during CCAR deliberations.

    Agency Affected: Federal Reserve System

  17. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: To improve the Federal Reserve's ability to manage model risk and ensure that decisions based on supervisory stress test results are informed by an understanding of model risk, the Federal Reserve should design and implement a process for the Board and senior staff to articulate tolerance levels for key risks identified through sensitivity testing and for the degree of uncertainty in the projected capital ratios.

    Agency Affected: Federal Reserve System

 

Explore the full database of GAO's Open Recommendations »

Apr 19, 2017

Mar 29, 2017

Feb 24, 2017

Dec 29, 2016

Dec 15, 2016

Nov 15, 2016

Nov 10, 2016

Nov 4, 2016

Looking for more? Browse all our products here