Skip to Highlights
Highlights

What GAO Found

Allowance rates—the rate at which Social Security Administration (SSA) administrative law judges allowed disability benefits to be paid when claimants appealed—varied across judges, even after holding constant certain characteristics of claimants, judges, hearing offices, and other factors that could otherwise explain differences in allowance rates. Specifically, GAO estimated that the allowance rate could vary by as much as 46 percentage points if different judges heard a typical claim (one that was average in all other factors GAO analyzed). SSA officials said that this level of variation is not surprising, given the complexity of appeals and judicial discretion. Nonetheless, the variation declined by 5 percentage points between fiscal years 2007 and 2015 (see figure), a change officials attributed to enhanced quality assurance efforts and training for judges. GAO also identified various factors that were associated with a greater chance that a claimant would be allowed benefits. In addition to characteristics related to disability criteria, such as the claimant's impairment and age, GAO found that claimants who had representatives, such as an attorney or family member, were allowed benefits at a rate nearly 3 times higher than those without representatives. Other factors did not appear related to allowance rates, such as the percentage of backlogged claims in a hearing office.

Estimated Allowance Rates across Social Security Administrative Law Judges for Typical Disability Claims, Fiscal Years 2007-2015

Estimated Allowance Rates across Social Security Administrative Law Judges for Typical Disability Claims, Fiscal Years 2007-2015

Note: The range is the difference in allowance rates for judges at the 95th and 5th percentiles. We also identify the 75th and 25th percentiles. A typical claim had average values on all other factors we analyzed.

SSA has various reviews to monitor the accuracy and consistency of hearings decisions by administrative law judges, but some of these reviews may overlap and SSA has not systematically evaluated them. Specifically, SSA conducts five types of quality assurance reviews of hearings decisions, several of which have similar goals and may look at similar claims. SSA has not evaluated the efficiency or effectiveness of these reviews, despite spending at least $11 million on them in fiscal year 2016. Moreover, the agency has struggled to sustain all of its quality reviews due to competing priorities—two of the five reviews were curtailed in 2016 because SSA reassigned staff to help expedite claims decisions. By evaluating which quality assurance reviews are most effective and efficient in improving accuracy and consistency, SSA would be better positioned to meet its goals within its resources.

Why GAO Did This Study

Individuals who do not agree with the initial decision on a claim for Social Security disability benefits can ultimately appeal the decision by requesting a hearing before one of SSA's approximately 1,500 administrative law judges. However, the rate at which these judges have allowed benefits has varied, raising questions about the reasons for this variation. GAO was asked to review aspects of SSA's oversight of judges' decisions.

This report examines (1) to what extent allowance rates vary across administrative law judges, and factors associated with this variation; and (2) the extent to which SSA has processes to monitor the accuracy and consistency of hearings decisions.

GAO developed a statistical model to analyze SSA data on adult disability decisions made by administrative law judges from fiscal years 2007 through 2015, the most current data available at the time of GAO's analysis; reviewed relevant federal laws, regulations, and agency documents; and interviewed SSA officials and chief judges in SSA's 10 regions, as well as officials from organizations representing judges, disability claimants, and claimant representatives.

Skip to Recommendations

Recommendations

GAO is making two recommendations, including that SSA systematically evaluate its quality assurance reviews and take steps to reduce or better manage any unnecessary overlap among them. SSA concurred and plans to address them through a comprehensive assessment of its oversight.

Recommendations for Executive Action

Agency Affected Recommendation Status
Social Security Administration The Commissioner of SSA should develop a set of public performance measures, to include accuracy and consistency, as well as timeliness, of administrative law judges' (ALJ) disability decisions. SSA could consider whether existing quality review or monitoring efforts could provide suitable data for such measures. (Recommendation 1)
Open
SSA agreed with this recommendation and stated that the agency would address it as part of a comprehensive assessment and refinement of its oversight roles and processes. As of January 2021, SSA officials reported that the agency has expectations, measures, and processes in place to ensure administrative law judges comply with policy and adjudicate a sufficient number of cases in a quality and timely manner. Further, officials noted that they continuously evaluate whether there are other metrics and methodologies that can help them manage administrative law judges' performance. However, the agency has decided not to make those performance expectations and measures available to the public and did not provide additional information on the types of measures that are in place. By not sharing performance information on the accuracy and consistency of hearings-level decisions, SSA may not be providing the public with adequate information on progress toward its objective to improve the quality, consistency, and timeliness of disability decisions.
Social Security Administration The Commissioner of SSA should systematically evaluate the efficiency and effectiveness of its quality assurance reviews and take steps to reduce or better manage any unnecessary overlap among them to ensure strategic use of resources. Such steps could include enhancing collaboration where reviews overlap or only conducting the reviews that are most efficient and effective in achieving agency goals for improving accuracy and consistency of ALJ disability decisions. (Recommendation 2)
Open
SSA agreed with this recommendation and stated that it would address this recommendation as part of a comprehensive assessment and refinement of its oversight roles and processes. In March 2018, SSA noted that the Offices of Quality Review and Appellate Operations have been consolidated under the Office of Analytics, Review, and Oversight (OARO). According to SSA, OARO was created to enhance collaboration, eliminate unnecessary overlap, and enhance the efficiency of reviews. SSA also has taken some steps to evaluate the effectiveness of its hearings-level quality assurance reviews. Specifically, in September 2020, OARO analyzed a national sample of favorable and unfavorable hearing decisions from fiscal year 2019 and compared the results from the national sample to results from most of the existing quality assurance reviews. SSA officials reported that the results were consistent across reviews and, as such, they have a high level of confidence that the existing quality assurance and monitoring efforts provide suitable data and should remain in effect. However, officials reported in December 2020 that the Office of Quality Review is reinstating a review it last performed in fiscal year 2018 to determine the best means of assessing the accuracy of administrative law judges' decisions. In addition, while SSA officials said that OARO was in the process of examining the overall efficiency and effectiveness of its quality assurance program, they did not provide information on any specific steps the agency has taken to reduce or better manage any unnecessary overlap among reviews to ensure strategic use of resources.

Full Report

GAO Contacts