Social Security Disability:
Additional Measures and Evaluation Needed to Enhance Accuracy and Consistency of Hearings Decisions
GAO-18-37: Published: Dec 7, 2017. Publicly Released: Jan 8, 2018.
- Highlights Page:
- Full Report:
- Accessible Version:
What GAO Found
Allowance rates—the rate at which Social Security Administration (SSA) administrative law judges allowed disability benefits to be paid when claimants appealed—varied across judges, even after holding constant certain characteristics of claimants, judges, hearing offices, and other factors that could otherwise explain differences in allowance rates. Specifically, GAO estimated that the allowance rate could vary by as much as 46 percentage points if different judges heard a typical claim (one that was average in all other factors GAO analyzed). SSA officials said that this level of variation is not surprising, given the complexity of appeals and judicial discretion. Nonetheless, the variation declined by 5 percentage points between fiscal years 2007 and 2015 (see figure), a change officials attributed to enhanced quality assurance efforts and training for judges. GAO also identified various factors that were associated with a greater chance that a claimant would be allowed benefits. In addition to characteristics related to disability criteria, such as the claimant's impairment and age, GAO found that claimants who had representatives, such as an attorney or family member, were allowed benefits at a rate nearly 3 times higher than those without representatives. Other factors did not appear related to allowance rates, such as the percentage of backlogged claims in a hearing office.
Estimated Allowance Rates across Social Security Administrative Law Judges for Typical Disability Claims, Fiscal Years 2007-2015
Note: The range is the difference in allowance rates for judges at the 95th and 5th percentiles. We also identify the 75th and 25th percentiles. A typical claim had average values on all other factors we analyzed.
SSA has various reviews to monitor the accuracy and consistency of hearings decisions by administrative law judges, but some of these reviews may overlap and SSA has not systematically evaluated them. Specifically, SSA conducts five types of quality assurance reviews of hearings decisions, several of which have similar goals and may look at similar claims. SSA has not evaluated the efficiency or effectiveness of these reviews, despite spending at least $11 million on them in fiscal year 2016. Moreover, the agency has struggled to sustain all of its quality reviews due to competing priorities—two of the five reviews were curtailed in 2016 because SSA reassigned staff to help expedite claims decisions. By evaluating which quality assurance reviews are most effective and efficient in improving accuracy and consistency, SSA would be better positioned to meet its goals within its resources.
Why GAO Did This Study
Individuals who do not agree with the initial decision on a claim for Social Security disability benefits can ultimately appeal the decision by requesting a hearing before one of SSA's approximately 1,500 administrative law judges. However, the rate at which these judges have allowed benefits has varied, raising questions about the reasons for this variation. GAO was asked to review aspects of SSA's oversight of judges' decisions.
This report examines (1) to what extent allowance rates vary across administrative law judges, and factors associated with this variation; and (2) the extent to which SSA has processes to monitor the accuracy and consistency of hearings decisions.
GAO developed a statistical model to analyze SSA data on adult disability decisions made by administrative law judges from fiscal years 2007 through 2015, the most current data available at the time of GAO's analysis; reviewed relevant federal laws, regulations, and agency documents; and interviewed SSA officials and chief judges in SSA's 10 regions, as well as officials from organizations representing judges, disability claimants, and claimant representatives.
What GAO Recommends
GAO is making two recommendations, including that SSA systematically evaluate its quality assurance reviews and take steps to reduce or better manage any unnecessary overlap among them. SSA concurred and plans to address them through a comprehensive assessment of its oversight.
For more information, contact Charles Jeszeck at (202) 512-7215 or email@example.com.
Recommendations for Executive Action
Comments: SSA agreed with this recommendation and stated that it would address it as part of a comprehensive assessment and refinement of its oversight roles and processes. As of February 2020, SSA officials reported that the agency developed a new review of hearing decisions to assess decisional quality on a national level. Officials said they implemented the review throughout fiscal year 2019 and will draft a report on the findings in fiscal year 2020. As they review results from this review, SSA officials said that they will assess whether existing quality review measures will provide suitable data for performance measures related to timeliness, consistency, and accuracy. Officials said they do not have a timeline for implementing this recommendation.
Recommendation: The Commissioner of SSA should develop a set of public performance measures, to include accuracy and consistency, as well as timeliness, of administrative law judges' (ALJ) disability decisions. SSA could consider whether existing quality review or monitoring efforts could provide suitable data for such measures. (Recommendation 1)
Agency Affected: Social Security Administration
Comments: SSA agreed with this recommendation and stated that it would address this recommendation as part of a comprehensive assessment and refinement of its oversight roles and processes. In March 2018, SSA noted that the Offices of Quality Review and Appellate Operations are now consolidated under the Office of Analytics, Review, and Oversight (OARO). According to SSA, OARO was created with the purposes of enhancing collaboration, eliminating unnecessary overlap, and enhancing the efficiency of reviews. As of February 2020, SSA officials said that they developed a new review of favorable and unfavorable hearing decisions to provide statistically valid results on the quality of decisions on appealed disability claims at the national level. Officials said they will use the results of this one-time review to help assess the utility of their other quality reviews. Successful completion of these efforts should help to ensure effective use of resources in overseeing the quality of decisions on appealed disability claims.
Recommendation: The Commissioner of SSA should systematically evaluate the efficiency and effectiveness of its quality assurance reviews and take steps to reduce or better manage any unnecessary overlap among them to ensure strategic use of resources. Such steps could include enhancing collaboration where reviews overlap or only conducting the reviews that are most efficient and effective in achieving agency goals for improving accuracy and consistency of ALJ disability decisions. (Recommendation 2)
Agency Affected: Social Security Administration