General Accuracy of District and Appellate Judgeship Case-Related Workload Measures
GAO-03-937T: Published: Jun 24, 2003. Publicly Released: Jun 24, 2003.
GAO appeared before the Subcommittee on Courts, the Internet, and Intellectual Property, House Committee on the Judiciary to discuss the results of our review and assessment of case-related workload measures for district court and courts of appeals judges. Biennially, the Judicial Conference of the United States, the federal judiciary's principal policymaking body, assesses the judiciary's needs for additional judgeships. If the Conference determines that additional judgeships are needed, it transmits a request to Congress identifying the number, type (courts of appeals, district, or bankruptcy), and location of the judgeships it is requesting. In assessing the need for additional district and appellate court judgeships, the Judicial Conference considers a variety of information, including responses to its biennial survey of individual courts, temporary increases or decreases in case filings, and other factors specific to an individual court. However, the Conference's analysis begins with the quantitative case-related workload measures it has adopted for the district courts and courts of appeals--weighted case filings and adjusted case filings, respectively. These two measures recognize, to different degrees, that the time demands on judges are largely a function of both the number and complexity of the cases on their dockets. Some types of cases may demand relatively little time and others may require many hours of work.
GAO found that the district court weighted case filings, as approved in 1993, appear to be a reasonably accurate measure of the average time demands that a specific number and mix of cases filed in a district court could be expected to place on the district judges in that district. The methodology used to develop the case weights was based on a valid sampling procedure, developed weights based on actual case-related time recorded by judges from case filing to disposition, and included a measure (standard errors) of the statistical confidence in the final weight for each weighted case type. The case weights, however, are about 10 years old, and the data on which the weights are based are as much as 15 years old. Changes since 1993, such as the characteristics of cases filed in federal district courts and changes in case management practices, may have affected whether the 1993 case weights continue to be a reasonably accurate measure of the average time burden on district court judges resulting from a specific volume and mix of cases. The Judicial Conference's Subcommittee on Judicial Statistics has approved a research design for updating the current case weights, and we have some concerns about that design. The design would include limited data on the time judges actually spend on specific types of cases. The proposed design would not include collecting actual data on the non-courtroom time that judges spend on different types of cases. Estimates of the non-courtroom time required for specific types of cases would be based on estimates derived from the structured, guided discussions of about 100 experienced judges meeting in 12 separate groups (one for each geographic circuit). The accuracy of case weights developed on such consensus data cannot be assessed using standard statistical methods, such as the calculation of standard errors. Thus, it would not be possible to objectively, statistically assess how accurate the new case weights are. Adjusted case filings, the principal quantitative measure used to assess the case-related workload of courts of appeals judges, are based on available data from standard statistical reports from the courts of appeals. The measure is not based on any empirical data about the judge time required by different types of cases in the courts of appeals. The measure essentially assumes that all cases filed in the courts of appeals, with the exception of pro se cases--those in which one or both parties are not represented by an attorney--require the same amount of judge time. On the basis of the documentation we reviewed, there is no empirical basis on which to assess the accuracy of adjusted filings as a measure of case-related workload for courts of appeals judges. Whether the district court case weights are a reasonably accurate measure of district judge case-related workload is dependent upon two variables: (1) the accuracy of the case weights themselves and (2) the accuracy of classifying cases filed in district courts by the case type used for the case weights. If case filings are inaccurately identified by case type, then the weights are inaccurately calculated.