This is the accessible text file for GAO report number GAO-02-588 entitled 'District Of Columbia: Performance Report Reflects Progress and Opportunities for Improvement' which was released on April 15, 2002. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States General Accounting Office: GAO: Report to Congressional Subcommittees: District Of Columbia: Performance Report Reflects Progress and Opportunities for Improvement: GAO-02-588: Contents: Letter: Results in Brief: Background: Objectives, Scope, and Methodology: The 2001 Report Included Most of the District's Significant Activities: The District's 2001 Plan and Report Addressed a Consistent Set of Performance Goals: The Report Generally Adhered to Statutory Requirements: Steps Are Needed to Improve Future Performance Accountability Reports: Conclusions: Recommendations: Agency Comments and Our Evaluation: Appendixes: Appendix I: Expenditures by Agencies Included in the District's Fiscal Year 2001 Performance Accountability Report: Appendix II: Agencies and Budget Activities Not Included in the District's Fiscal Year 2001 Performance Accountability Report: Appendix III: Comments from the Government of the District of Columbia: Tables: Table 1: Instances in Which the District Has Not Reported the Level of Performance in its Fiscal Year 2001 Performance Accountability Report: Table 2: Actual Expenditure Totals by Agencies Included in the District's Fiscal Year 2001 Performance Accountability Report: Table 3: Ten Agencies the District Reported That It Did Not Include in Its Fiscal Year 2001 Performance Accountability Report and Related Expenditures: Table 4: Additional Budget Activities Not Included in the District's Fiscal Year 2001 Performance Accountability Report and Related Expenditure Levels: Abbreviations: ANC: Advisory Neighborhood Commissions: COG: Council of Governments: DCPS: District of Columbia Public Schools: GPRA: Government Performance and Results Act: OIG: Office of Inspector General: [End of section] United States General Accounting Office: Washington, D.C. 20548: April 15, 2002: Congressional Subcommittees: In view of continuing congressional interest in working with the District of Columbia's government to ensure that a sound performance management system is in place, this is the third year that we have evaluated the District's annual performance accountability report as mandated in law.[Footnote 1] This mandate provides a timely opportunity to examine the progress the District has made in developing and issuing its fiscal year 2001 performance accountability report and identify areas that the District and Congress may want to focus on to continue making progress in addressing the performance challenges facing our nation's capital. The District has continued to make progress in producing more useful performance documents and information that can help inform decision makers and improve the District's accountability and performance. The District of Columbia is required to submit to Congress a performance accountability plan with goals for the coming fiscal year and, after the end of the fiscal year, a performance accountability report on the extent to which the District achieved these goals. [Footnote 2] This requirement for the District government to issue performance accountability plans and reports is similar to the requirements for executive branch federal agencies under the Government Performance and Results Act of 1993 (GPRA).[Footnote 3] Our objectives in this report are to examine (1) the extent to which the District's performance accountability report includes all significant activities, (2) how well the District reports progress toward a consistent set of goals and explains any changes in the goals, (3) the extent to which the report adheres to the statutory requirements, and (4) areas for future improvement. Results in Brief: In the District of Columbia's FY 2001 Performance Accountability Report, the District provided a more complete picture of its performance and made progress in complying with statutory reporting requirements. The District has expanded the report's scope to include most of the District government's significant activities by covering 66 agencies-26 more than last year—that account for 83 percent of the District's total fiscal year 2001 general fund expenditures of $5.9 billion. The report presented a clearer picture of performance and enhanced accountability by using a consistent set of goals, enabling the District to measure and report progress toward the goals it had established in its 2001 performance plan.[Footnote 4] In addition, the report was timely and generally adhered to Federal Payment Reauthorization Act requirements. Specifically, it reported information on the level of performance achieved toward most of its goals, provided the titles of managers and their supervisors responsible for each of the goals, and described the status of certain court orders. The progress the District has made in its performance accountability reports over the last 3 years establishes a positive direction for future enhancements, and there are three key areas where improvements can be made. * Although the District included information on the terms and status of certain court orders, it should take steps to update this information by including information on the status of court orders it has not previously reported on, as well as those previously reported, that are applicable during the fiscal year. The act mandates that the District's report include a statement on the status of any court orders applicable during the fiscal year and the steps taken to comply with those orders. We believe that the District, in order to more fully comply with the statutory requirements, needs to take steps to ensure that the information provided in its performance accountability reports is accurate and up-to-date. We are recommending that the District establish objective criteria for the types of court orders for which it will provide specific compliance information in its performance accountability reports. These criteria could include factors such as the time, effort, and cost to respond to these court orders. If the District government has not acted to comply with the court orders it should include an explanation as to why no action was taken. In addition, the District should provide summary information on other applicable court orders in its future performance accountability reports. * One of the areas that the District highlighted in its 2001 report is its fiscal year 2003 performance based budgeting pilot. As our work with executive branch agencies has shown, linking expected performance, resources requested, and resources consumed helps promote performance management efforts and increases the need for reliable budget and financial data.[Footnote 5] Therefore, the District will need to verify and validate the data it uses to support the achievement of performance goals and present this information in its performance accountability reports. The District's plan to develop data collection standards is a good first step in this direction. We are recommending that the District government improve its future performance accountability reports by providing assurance that its program and financial data are timely, complete, accurate, useful, and consistent. The District's performance and accountability reports should also include discussions of strategies to address known data limitations. * Another area that the District should work toward is including performance goals and measures for all of its major activities, and to link related expenditure information to the agencies that are responsible for these expenditures. For example, including information on the funds agencies administer would provide a more complete picture of the resources targeted toward achieving an agency's goals and therefore help ensure transparency and accountability for the District's expenditures. The District's 2001 report states that it plans to include goals and measures for major areas, such as the public schools and the Child and Family Services Agency, in the fiscal year 2003 proposed budget and financial plan that it will submit to Congress. The District's plan to use these goals and measures as the basis for the fiscal year 2002 performance and accountability report should help provide a more complete picture of performance for decision makers and the District's citizens. Background: The Federal Payment Reauthorization Act of 1994 requires that the mayor of the District of Columbia submit to Congress a statement of measurable and objective performance goals for the significant activities of the District government (i.e., the performance accountability plan). After the end of the each fiscal year, the District is to report on its performance (i.e., the performance accountability report). The District's performance report is to include: * a statement of the actual level of performance achieved compared to each of the goals stated in the performance accountability plan for the year, * the title of the District of Columbia management employee most directly responsible for the achievement of each goal and the title of the employee's immediate supervisor or superior, and, * a statement of the status of any court orders applicable to the government of the District of Columbia during the year and the steps taken by the government to comply with such orders. The law also requires that GAO, in consultation with the director of the Office of Management and Budget, review and evaluate the District performance accountability report and submit it not later than April 15 to your committees. Our June 2001 report on the District's fiscal year 2000 performance accountability report included recommendations that the District (1) settle on a set of results-oriented goals that are more consistently reflected in its performance planning, reporting, and accountability efforts, (2) provide specific information in its performance reports for each goal that changed, including a description of how, when, and why the change occurred, and (3) adhere to the statutory requirement that all significant activities of the District government be addressed in subsequent performance accountability reports.[Footnote 6] Our review had determined that the District's fiscal year 2000 report was of limited usefulness because the District had introduced new plans, goals, and measures throughout the year, the goals and measures were in a state of flux due to these changes, and its report did not cover significant activities, such as the District's public schools, an activity that accounted for more than 15 percent of the District's budget. In response, the District concurred with our findings and acknowledged that additional work was needed to make the District's performance management system serve the needs of its citizens and Congress. The comments stated that the District planned, for example, to consolidate its goals and expand the coverage of its fiscal year 2001 report to more fully comply with its mandated reporting requirements. Objectives, Scope, and Methodology: We examined the progress the District has made in developing its performance accountability report and identified areas where improvements are needed. Specifically, the objectives of this report were to examine (1) the extent to which the District's performance accountability report includes its significant activities, (2) how well the District reports progress toward a consistent set of goals and explains any changes in the goals, (3) the extent to which the report adheres to the statutory requirements, and (4) areas for future improvement. To meet these objectives, we reviewed and analyzed the information presented in the District's fiscal year 2001 performance accountability report and interviewed key District officials. * To examine the extent to which the District's performance accountability report included significant activities, we compared the information in the 2001 performance and accountability report with budget information on actual expenditures presented in the District's budget.[Footnote 7] * To determine how well the District reported progress toward a consistent set of goals, we compared the report's goals with those contained in the District's fiscal year 2002 Proposed Budget and Financial Plan which served as the District's 2001 performance plan and then reviewed any changes. * To determine the extent to which the report adhered to the statutory requirements, we analyzed the information contained in the District's report in conjunction with the requirements contained in the Federal Payment Reauthorization Act of 1994. We also reviewed the performance contracts for the District's cabinet-level officials. * To identify areas for future improvement, we compared the fiscal year 2001 report with the District's fiscal year 2000 and 1999 performance accountability reports to identify baseline and trend information. We based our analysis on the information developed from work addressing our other objectives, recommendations from our June 8, 2001, report commenting on the District's fiscal year 2000 report, and our other recent work related to performance management issues. We conducted our work from December 2001 through April 2002 at the Office of the Mayor of the District of Columbia, Washington, D.C., in accordance with generally accepted government auditing standards. In accordance with requirements contained in P.L. 103-373, we consulted with a representative of the director of the Office of Management and Budget concerning our review. We did not verify the accuracy or reliability of the performance data included in the District's report, including information on the court orders in effect for fiscal year 2001. We provided a draft of this report to the mayor of the District of Columbia for review and comment. The deputy mayor/city administrator provided oral and written comments that are summarized at the end of this report, along with our response. The written comments are reprinted in their entirety in appendix III. The 2001 Report Included Most of the District's Significant Activities: The fiscal year 2001 performance accountability report includes most of the District's significant activities, providing performance information for 66 District agencies that represent 83 percent of the District's total expenditures of $5.9 billion during that year. The District included 26 additional agencies in this year's report, compared with 40 in its prior report for fiscal year 2000. Appendix I lists the 66 agencies included in the District's 2001 performance accountability report, along with the 2001 actual expenditures for each of these agencies. However, the absence of goals and measures related to educational activities remains the most significant gap. The District reports that it is continuing its efforts to include performance information on its significant activities in its performance accountability reports. For example, the 2001 performance accountability report notes that the District of Columbia Public Schools (DCPS) did not include performance goals or measures because they were in the early stages of a long-term strategic planning process initiated by the newly installed school board. DCPS accounted for about 14 percent of the District's fiscal year 2001 actual expenditures, and public charter schools, which also were not included, accounted for another 2 percent of the District's 2001 expenditures. The 2001 report states that in lieu of a formal performance accountability report for DCPS, the District included a copy of the Superintendent's testimony before the Subcommittee on the District of Columbia, Committee on Government Reform, U.S. House of Representatives. The District acknowledged that the inclusion of this information does not fully comply with the statutory requirement and set forth a plan to include DCPS performance goals and measures in the fiscal year 2003 proposed budget and financial plan that will serve as the basis for the DCPS performance accountability report for fiscal year 2002. The 2001 report lists another 10 agencies that were not included, primarily, according to the report, because they did not publish performance goals and measures in the fiscal year 2002 proposed budget. These 10 agencies accounted for about $330 million in fiscal year 2001 actual expenditures, or about 6 percent of the District's total fiscal year 2001 actual expenditures. These agencies included the Child and Family Services Agency, which was under receivership until June 15, 2001 (with fiscal year 2001 actual expenditures of $189 million) and public charter schools (with fiscal year 2001 expenditures of $137 million). Although it may not be appropriate to include agency performance information in some cases, the performance accountability report should provide a rationale for excluding them. For example, Advisory Neighborhood Commissions, according to the deputy mayor, have a wide range of agendas that cannot be captured in a single set of meaningful measures. Table 3 lists these 10 agencies and their fiscal year 2001 actual expenditures. In addition to these 10 agencies, the District also did not specifically include other areas constituting 11 percent of the District's fiscal year 2001 actual expenditures.[Footnote 8] In view of the District's interest in tying resources to results, the District could further improve its performance accountability reports by linking these budget activities as appropriate to the agencies that are responsible for these expenditures or provide a rationale for exclusion. For example, the Department of Employment Services administers the unemployment and disability funds (with fiscal year 2001 expenditures totaling about $32 million). Similarly, the Office of the Corporation Counsel administers the settlement and judgments fund, which was set up to settle claims and lawsuits and pay judgments in tort cases entered against the District (with fiscal year 2001 expenditures of about $26 million). Table 4 contains a list of these budget activities and fiscal year 2001 actual expenditures. The District's 2001 Plan and Report Addressed a Consistent Set of Performance Goals: The goals in the fiscal year 2001 performance accountability report were consistent with the goals in the District's 2001 performance plan. Using a consistent set of goals enhanced the understandability of the report by demonstrating how performance measured throughout the year contributed toward achieving the District's goals. The District also used clear criteria for rating performance on a five-point scale and reported that these ratings were included in the performance evaluations of cabinet agency directors who had performance contracts with the mayor. In addition, according to a District official, the District will be able to provide information on any future changes made to its performance goals through its new performance management database. The District has made substantial progress in improving its performance planning and reporting efforts by focusing on measuring progress toward achieving a consistent set of goals. In our June 2001 review of the District's 2000 performance accountability report, we had raised concerns that the District's performance management process was in flux, with goals changing continually throughout the year. Further, the District did not discuss the reasons for these changes. This year, the goals were consistent and the District provided some information about upcoming changes that could be anticipated in fiscal year 2002 goals. In addition, according to the 2001 report, the District has developed a performance measures database to allow it to document changes to individual goals and measures that are proposed in the agencies' fiscal year 2003 budget submissions. One of the District's enhancements to its 2001 performance accountability report was reporting on a five-point performance rating scale, as compared to the three-point performance rating scale it used in its fiscal year 2000 report. The five-point scale was designed to be consistent with the rating scale used in the District's Performance Management Program, under which management supervisory service, excepted service, and selected career service personnel develop individual performance plans against which they are evaluated at the end of the year. The five ratings are: (1) below expectations, (2) needs improvement, (3) meets expectations, (4) exceeds expectations, and (5) significantly exceeds expectations. According to the fiscal year 2001 performance accountability report, this scale was used to evaluate the performance of cabinet agency directors who held performance contracts with the mayor. It stated that 60-percent of each director's performance rating was based on the agency-specific goals included in the agency's performance accountability report, with the other 40-percent based on operational support requirements such as responsiveness to customers, risk management, and local business contracting. Our work has found that performance agreements can become an increasingly vital part of overall efforts to improve programmatic performance and better achieve results.[Footnote 9] We found that the use of results-oriented performance agreements: * strengthened alignment of results-oriented goals with daily operations, * fostered collaboration across organizational boundaries, * enhanced opportunities to discuss and routinely use performance information to make program improvements, * provided a results-oriented basis for individual accountability, and, * maintained continuity of program goals during leadership transitions. The Report Generally Adhered to Statutory Requirements: The District's fiscal year 2001 performance accountability report reflected improvement in adhering to the statutory requirements in the Federal Payment Reauthorization Act. The District's 2001 report was timely and included information on the level of performance achieved for most goals listed. It included the titles of the District management employee most directly responsible for the achievement of each of the goals and the title of that employee's immediate supervisor, as required by the statute. We also found that the names and titles on the performance contracts of the cabinet level officials we reviewed matched the names in the performance report as the immediate supervisor for all of the goals. Although the report contains information on certain court orders, the report could be improved by providing clearer and more complete information on the steps the District government has taken during the reporting year to comply with those orders and by including updated information on the court orders applicable to the District as required by the act. The Report Identified Performance Levels Achieved toward Most of the District's Goals: The District identified the level of performance achieved for most of the goals in its 2001 report. The report contains a total of 214 performance goals that are associated with the 66 agencies covered. Of these 214 performance goals, 201 goals (or 94 percent) include information on whether or not the goal was achieved, and only 13 did not include information on the level of performance. As shown in table 1, the 13 goals that did not include the level of performance were associated with eight agencies. For example, the District's State Education Office did not provide this information for four of its seven goals because the reports and information needed to achieve the goals had not been completed. Table 1: Instances in Which the District Has Not Reported the Level of Performance in its Fiscal Year 2001 Performance Accountability Report: Agency: Office of Property Management; Number of goals for which the level of performance achieved is not shown: 1. Agency: Board of Elections and Ethics; Number of goals for which the level of performance achieved is not shown: 1. Agency: Office of the Chief Financial Officer; Number of goals for which the level of performance achieved is not shown: 1. Agency: Board of Real Property Assessments and Appeals; Number of goals for which the level of performance achieved is not shown: 1. Agency: Office of the People's Counsel; Number of goals for which the level of performance achieved is not shown: 1. Agency: Citizen Complaint Review Board; Number of goals for which the level of performance achieved is not shown: 3. Agency: Office of the Chief Medical Examiner; Number of goals for which the level of performance achieved is not shown: 1. Agency: State Education Office; Number of goals for which the level of performance achieved is not shown: 4. Agency: Total; Number of goals for which the level of performance achieved is not shown: 13. Source: GAO analysis. [End of table] Information the District Included on Court Orders Has Limitations: Although the District's 2001 performance accountability report included some information on certain court orders imposed upon the District and the status of its compliance with those orders, the act calls for a statement of the status of any court orders applicable to the District of Columbia government during the year and the steps taken by the government to comply with such orders. The 2001 report contains information on the same 12 court orders involving civil actions against the District reported on for fiscal years 1999 and 2000. Among these 12 orders are 2 orders that the fiscal year 2001 report lists as no longer in effect in 2001. One of these court orders involved a receivership that terminated in May 2000. The other involved a maximum-security facility that closed at the end of January 2001. The 2001 report does not disclose whether or not any new court orders were imposed on the District during fiscal year 2001. The summaries that the District provides on the status of these court orders could be more informative if they contained clearer and more complete information on the steps taken by the District government to comply with the court orders. For example, according to the District's 2001 report, the case Nikita Petties v. DC relates to DCPS transportation services to special education students and the timely payment of tuition and related services to schools and providers. The report's summary on the status of this case states: "The School system has resumed most of the transportation responsibilities previously performed by a private contractor. A transportation Administrator with broad powers had been appointed to coordinate compliance with Court orders. He has completed his appointment and this position has been abolished." This summary does not provide a clear picture of what steps the school system is taking to comply with the requirements resulting from this court order. The act, however, calls for the District to report on the steps taken by the government to comply with such orders. Steps Are Needed to Improve Future Performance Accountability Reports: The District recognized in its 2001 performance and accountability report that its performance management system is a work-in-progress and stated that there are several fronts on which improvements can be made. In the spirit of building on the progress that the District has made in improving its performance accountability reports over the last 2 years, there are three key areas where we believe that improvements in future performance accountability reports are needed. First, the District needs to be more inclusive in reporting on court orders to more fully comply with the act's requirements. Second, as part of the District's emphasis on expanding its performance-based budgeting approach, the District needs to validate and verify the performance data it relies on to measure performance and assess progress, present this information in its performance accountability reports, and describe its strategies to address any known data limitations. Finally, the District needs to continue its efforts to include goals and measures for its major activities, and it should include related expenditure information to provide a more complete picture of the resources targeted toward achieving an agency's goals and therefore help to enhance transparency and accountability. The District Should Be More Inclusive in Reporting on Court Orders: Since this is the third year that the District has had to develop performance and accountability reports, the District has had sufficient time to determine how best to present information on the status of any court orders that are applicable to the District of Columbia during the fiscal year and the steps taken to comply with those orders. However, the District has continued to report on the same 12 court orders for fiscal years 1999, 2000, and 2001. By limiting its presentation to the same 12 court orders, the District's current report does not provide assurance that the information in its performance accountability report reflects court orders applicable during the fiscal year. Court orders have an important effect on the District's performance, as reflected by the chief financial officer's statement that the District's "unforeseen expenses are often driven by new legislative imperatives, court- ordered mandates, and suits and settlements."[Footnote 10] As another indication of their importance, 1 of the 11 general clauses in performance contracts with agency directors addresses the directors' responsiveness to court orders. To make future reports more useful, the District should include information on the status of court orders it has not previously reported on as well as those applicable during the fiscal year, including those that may have been vacated during the fiscal year and the steps taken to comply with them. The District should establish objective criteria for determining the types of court orders for which it will provide specific compliance information for future performance accountability reports, and it should consider ways to provide summary information related to any other court orders. In establishing objective criteria, the factors could include the cost, time, and magnitude of effort involved in complying with a court order. If the District government has not acted to comply with a court order it should include an explanation as to why no action was taken. The District's 2001 report contains a statement that "Following the publication of the FY 1999 Performance Accountability Report, GAO and the District's Office of Corporation Counsel agreed upon a list of 12 qualifying orders that should be included in the District's future Performance Accountability Reports." We did not intend to limit future reporting to only the 12 court orders first reported by the District for fiscal year 1999. We agreed on the list of 12 court orders because, at that time, the District had difficulty identifying all the court orders as required by statute. However, we believe that the District now has had time to develop criteria and a system for ensuring that updated and accurate information on the status of applicable court orders can be presented in its future performance accountability reports. Therefore, we are recommending that the mayor ensure that such steps are taken. The District Faces Challenges in Verifying and Validating Its Performance Information: The District has identified data collection standards as one of the areas it is working to improve. As with federal agencies, one of the biggest challenges the District faces is developing performance reports with reliable information to assess whether goals are being met or how performance can be improved. Data must be verified and validated to ensure the performance measures used are complete, accurate, consistent, and of sufficient quality to document performance and support decision making. Data verification and validation are key steps in assessing whether the measures are timely, reliable, and adequately represent actual performance. The District's performance and accountability reports should include information obtained from verification and validation efforts and should discuss strategies to address known data limitations. As reported in our June 2001 report on the District's fiscal year 2000 performance accountability report, the District had planned to issue performance review guidelines by the end of the summer of 2001. These guidelines were to be issued in response to an Inspector General's finding that the agencies did not maintain records and other supporting documentation for the accomplishments they reported regarding the fiscal year 2000 performance contracts. The District included information in its fiscal year 2003 budget instructions regarding performance measures emphasizing the importance of high quality data. Although not required for agencies' budget submissions, the guidance called for every agency to maintain, at a minimum, documentation on how it calculated each measure and the data source for each measure. In its 2001 performance accountability report, the District said it plans to address the development of data collection standards. The District plans to begin developing manuals to document how data for each performance measure is collected, how the measure is calculated, and who is responsible for collecting, analyzing, and reporting the data. A further step the District can consider is ensuring that these data are independently verified and validated. A District official acknowledged that validating and verifying performance information is something the District would deal with in the future. Credible performance information is essential for accurately assessing agencies' progress toward the achievement of their goals and pinpointing specific solutions to performance shortfalls. Agencies also need reliable information during their planning efforts to set realistic goals.[Footnote 11] Decision makers must have reliable and timely performance and financial information to ensure adequate accountability, manage for results, and make timely and well-informed judgments. Data limitations should also be documented and disclosed. Without reliable information on costs, for example, decision makers cannot effectively control and reduce costs, assess performance, and evaluate programs. Toward that end, the District must ensure that its new financial management system is effectively implemented to produce crucial financial information, such as the cost of services at the program level, on a timely and reliable basis.[Footnote 12] The District Should Enhance Its Efforts to Include Goals, Measures, and Related Expenditure Information: Although the District has made progress in presenting program performance goals and measures, the 2001 report did not contain goals and measures for all of its major activities and it did not include information on other areas that accounted for 11 percent of its annual expenditures. The District could enhance the transparency and accountability of its reports by continuing its efforts to ensure that agencies establish goals and measures that they will use to track performance during the year and by taking steps to ensure that agencies responsible for other budget activities (as shown in table 4) include these areas in their performance reports. The District did not include, for example, goals and measures for DCPS, although it did provide a copy of a testimony and stated that this was included, at least in part, to address concerns we had raised in our June 2001 report that the District's fiscal year 2000 performance accountability report did not cover DCPS.[Footnote 13] The District also did not include another 10 agencies in its 2001 performance accountability report and indicated that it is taking steps to include relevant goals and measures for some of these agencies in the next year's report. In addition to including goals and measures for the District's significant activities, the District should consider including related expenditure information to help ensure transparency and accountability. We found, for example, that the Department of Employment Services administers the unemployment and disability funds but this information was not linked in the District's 2001 performance accountability report. By linking expenditures to agencies that are responsible for them, the District can further improve its future performance accountability reports by providing a more complete picture of performance. Conclusions: The District, like several federal agencies, has found that it needed to change its performance goals—in some cases substantially—as it learned and gained experience during the early years of its performance measurement efforts. The District has continued to make progress in implementing a more results-oriented approach to management and accountability and issuing a timely and more complete performance accountability report. As we have seen with federal agencies, cultural transformations do not come quickly or easily, and improvements in the District's performance management system are still underway. Despite the important progress that has been made, opportunities exist for the District to strengthen its efforts as it moves forward. Recommendations: In order to more fully comply with the Federal Payment Reauthorization Act of 1994, which requires the District to provide a statement of the status of any court orders applicable to the government of the District of Columbia during the year and the steps taken by the government to comply with such orders, the mayor should ensure that the District establish objective criteria to determine the types of court orders for which it will provide specific compliance information for future performance accountability reports. In establishing objective criteria, the factors could include the cost, time, and magnitude of effort involved in complying with these court orders. If the District government has not acted to comply with the court orders it should include an explanation as to why no action was taken. In addition, the District should provide summary information related to other applicable court orders in its performance accountability reports. The Mayor of the District of Columbia should also ensure that future performance accountability reports: * include information on the extent to which its performance measures and data have been verified and validated and discuss strategies to address known data limitations, and; * include goals and performance measures for the District's significant activities and link related expenditure information to help ensure transparency and accountability. Agency Comments and Our Evaluation: On April 2, 2002, we provided a draft of our report to the mayor of the District of Columbia for his review. In response to our request, the deputy mayor/city administrator met with us on April 4 to discuss the draft and provided us with written comments on April 8. His written comments appear in appendix III. Overall, the deputy mayor stated that he agreed with the findings of the report and concurred with the report's recommendations. He stated that clear and meaningful performance reports are essential to communicate the extent to which the District has or has not met its goals and commitments to make those improvements. Further, he stated that the findings and recommendations in this report were consistent with the District government's intent of further improving its public reporting. The deputy mayor stated that the District would adopt our recommendation to develop objective criteria to determine the types of court orders for which it will provide specific compliance information for future performance accountability reports. Our recommendation also stated that the District should more fully comply with the statute by reporting information on the steps taken by the District government to comply with these orders. The deputy mayor said that they would provide such additional information although he stated that the statute does not specifically require that this information be provided. However, the Federal Payment Reauthorization Act of 1994 (Pi. 103-373) section 456(b)(C) requires that the District's performance accountability report contain "a statement of the status of any court orders applicable to the government of the District of Columbia during the year and the steps taken by the government to comply with such orders." We encourage the District government to comply with this requirement and concur with its comment that providing this information would make the report more informative and useful to Congress and the general public. The deputy mayor also concurred with our recommendation that the District's future performance reports include information on the extent to which its performance data have been validated and verified. The deputy mayor said that seven District agencies participating in the District's performance based budgeting pilot would be developing data collection manuals this summer. We encourage the District to proceed with this effort as well as to develop and report on strategies for addressing limitations in its data collections efforts. We have suggested in prior reports that when federal agencies have low quality or unavailable performance data, they should discuss how they plan to deal with such limitations in their performance plans and reports. Assessments of data quality do not lead to improved data for accountability and program management unless steps are taken to respond to the data limitations that are identified. In addition, alerting decisionmakers and stakeholders to significant data limitations allows them to judge the data's credibility for their intended use and to use the data in appropriate ways. Regarding the independent verification of performance data, the deputy mayor stated that the District's ability to secure independent verification of more than selected goals and measures is limited by the resources available to the District's Office of the Inspector General (OIG). He said that the OIG conducted spot-check audits of selected scorecard goals in the fiscal year 2000 performance accountability report and although these limited audits allowed the District to determine the validity of only those particular measures, this effort provided valuable observations and suggestions on how District agencies could improve its data collection practices. He also said that his office has discussed initiating additional spot-check audits of selected goals and measures with the OIG during fiscal year 2002. We agree that such spot checks would be useful. The knowledge that the OIG will be spot-checking some performance data during each fiscal year provides a good incentive to develop and use accurate, high-quality data. In our prior work, we have encouraged federal agencies to use a variety of strategies to verify and validate their performance information, depending upon the unique characteristics of their programs, stakeholder concerns, performance measures, and data resources.[Footnote 14] In addition to relying on inspector general assessments of data systems and performance measures, the District can use feedback from data users and external stakeholders to help ensure that measures are valid for their intended use. Other approaches can include taking steps to comply with quality standards established by professional organizations and/or using technical or peer review panels to ensure that performance data meet quality specifications. The District can also test the accuracy of its performance data by comparing it with other sources of similar data, such as data obtained from external studies, prior research, and program evaluations. The deputy mayor said that the District would be making efforts to include additional agencies and budget activities in future performance reports. We encourage the District to proceed with these efforts. Of the 10 agencies that were not included in the fiscal year 2001 performance report, the District has already included 3 agencies (the Office of Asian and Pacific Islander Affairs, the Child and Family Services Agency, and the Office of Veteran Affairs) in its fiscal year 2002 performance plan issued in March 2002. In addition, the deputy mayor stated that three additional agencies (the Office of the Secretary, the Housing Finance Agency, and the National Capital Revitalization Corporation) would be included in the District's consensus budget to be submitted to the Council of the District of Columbia in June 2002. With regard to the budget activities that were not included in the District's fiscal year 2001 performance report, the deputy mayor agreed that it would be appropriate to develop performance measures for six funds, such as settlements and judgments and administration of the disability compensation fund. The deputy mayor acknowledged that establishing performance measures for administering an additional six funds, such as the Public Benefit Corporation, would have been appropriate but they no longer exist. The deputy mayor said that the District of Columbia Retirement Board manages two funds that had relevant performance measures in the District's 2001 report. We noted, however, that these two retirement funds were not specifically identified in the 2001 performance accountability report. We are sending copies of this report to the Honorable Anthony A. Williams, Mayor of the District of Columbia. We will make copies available to others upon request. Key contributors to this report were Katherine Cunningham, Steven Lozano, Sylvia Shanks, and Susan Ragland. Please contact me or Ms. Ragland on (202) 512-6806 if you have any questions on the material in this report. Signed by: Patricia A. Dalton: Director, Strategic Issues: List of Congressional Subcommittees: The Honorable Mary L. Landrieu: Chair: The Honorable Mike DeWine: Ranking Minority Member: Subcommittee on the District of Columbia: Committee on Appropriations: United States Senate: The Honorable Richard J. Durbin: Chairman: The Honorable George V. Voinovich: Ranking Minority Member: Subcommittee on Oversight of Government Management, Restructuring, and the District of Columbia: Committee on Governmental Affairs: United States Senate: The Honorable Joe Knollenberg: Chairman: The Honorable Chaka Fattah: Ranking Minority Member: Subcommittee on the District of Columbia: Committee on Appropriations: U.S. House of Representatives: The Honorable Constance A. Morella: Chair: The Honorable Eleanor Holmes Norton: Ranking Minority Member: Subcommittee on the District of Columbia: Committee on Government Reform: U.S. House of Representatives: [End of section] Appendix I: Expenditures by Agencies Included in the District's Fiscal Year 2001 Performance Accountability Report: The District's fiscal year 2001 performance accountability report included 66 agencies accounting for 83 percent of the District's operating budget for fiscal year 2001. Table 2 lists these agencies and their fiscal year 2001 actual expenditures. Table 2: Actual Expenditure Totals by Agencies Included in the District's Fiscal Year 2001 Performance Accountability Report: Agency: 1. D.C. Council; Fiscal year 2001 actual expenditures (in thousands): $14,140. Agency: 2. D.C. Auditor; Fiscal year 2001 actual expenditures (in thousands): $1,274. Agency: 3. Office of the Mayor; Fiscal year 2001 actual expenditures (in thousands): $7,229. Agency: 4. Citywide Call Center; Fiscal year 2001 actual expenditures (in thousands): $1,420. Agency: 5. Office of the City Administrator; Fiscal year 2001 actual expenditures (in thousands): $16,561. Agency: 6. Office of Personnel; Fiscal year 2001 actual expenditures (in thousands): $10,831. Agency: 7. Human Resources Development; Fiscal year 2001 actual expenditures (in thousands): $3,673. Agency: 8. Office of Finance and Resource Management; Fiscal year 2001 actual expenditures (in thousands): $147,456. Agency: 9. Office of Contracting and Procurement; Fiscal year 2001 actual expenditures (in thousands): $12,836. Agency: 10. Office of Chief Technology Officer; Fiscal year 2001 actual expenditures (in thousands): $23,189. Agency: 11. Office of Property Management; Fiscal year 2001 actual expenditures (in thousands): $59,513. Agency: 12. Contract Appeals Board; Fiscal year 2001 actual expenditures (in thousands): $738. Agency: 13. Board of Elections and Ethics; Fiscal year 2001 actual expenditures (in thousands): $3,266. Agency: 14. Office of Campaign Finance; Fiscal year 2001 actual expenditures (in thousands): $1,260. Agency: 15. Public Employee Relations Board; Fiscal year 2001 actual expenditures (in thousands): $586. Agency: 16. Office of Employee Appeals; Fiscal year 2001 actual expenditures (in thousands): $1,400. Agency: 17. Office of the Corporation Counsel; Fiscal year 2001 actual expenditures (in thousands): $48,987. Agency: 18. Office of the Inspector General; Fiscal year 2001 actual expenditures (in thousands): $11,652. Agency: 19. Office of the Chief Financial Officer; Fiscal year 2001 actual expenditures (in thousands): $91,634. Agency: 20. Business Services and Economic Development; Fiscal year 2001 actual expenditures (in thousands): $22,703. Agency: 21. Office of Zoning; Fiscal year 2001 actual expenditures (in thousands): $1,813. Agency: 22. Department of Housing and Community Development; Fiscal year 2001 actual expenditures (in thousands): $53,900. Agency: 23. Department of Employment Services; Fiscal year 2001 actual expenditures (in thousands): $67,658. Agency: 24. Board of Appeals and Review; Fiscal year 2001 actual expenditures (in thousands): $240. Agency: 25. Board of Real Property Assessments and Appeals; Fiscal year 2001 actual expenditures (in thousands): $283. Agency: 26. Department of Consumer and Regulatory Affairs; Fiscal year 2001 actual expenditures (in thousands): $25,487. Agency: 27. Alcoholic Beverage Regulation Administration[A]; Fiscal year 2001 actual expenditures (in thousands): 0. Agency: 28. Department of Banking and Financial Institutions; Fiscal year 2001 actual expenditures (in thousands): $2,251. Agency: 29. Public Service Commission; Fiscal year 2001 actual expenditures (in thousands): $6,093. Agency: 30. Office of the People's Counsel; Fiscal year 2001 actual expenditures (in thousands): $3,052. Agency: 31. Department of Insurance and Securities Regulation; Fiscal year 2001 actual expenditures (in thousands): $7,518. Agency: 32. Office of Cable Television and Telecommunications; Fiscal year 2001 actual expenditures (in thousands): $8,190. Agency: 33. Metropolitan Police Department; Fiscal year 2001 actual expenditures (in thousands): $325,582. Agency: 34. Fire and Emergency Medical Services Department; Fiscal year 2001 actual expenditures (in thousands): $129,392. Agency: 35. Department of Corrections; Fiscal year 2001 actual expenditures (in thousands): $213,452. Agency: 36. D.C. National Guard; Fiscal year 2001 actual expenditures (in thousands): $2,044. Agency: 37. D.C. Emergency Management Agency; Fiscal year 2001 actual expenditures (in thousands): $9,806. Agency: 38. Commission on Judicial Disabilities and Tenure; Fiscal year 2001 actual expenditures (in thousands): $165. Agency: 39. Judicial Nomination Commission; Fiscal year 2001 actual expenditures (in thousands): $84. Agency: 40. Citizen Complaint Review Board; Fiscal year 2001 actual expenditures (in thousands): $1,117. Agency: 41. Advisory Commission on Sentencing; Fiscal year 2001 actual expenditures (in thousands): $392. Agency: 42. Office of the Chief Medical Examiner; Fiscal year 2001 actual expenditures (in thousands): $4,457. Agency: 43. State Education Office (includes D.C. Resident Tuition Support); Fiscal year 2001 actual expenditures (in thousands): $37,410. Agency: 44. University of the District of Columbia; Fiscal year 2001 actual expenditures (in thousands): $46,933. Agency: 45. Public Library; Fiscal year 2001 actual expenditures (in thousands): $27,229. Agency: 46. Commission on the Arts and Humanities; Fiscal year 2001 actual expenditures (in thousands): $2,700. Agency: 47. Department of Human Services; Fiscal year 2001 actual expenditures (in thousands): $390,775. Agency: 48. Department of Health; Fiscal year 2001 actual expenditures (in thousands): $1,112,948. Agency: 49. Department of Parks and Recreation; Fiscal year 2001 actual expenditures (in thousands): $38,577. Agency: 50. Office on Aging; Fiscal year 2001 actual expenditures (in thousands): $19,944. Agency: 51. Office of Human Rights; Fiscal year 2001 actual expenditures (in thousands): $1,237. Agency: 52. Office on Latino Affairs; Fiscal year 2001 actual expenditures (in thousands): $3,035. Agency: 53. D.C. Energy Office; Fiscal year 2001 actual expenditures (in thousands): $11,035. Agency: 54. Department of Public Works (includes School Transit Subsidy and Transportation); Fiscal year 2001 actual expenditures (in thousands): $116,667. Agency: 55. Department of Motor Vehicles; Fiscal year 2001 actual expenditures (in thousands): $27,204. Agency: 56. Taxicab Commission; Fiscal year 2001 actual expenditures (in thousands): $770. Agency: 57. Washington Metropolitan Area Transit Commission; Fiscal year 2001 actual expenditures (in thousands): $82. Agency: 58. Washington Metropolitan Area Transit Authority; Fiscal year 2001 actual expenditures (in thousands): $163,073. Agency: 59. Department of Mental Health; Fiscal year 2001 actual expenditures (in thousands): $289,293. Agency: 60.Water and Sewer Authority; Fiscal year 2001 actual expenditures (in thousands): $205,803. Agency: 61. Washington Aqueduct[B]; Fiscal year 2001 actual expenditures (in thousands): 0. Agency: 62. D.C. Lottery and Charitable Games Control Board; Fiscal year 2001 actual expenditures (in thousands): $227,089. Agency: 63. D.C. Sports and Entertainment Commission; Fiscal year 2001 actual expenditures (in thousands): $3,330. Agency: 64. District of Columbia Retirement Board; Fiscal year 2001 actual expenditures (in thousands): $6,748. Agency: 65. D.C. Public Schools; Fiscal year 2001 actual expenditures (in thousands): $860,024. Agency: 66. Washington Convention Center Authority; Fiscal year 2001 actual expenditures (in thousands): $24,874. Agency: Total; Fiscal year 2001 actual expenditures (in thousands): $4,960,104. [A] This is a new independent agency that had been part of the Department of Consumer and Regulatory Affairs in fiscal year 2001. [B] The Washington Aqueduct is a division of the U.S. Army Corps of Engineers and is considered a federal entity. Congress authorizes funding for the aqueduct but the agency submits its budget to the District. Source: Fiscal Year 2003 Proposed Budget and Financial Plan, Government of the District of Columbia: March 18, 2002 and District of Columbia's FY 2001 Performance Accountability Report, Government of the District of Columbia: March 2002. [End of table] [End of section] Appendix II: Agencies and Budget Activities Not Included in the District's Fiscal Year 2001 Performance Accountability Report: The District's fiscal year 2001 performance accountability report did not include 10 District agencies primarily because they did not publish performance goals in the District's 2001 performance plan. Table 3 lists these agencies and their fiscal year 2001 actual expenditures. Table 3: Ten Agencies the District Reported That It Did Not Include in Its Fiscal Year 2001 Performance Accountability Report and Related Expenditures: Agency: 1. Office of Asian and Pacific Islander Affairs[A]; Fiscal year 2001 actual expenditures (in thousands): 0. Agency: 2. National Capital Revitalization Corporation[A]; Fiscal year 2001 actual expenditures (in thousands): 0. Agency: 3. Office of Veterans Affairs[A]; Fiscal year 2001 actual expenditures (in thousands): 0. Agency: 4. Housing Finance Agency[B]; Fiscal year 2001 actual expenditures (in thousands): 0. Agency: 5. Metropolitan Washington Council of Governments (COG)[C]; Fiscal year 2001 actual expenditures (in thousands): $367. Agency: 6. Advisory Neighborhood Commissions (ANC)[C]; Fiscal year 2001 actual expenditures (in thousands): $593. Agency: 7. Storm Water[D]; Fiscal year 2001 actual expenditures (in thousands): $326. Agency: 8. Office of the Secretary; Fiscal year 2001 actual expenditures (in thousands): $2,352. Agency: 9. Public Charter Schools; Fiscal year 2001 actual expenditures (in thousands): $136,867. Agency: 10. Child and Family Services Agency; Fiscal year 2001 actual expenditures (in thousands): $189,858. Agency: Total; Fiscal year 2001 actual expenditures (in thousands): $330,363. [A] This is a new office so it did not expend funds in fiscal year 2001. [B] The Housing Finance Agency is funded through the issuance of bonds, fees, earned income, and grants, and had no general fund expenditures in fiscal year 2001. [C] These agencies may not be included in subsequent performance accountability reports because, according to the Deputy Mayor (1) the COG should be excluded because this involves membership payments to a regional organization, and (2) the ANC has a wide range of agendas that cannot be captured in a single set of meaningful measures. [D] According to the Deputy Mayor, storm water expenditures should be included in Water and Sewer Authority performance goals and measures. Source: Fiscal Year 2003 Proposed Budget and Financial Plan, Government of the District of Columbia: March 18, 2002 and District of Columbia's FY 2001 Performance Accountability Report, Government of the District of Columbia: March 2002. In addition to these 10 agencies, we identified several budget activities—accounting for 11 percent of the District's total fiscal year 2001 actual expenditures—that were not included in the fiscal year 2001 performance accountability report. Table 4 lists these activities and related fiscal year 2001 actual expenditures. Table 4: Additional Budget Activities Not Included in the District's Fiscal Year 2001 Performance Accountability Report and Related Expenditure Levels: Budget activity: Settlements and Judgments; Fiscal year 2001 actual expenditures (in thousands): $25,881. Budget activity: Unemployment Compensation Fund; Fiscal year 2001 actual expenditures (in thousands): $3,838. Budget activity: Disability Compensation Fund; Fiscal year 2001 actual expenditures (in thousands): $27,805. Budget activity: Children and Youth Investment Fund; Fiscal year 2001 actual expenditures (in thousands): $5,000. Budget activity: Brownfield Remediation; Fiscal year 2001 actual expenditures (in thousands): $983. Budget activity: Incentives for Adoption of Children; Fiscal year 2001 actual expenditures (in thousands): $57. Budget activity: Police Officers' and Firefighters' Retirement System[A]; Fiscal year 2001 actual expenditures (in thousands): $49,000. Budget activity: Teachers' Retirement Fund[A]; Fiscal year 2001 actual expenditures (in thousands): $200. Budget activity: Wilson Building[A]; Fiscal year 2001 actual expenditures (in thousands): $9,897. Budget activity: Inaugural Expenses; Fiscal year 2001 actual expenditures (in thousands): $5,755. Budget activity: Repayment of General Fund Deficit; Fiscal year 2001 actual expenditures (in thousands): $38,366. Budget activity: Repayment of Loans and Interest; Fiscal year 2001 actual expenditures (in thousands): $228,364. Budget activity: Certificates of Participation; Fiscal year 2001 actual expenditures (in thousands): $7,929. Budget activity: Section 103 Payment; Fiscal year 2001 actual expenditures (in thousands): $11,000. Budget activity: PBC Transition[B]; Fiscal year 2001 actual expenditures (in thousands): $91,599. Budget activity: Public Benefit Corporation[B]; Fiscal year 2001 actual expenditures (in thousands): $92,731. Budget activity: Public Benefit Corporation Subsidy[B]; Fiscal year 2001 actual expenditures (in thousands): $45,313. Budget activity: Corrections Medical Receiver[B]; Fiscal year 2001 actual expenditures (in thousands): $10,820. Budget activity: Correctional Industries[B]; Fiscal year 2001 actual expenditures (in thousands): $1,908. Budget activity: D.C. Financial Authority[B]; Fiscal year 2001 actual expenditures (in thousands): $3,140. Budget activity: Total; Fiscal year 2001 actual expenditures (in thousands): $659,586. [A] According to the deputy mayor, these three budget activities are managed by agencies that have relevant performance measures. There is no specific mention of these activities, however, in the District's 2001 performance accountability report. [B] According to the deputy mayor, these funds are no longer in existence. Source: Fiscal Year 2003 Proposed Budget and Financial Plan, Government of the District of Columbia: March 18, 2002. [End of section] Appendix III: Comments from the Government of the District of Columbia: Government Of The District Of Columbia: Executive Office Of The Mayor: Office of the City Administrator: The John A. Wilson Building: 1350 Pennsylvania Avenue, NW: Washington, DC 20004: April 8, 2002: Patricia Dalton: Director, Strategic Issues: U.S. General Accounting Office: 441 G Street, NW, Suite 2908: Washington, DC 20548: Dear Ms. Dalton: This letter is in response to the draft report[Footnote 15] by the General Accounting Office (GAO) on the level of compliance of the District of Columbia's FY 2001 Performance Accountability Report with the requirements for the Performance Accountability Report established in the Federal Payment Reauthorization Act of 1994 (PL 103-373). The GAO report notes that the District has made consistent and significant progress in improving its performance accountability reports over the last three years and we appreciate that acknowledgement. We also appreciate the suggestions for three areas in which the District could continue to improve its future performance accountability reports: 1. Providing more information on the status of court orders affecting District government activities; 2. Establishing data collection standards and practices to improve the validity and reliability of performance data in future reports; and; 3. Continuing to expand the range of agencies and activities reported on in performance accountability reports. In general, we agree with your findings and stand ready to implement your recommendations to the extent that current and future resources allow. However, as discussed in our meeting of April 4, 2002, it's important for the reader of the report to note that, while the format calls for repetition of GAO's findings and recommendations several times in the report, the District is now, for the first time, in general compliance with the requirements for performance management. Reporting on the Status of Court Orders: GAO's recommendation that the District develop criteria to determine whether recent court orders or civil actions should be added to the list of court orders that are reported on in each year's performance accountability report will be adopted. The proposed criteria provide the District with a starting point to improve this section of the FY 2002 Performance Accountability Report that will be submitted to Congress in March 2003. In addition, GAO recommends that the District provide more detail on the specific actions it takes in response to court orders, although the statute does not specifically require that information be provided. The District agrees that providing such additional information would make the District's performance accountability report more informative and useful to Congress, GAO and the general public, and will implement this recommendation in future reports. Data Validity and Reliability: Development of Data Collection Standards: The District concurs with GAO's finding that the District needs to develop data collection standards. Development of such standards is an integral part of the District's performance-based budgeting initiative, which is being implemented in a multi-phase process over the time period FY 2001-FY 2006. As stated in the District's FY 2001 Performance Accountability Report, the seven FY 2003 PBB agencies will develop data collection manuals in summer 2002 that will document how data for each performance measure is collected, how the measure is calculated and what staff are responsible for collecting, analyzing and reporting the data. While the Office of Budget and Planning will directly support only the seven PBB agencies in developing these manuals by next fall, the District will distribute the guidelines to the cabinet agencies engaging in performance-based budgeting for FY 2004 and to all non-cabinet agencies so they may begin to independently compile the information needed for such manuals. Independent Verification: GAO further notes that it is important that the District secure independent verification of its performance data. GAO's recommendation regarding independent verification is a viable option in a federal setting in which agencies have internal Inspectors General. However, the District's ability to secure independent verification of more than selected goals and measures is limited by the resources available to the District's Office of the Inspector General (OIG). As reported in the District's FY 2000 Performance Accountability Report, at our request the OIG conducted spot-check audits of selected Scorecard Goals. My office has discussed initiating additional spot-check audits of selected goals and measures with the Office of the Inspector General during FY 2002. While these limited audits allow the District to determine the validity of only those particular measures, the previous effort provided valuable observations and suggestions on how District agencies could improve their data collection practices, including the development of data collection manuals, discussed above. As a consequence of this timetable, the FY 2002 Performance Accountability Report that will be submitted to Congress in March 2003 will contain limited performance data that has been externally validated. We will continue to work toward having the majority of District agencies' performance data meet basic validation criteria. But the Districts ability to obtain independent verification of the validity and reliability of its performance data will likely remain limited to a sample of goals and measures from a rotating pool of selected agencies. Expanded Coverage of the District's Performance Accountability Reports: GAO addresses two distinct areas in which the District could increase coverage of "significant activities" in future performance accountability reports: 1) expanding the number of agencies covered and 2) expanding coverage to include a variety of special purpose funds. Inclusion of Additional Agencies: The first suggestion is that the District needs to expand coverage of District agencies that have not been included in this year's report. Specifically, GAO cites the Child and Family Services Agency (CFSA) and the D.C. Public Schools (DCPS) as agencies whose performance was not rated, in large part because neither included performance goals or measures in the FY 2002 Proposed Budget and Financial Plan submitted to Congress in June 2001. The District has developed performance goals and measures for CFSA that appear in the Mayor's FY 2003 Proposed Budget and Financial Plan, submitted to Council of the District of Columbia in March 2002. In addition, the District has requested that DCPS identify performance goals and measures from its Strategic Business Plan to include in the consensus budget, to be submitted to Council in June 2002. The District has also taken steps to develop FY 2002 goals and targets that will be published in the FY 2003 budget for the other agencies listed by GAO and those agencies will report on performance against their goals in the FY 2002 Performance Accountability Report (see Attachment I). Providing Performance Information of Special Purpose Funds: The second issue that GAO raises is that the District has not included performance information for a range of special funds that comprise approximately 11 percent of the District's FY 2001 budget[Footnote 16] The District acknowledges that the collective amount of these funds is a significant amount of the entire budget and that providing performance information in future performance accountability reports would present a more complete description of its activities. However, as discussed in our meeting of April 4, 2002, some of these funds are managed by other agencies that already have relevant performance measures. In these instances, establishing additional performance measures would not be appropriate. For example the DC Retirement Board (DCRB) manages the Teachers Retirement Fund and the Police and Firefighters Retirement System and has measures related to rates of return on its investments that are included in the FY 2001 Performance Accountability Report. On the other hand, while funds related to repayment of debt would not have dedicated performance measures, the District agrees it would be appropriate for the Office of the Chief Financial Officer to consider supplementing its existing measures related to debt management. These could include the District's overall bond rating, as well as factors that the major rating agencies consider when setting the bond rating, including, but not limited to the amount of outstanding debt per capita and timeliness of payments. Attachment II is the District's initial projection of which of these funds it would be appropriate to set performance goals and measures for in future years. The District will inform GAO of the status of that effort prior to submitting the FY 2003 budget to Congress. Conclusion: While the District has made great strides in improving the extent to which its performance reports comply with the statute, our central motivation in this effort remains improving the performance of all aspects of the District government to benefit our residents, commuters and visitors. Clear and meaningful performance reports are essential to communicate the extent to which the District has or has not met its goals and commitments to make those improvements. GAO's findings and recommendations are consistent with our own aims to further improve our public reporting. The collegial and constructive manner in which this year's evaluation team worked with the District prior to the submission of our FY 2001 report to Congress contributed to some of the improvements already incorporated and we appreciate GAO's ongoing support and assistance. Sincerely, Signed by: John A. Koskinen: Attachments: [End of letter] Attachment I: Agencies Not Included in FY 2001 Performance Accountability Report: Agency: Office of Asian and Pacific Islander Affairs; Reason for Exclusion: No funding in FY 2001; Status of Goals and Measures Development: Included in March budget. Agency: Office of the Secretary; Reason for Exclusion: Did not develop goals and measures in prior years; Status of Goals and Measures Development: To be included in June budget. Agency: Metropolitan Washington Council of Governments; Reason for Exclusion: Membership payment to regional organization; Status of Goals and Measures Development: Will not develop goals and measures. Agency: Public Charter Schools; Reason for Exclusion: Did not develop goals and measures in prior years; Status of Goals and Measures Development: To Be Determined[A]. Agency: Housing Finance Agency; Reason for Exclusion: Did not develop goals and measures in prior years; Status of Goals and Measures Development: Will request for inclusion in June budget. Agency: Advisory Neighborhood Commissions; Reason for Exclusion: ANCs have wide range of agendas that cannot be captured in a single set of meaningful measures; Status of Goals and Measures Development: Will not develop goals and measures. Agency: National Capital Revitalization Corporation; Reason for Exclusion: No funding in FY 2001; Status of Goals and Measures Development: Will request for inclusion in June budget. Agency: Child and Family Services Agency; Reason for Exclusion: CFSA in transition from receivership at time of FY 2002 Budget Development; Status of Goals and Measures Development: Included in March budget. Agency: Storm Water; Reason for Exclusion: Under management of WASA, which is included in FY01 report; Status of Goals and Measures Development: Will not develop separate goals and measures. Agency: Veterans Affairs; Reason for Exclusion: No funding in FY 2001; Status of Goals and Measures Development: Included in March budget. [A] As noted in the text of this letter, the District is requesting performance goals and measures from DC Public Schools for inclusion in the FY 2003 budget. The District will propose that Public Charter Schools review those measures when complete and consider adopting similar measures, especially where related to student performance. As a result, Charter School Measures will not be included in the FY 2003 budget, but the District will seek to provide performance data in the FY 2002 performance accountability report to establish a baseline for 2003 and beyond. [End of Attachment I] Attachment II: Special Purpose Funds for which Data Will Be Provided in Future Performance Reports: Special Purpose Fund: 1. Teachers Retirement Fund; Appropriateness of Performance Measures: Captured in DC Retirement Board. Special Purpose Fund: 2. Police Officers & Firefighters Retirement System; Appropriateness of Performance Measures: Captured in DC Retirement Board. Special Purpose Fund: 3. Settlements & Judgments; Appropriateness of Performance Measures: Goals and measures are appropriate to set for future years. Special Purpose Fund: 4. Unemployment Compensation Fund; Appropriateness of Performance Measures: Goals and measures are appropriate to set for future years. Special Purpose Fund: 5. Disability Compensation Fund; Appropriateness of Performance Measures: Goals and measures are appropriate to set for future years. Special Purpose Fund: 6. Children & Youth Investment Fund; Appropriateness of Performance Measures: Goals and measures are appropriate to set for future years. Special Purpose Fund: 7. Brownfield Remediation; Appropriateness of Performance Measures: Goals and measures are appropriate to set for future years. Special Purpose Fund: 8. Incentives for Adoption for Children; Appropriateness of Performance Measures: Goals and measures are appropriate to set for future years. Special Purpose Fund: 9. Wilson Building; Appropriateness of Performance Measures: Goals and measures related to Wilson Building Renovations and Occupancy included in Office of Property Management (AMO). Special Purpose Fund: 10. Section 103 Payment; Appropriateness of Performance Measures: Goals and measures not appropriate for payment of this type. Special Purpose Fund: 11. Repayment of Loans & Interest; Appropriateness of Performance Measures: Goals and measures not appropriate for payment of this type. Special Purpose Fund: 12. Certificates of Participation; Appropriateness of Performance Measures: Goals and measures not appropriate for payment of this type. Special Purpose Fund: 13. Repayment of General Fund Deficit; Appropriateness of Performance Measures: Goals and measures not appropriate for payment of this type. Special Purpose Fund: 14. Public Benefit Corporation Subsidy; Appropriateness of Performance Measures: Measures would have been appropriate but this specific fund is no longer in existence. Special Purpose Fund: 15. PBC Transition; Appropriateness of Performance Measures: Measures would have been appropriate but this specific fund is no longer in existence. Special Purpose Fund: 16. Public Benefit Corporation; Appropriateness of Performance Measures: Measures would have been appropriate but this specific fund is no longer in existence. Special Purpose Fund: 17. Corrections Medical Receiver; Appropriateness of Performance Measures: Measures would have been appropriate but this specific fund is no longer in existence. Special Purpose Fund: 18. DC Financial Authority; Appropriateness of Performance Measures: Measures would have been appropriate but this specific fund is no longer in existence. Special Purpose Fund: 19. Correctional Industries; Appropriateness of Performance Measures: Measures would have been appropriate but this specific fund is no longer in existence. Special Purpose Fund: 20. Inaugural Expenses; Appropriateness of Performance Measures: Once every four years; not likely to set performance measures beyond financial controls. [End of Attachment II] [End of section] Footnotes: [1] U.S. General Accounting Office, District of Columbia: Comments on Fiscal Year 2000 Performance Report, [hyperlink, http://www.gao.gov/products/GAO-01-804] (Washington, D.C. June 8, 2001); and U.S. General Accounting Office, District of Columbia Government: Performance Report's Adherence to Statutory Requirements, [hyperlink, http://www.gao.gov/products/GAO/GGD-00-107] (Washington, D.C.: April 14, 2000). [2] These requirements are contained in the Federal Payment Reauthorization Act of 1994, Public Law No. 103-373. [3] U.S. General Accounting Office, Managing For Results: Using GPRA to Help Congressional Decisionmaking and Strengthen Oversight, [hyperlink, http://www.gao.gov/products/GAO/T-GGD-00-95] (Washington, D.C.: Mar. 22, 2000). [4] The 2001 performance plan was contained in the District's Fiscal Year 2002 Proposed Budget and Financial Plan. [5] U.S. General Accounting Office, Managing for Results: Agency Progress in Linking Performance Plans With Budgets and Financial Statements, [hyperlink, http://www.gao.gov/products/GAO-02-236] (Washington, D.C.: Jan. 4, 2002). [6] 13.L. 103-373, Sec. 456 (2). [7] Fiscal Year 2002 Proposed Budget and Financial Plan, Government of the District of Columbia: June 4, 2001 and Fiscal Year 2003 Proposed Budget and Financial Plan, Government of the District of Columbia: Mar. 18, 2002. [8] Because the District of Columbia Retirement Board administers the Police Officers' and Firefighters' Retirement System and the Teachers' Retirement Fund, District officials contend that they were included in the fiscal year 2001 performance accountability report as part of the Board's performance measures. However, we found that there was no mention of the Police Officers' and Firefighters' Retirement System or the Teachers' Retirement Fund in the District's fiscal year 2001 performance accountability report. [9] U.S. General Accounting Office, Managing for Results: Emerging Benefits From Selected Agencies' Use of Performance Agreements, [hyperlink, http://www.gao.gov/products/GAO-01-115] (Washington, D.C.: Oct. 30, 2000). [10] Testimony of Natwar M. Gandhi, Chief Financial Officer, Before the Committee on Finance and Revenue, Council of the District of Columbia (Washington, D.C.: Feb. 28, 2002). [11] U.S. General Accounting Office, District of Columbia Government: Progress and Challenges in Performance Management, [hyperlink, http://www.gao.gov/products/GAO-01-96T] (Washington, D.C.: Oct. 3, 2000). [12] U.S. General Accounting Office, District of Columbia: Weaknesses in Financial Management System Implementation, [hyperlink, http://www.gao.gov/products/GAO-01-489] (Washington, D.C.: April 30, 2001), and U.S. General Accounting Office, District of Columbia: Observations on Management Issues, [hyperlink, http://www.gao.gov/products/GAO-01-743T] (Washington, D.C.: May 16, 2001). [13] [hyperlink, http://www.gao.gov/products/GAO-01-804]. [14] U.S. General Accounting Office, Performance Plans: Selected Approaches for Verification and Validation of Agency Performance Information, [hyperlink, http://www.gao.gov/products/GAO/GGD-99-139] (Washington, D.C.: July 30, 1999). [15] Performance Report Reflects Progress and Opportunities for Improvement," (GAO-02-588). [16] GAO-02-588, Table 11.2, p. 28. Note: This is table 4 on page 26 of this report. [End of section] GAO’s Mission: The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site [hyperlink, http://www.gao.gov] contains abstracts and full text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to [hyperlink, http://www.gao.gov] and select “Subscribe to daily E-mail alert for newly released products” under the GAO Reports heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office: 441 G Street NW, Room LM: Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov: (202) 512-4800: U.S. General Accounting Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: