This is the accessible text file for GAO report number GAO/OIG-10-1 entitled 'Testimony Measure: Verification of Performance Data Could Be Improved' which was released on November 18, 2009. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Office of the Inspector General: United States Government Accountability Office: GAO: November 2009: Testimony Measure: Verification of Performance Data Could Be Improved: GAO/OIG-10-1: GAO/OIG: Memorandum: Date: November 18, 2009: To: Acting Comptroller General Gene L. Dodaro: From: [Signed by] Inspector General Frances Garcia: Subject: Testimony Measure: Verification of Performance Data Could Be Improved: As part of the Office of Inspector General’s periodic review of GAO’s performance measures, we examined the reliability of the fiscal year 2008 performance data for the agency’s testimony measure. GAO’s testimony measure is defined as the number of hearings where one or more GAO testimonies were provided. In our random sample of 54 of the 304 hearings reported in the agency’s fiscal year 2008 Performance and Accountability Report,[Footnote 1] we found errors that indicated the number of hearings was slightly less than the 304 reported.[Footnote 2] Based on the results of our work, GAO’s Office of Congressional Relations reviewed the fiscal year 2008 performance data and now states that the number of hearings is 298. The primary reason for the errors was that 5 hearings—at which two GAO officials provided separate testimonies—were counted twice for a total of 10 hearings. We identified several factors that contributed to these errors, such as the need to update existing procedures to verify the accuracy of testimony performance measure data. We are recommending that GAO’s Office of Congressional Relations revise its procedures to better ensure the accuracy of GAO’s performance data. Congressional Relations managers state they have corrected the errors in fiscal year 2008 data and are updating their procedures. In a prior review of this measure, we found no errors with the fiscal year 2006 data. Background: GAO’s mission is to support Congress in meeting its constitutional responsibilities and to help improve the performance and ensure the accountability of the federal government for the benefit of the American people. To accomplish its mission, GAO provides objective and reliable information and informed analysis to Congress, federal agencies, and the public. To help monitor its performance in serving Congress, GAO uses the testimony performance measure.[Footnote 3] On occasion, more than one GAO official may testify at the same hearing. Performance data for this measure comes from GAO’s Congressional Hearing System, which is managed by the Office of Congressional Relations. Data is entered into the system by administrative staff from (1) organizational units (generally referred to as mission teams) whose officials are testifying and (2) the Office of Congressional Relations under limited circumstances. The Congressional Hearing System captures information, for example, about the date and topic of the hearing, the name of the committee or subcommittee holding the hearing, and the name of the GAO witness. The Government Performance and Results Act of 1993 (GPRA) requires federal agencies to develop strategic plans with long-term, outcome- oriented goals and objectives; annual goals linked to the long-term goals; and annual reports on the results achieved.[Footnote 4] As a legislative branch agency, GAO is not required to comply with GPRA but generally does follow the intent of this law. To follow the intent of GPRA, GAO uses a strategic planning and management process based on strategic goals and objectives. To help monitor its performance, the agency has developed various performance measures, including the testimony measure. Improved Procedures Are Needed to Ensure Accurate Performance Data for Testimony Measure: We identified a total of eight errors in GAO’s fiscal year 2008 testimony performance data. From our random sample of 54 hearings, we found six errors. Five of the six errors resulted from 5 hearings—at which two GAO officials provided separate testimonies—being counted twice (in this case, as 10 hearings) in the Congressional Hearing System. The sixth error from our sample was the result of a planned testimony being issued as a statement for the record and the Congressional Hearing System’s data not being updated to reflect this change. Statements for the record are not to be included in the testimony measure.[Footnote 5] Separate from our sample, we learned about two other errors. The first error involved a January 2008 testimony that was not entered into the agency’s hearings database in a timely manner and thus was not included in the total number of hearings reported for fiscal year 2008. The second error was identified by the Office of Congressional Relations’ review of fiscal year 2008 testimony performance data. The error involved a January 2008 hearing, where two GAO officials provided separate testimonies, that was inadvertently entered twice into the Congressional Hearing System—and thus was counted as two hearings. Office of Congressional Relations managers state they have corrected the eight errors. Our work shows that several factors contributed to inaccurate fiscal year 2008 performance data for GAO’s testimony measure. First, while the Office of Congressional Relations has internal procedures that in part address maintaining the Congressional Hearing System database and producing related hearing reports, these procedures do not specifically mention the testimony measure, its definition, or the performance data.[Footnote 6] The procedures also do not include specific steps for checking and testing the performance data before it is reported in GAO’s annual performance report. For example, our work compared Congressional Hearing System data to a chronological listing of testimonies in GAO’s Publications Database, which is a separate information system that includes data on all GAO publications, such as reports, testimonies, and statements for the record. This data quality check helped us identify that five of the hearings in our sample had more than one GAO testimony. GAO’s guidance for assessing performance data emphasizes periodic testing of the data to verify its accuracy. [Footnote 7] Office of Congressional Relations managers agree that their procedures should be updated—a process that has already begun—in order to clearly specify how to enter data into the Congressional Hearing System when GAO officials deliver more than one testimony at the same hearing. Second, while Office of Congressional Relations procedures stress the importance of updating Congressional Hearing System data when changes occur, one of the errors we identified happened because data had not been updated when a planned testimony was issued as a statement for the record. The unit administrative staff member responsible for updating the data admitted not doing so because she thought statements for the record would be removed from the database by someone else before the performance data would be published. In addition, the administrative staff member did not inform the Office of Congressional Relations of the change, as required. This error also was not identified by the Congressional Relations administrative staff member, even though the procedures direct her to check unit-entered data for accuracy and completeness. Congressional Relations managers told us that the error has now been corrected. Third, unit administrative staff who enter data in the Congressional Hearing System told us that it is difficult for them to know when more than one GAO testimony will be delivered at a hearing. Typically, separate testimonies for a hearing are delivered by officials that work in different organizational units. Therefore, unit staff stated they are dependent on someone else—such as the official who is testifying for their unit or Office of Congressional Relations staff—to inform them if other testimonies will be delivered at the same hearing. In our sample, all five hearings that had two separate GAO testimonies were not correctly identified. Office of Congressional Relations managers informed us that the agency has many ways of knowing that two GAO testimonies will be delivered at the same hearing. For example, the Office of Congressional Relations receives letters of invitations for GAO witnesses to testify at hearings. Finally, a report used to check GAO hearing data was not effective in identifying data errors related to hearings being counted twice. Specifically, the Congressional Hearing System’s end-of-year report is not designed to readily identify hearings where two GAO officials provided separate testimonies, because it lists all testimonies and statements for the record according to which agency strategic goal they help achieve.[Footnote 8] As a result, separate GAO witnesses who represent different goals at the same hearing can be listed in different parts of the report. For example, the testimony of one witness at an April 24, 2008, hearing, is listed under goal one on page 6 of the fiscal year 2008 hearings report, while the testimony of a second GAO witness at this hearing is listed under goal three on page 26 of the report. To reduce the risk of miscounting hearings, staff from the Office of Congressional Relations told us that they have requested the development of an additional report from the Congressional Hearing System. This new report will list testimonies and statements for the record by the date of the hearing rather than by agency strategic goals. The intent is to provide an additional data quality check by manually reviewing the report periodically to identify hearings with more than one GAO witness. Office of Congressional Relations managers believe—and we agree—that this new report will help better ensure the accuracy of testimony performance data. Conclusion: We believe the actions initiated by the Office of Congressional Relations—updating its procedures and developing a new report from the Congressional Hearing System—will help reduce the risk of future errors in GAO testimony performance data. As procedures are updated, we believe consideration should be given to adding specific steps that address the three types of data errors identified in our review: (1) the miscounting of hearings where more than one GAO testimony is presented, (2) not properly updating hearing data when a planned testimony is not presented, and (3) untimely posting of data. In addition, we believe that steps are needed to verify the accuracy of testimony performance data prior to their publication in GAO’s annual performance report. Recommendation: We recommend that the Acting Comptroller General direct the Managing Director of the Office of Congressional Relations to include in its revised procedures specific steps for addressing the type of errors identified in our review and for verifying the accuracy of testimony performance data prior to their publication. Agency Comments: The Inspector General provided GAO with a draft of this report for review and comment. GAO agreed with our recommendation. The agency also provided technical comments that we incorporated, as appropriate. Actions taken in response to our recommendations are expected to be reported to my office within 60 days. We are sending copies of this report to the other members of GAO’s Executive Committee (the Chief Administrative Officer and the Acting General Counsel), the Managing Director of the Office of Congressional Relations, and GAO’s Audit Advisory Committee. The report also is available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-5748 or garciaf@gao.gov. Contact points for GAO’ s Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report were Cathy Helm (Deputy Inspector General) and Kurt Kershow. Attachments: [End of section] Attachment I: Methodology: To determine if GAO has reliable data for its testimony performance measure, we used a random sample of 54 of the 304 hearings reported for GAO’s fiscal year 2008 testimony performance data. Performance data for this measure come from the Office of Congressional Relations’ Congressional Hearing System. For the 54 hearings, we examined the system’s data in several ways to assess the data’s accuracy. First, we compared Congressional Hearing System data to information on congressional committee and subcommittee Web sites to verify * the date of the hearing, * the topic of the hearing, * the name of the committee or subcommittee holding the hearing, and, * the GAO witness providing the testimony. Second, we reviewed the Congressional Hearing System’s report of fiscal year 2008 data to see if any hearings in our sample had more than one GAO witness and if these hearings were accurately counted. Third, as an additional check to see if hearings in our sample were accurately counted, we compared Congressional Hearing System data to a chronological listing of fiscal year 2008 testimonies in GAO’s Publications Database. For errors we identified in fiscal year 2008 performance data, we conducted interviews with and obtained related documents from administrative staff and managers responsible for the accuracy of data for the hearings in our sample in order to understand why the errors occurred. In addition, we interviewed managers and staff in GAO’s Office of Quality and Continuous Improvement, which has overall responsibility for the annual performance report, to determine whether they verify the accuracy of testimony performance data. Finally, we assessed the effectiveness of internal controls for ensuring the reliability of the performance data. We conducted this performance audit from March 2009 to November 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Attachment II: Comments from the Office of Congressional Relations: GA0: Memorandum: Date: November 12, 2009: To: Inspector General — Frances Garcia: From: [Signed by] Managing Director of Congressional Relations - Ralph Dawn: Subject: Response to GAO Inspector General's Final Report: Testimony Measure: Verification of Performance Data Could Be Improved: Thank you for the opportunity to review the draft report evaluating the testimony measure. We concur with the recommendation in the report and are revising our internal procedures accordingly. Specifically, GAO's Congressional Relations Office will update its procedures to require teams to immediately notify them of changes to a hearing or statement of record. Additionally, we have developed a report from the Congressional Hearing System on GAO testimonies and hearings. Congressional Relations staff will use the report on an ongoing basis and compare it with testimonies posted on GAO's web site to ensure that: (1) the testimony was delivered; and, (2) hearings with more than one GAO witness are not counted twice. Congressional Relations has already implemented this manual check to verify the accuracy of data for fiscal year 2009 and will continue to do so in the future, including prior to the publication of GAO's annual Performance and Accountability Report. In closing, we believe that having updated procedures in place will safeguard against, the three types of data errors identified in your report and further GAO's mission. We will continue to keep you informed of our progress. cc: Gene Dodaro, Acting Comptroller General: Sallyanne Harper, Chief Administrative Officer: Lynn Gibson, Acting General Counsel: Tim Bowling, Chief Quality Officer: [End of section] Footnotes: [1] GAO, Performance and Accountability Report: Fiscal Year 2008, [hyperlink, http://www.gao.gov/products/GAO-091-SP] (Washington, D.C.: Nov. 14, 2008). [2] See attachment I for a discussion of our methodology. [3] GAO also uses the timeliness performance measure to help the agency assess its service to Congress. [4] GPRA, Pub. L. No. 103-62. [5] Statements for the record occur when GAO officials do not testify at a congressional hearing and instead submit statements for inclusion in the congressional record. [6] Office of Congressional Relations, Congressional Hearing Support Process (For Internal Congressional Relations Use Only). [7] GAO, The Results Act: An Evaluator’s Guide to Assessing Agency Annual Performance Plans, [hyperlink, http://www.gao.gov/products/GAO/GGD-10.1.20] (Washington, D.C.: April 1998). [8] GAO has four strategic goals, they are to: provide timely, quality service to the Congress and the federal government to (1) address current and emerging challenges to the well-being and financial security of the American people, (2) respond to changing security threats and the challenges of global interdependence, (3) help transform the federal government’s role and how it does business to meet 21st century challenges, and (4) maximize the value of GAO by being a model federal agency and a world-class professional services organization. [End of section] Reporting Fraud, Waste, and Abuse in GAO’s Internal Operations: To report fraud, waste, and abuse in GAO’s internal operations, do one of the following. (You may do so anonymously.) * Call toll-free (866) 680-7963 to speak with a hotline specialist, available 24 hours a day, 7 days a week. * Send an e-mail to OIGHotline@gao.gov. * Send a fax to the OIG Fraud, Waste, and Abuse Hotline at (202) 512- 8361. * Write to: GAO Office of Inspector General: 441 G Street NW, Room 1808: Washington, DC 20548: Obtaining Copies of GAO/OIG Reports and Testimony: To obtain copies of OIG reports and testimony, go to GAO’s Web site: [hyperlink, www.gao.gov/about/workforce/ig.html]. Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548: