Reliability of FTA's Rail Accident Database
GAO-11-217R: Published: Jan 31, 2011. Publicly Released: Jan 31, 2011.
As Congress requested, we conducted a review of challenges associated with enhancing safety on major rail transit systems. During the course of that review, we assessed the quality of data that the Federal Transit Administration (FTA) collects and maintains in its State Safety Oversight (SSO) Rail Accident Database. FTA, an agency within the Department of Transportation (DOT), collects these safety data, including data on types of accidents and causes, from SSO agencies and the rail transit agencies they oversee. FTA used the SSO Rail Accident Database to produce the agency's 2009 Rail Safety Statistics Report, which analyzed data from 2003 through 2008. Although we originally intended to report on safety trends using the SSO Rail Accident Database in the rail transit report, we determined that these data were not sufficiently reliable for such a purpose. As a result, in this review we further assess the SSO Rail Accident Database and FTA's processes for collecting and compiling the data. We determined that there are numerous inaccuracies in FTA's SSO Rail Accident Database and, consequently, the 2009 Rail Safety Statistics Report. FTA implemented changes to the data collection process over the past few years to address some of these issues. For example, FTA now requires SSO agencies to report incident specific information instead of aggregated or summary totals. In addition, FTA revised the definition of rail grade crossings to ensure consistent accident reporting by rail transit agencies, and now requests SSO agencies to provide unique incident tracking numbers to assist with data validation and prevention of duplicate entries. FTA is also working to validate and correct existing data in the SSO Rail Accident Database. Despite these changes, the SSO Rail Accident Database and the 2009 Rail Safety Statistics Report still include erroneous data. DOT is seeking legislative authority to directly regulate and enforce rail transit safety, in part on the basis of recent trends in accidents and injuries. There were several legislative proposals introduced during the 111th Congress to give FTA authority to establish safety regulations for rail transit agencies and, in cooperation with the states, oversee and enforce compliance by these rail transit agencies with these regulations. If FTA assumes greater oversight authority over rail transit agencies, accurate and reliable rail safety data will be critical. The purpose of this report is to convey our findings about how insufficient internal control over the entry and processing of rail transit safety data has negatively affected the reliability of the data contained in FTA's SSO Rail Accident Database and its 2009 Rail Safety Statistics Report.
Our analysis of the reliability of data contained in the SSO Rail Accident Database found data discrepancies, such as unverified and duplicative entries, as well as insufficient internal control. Consequently, FTA's 2009 Rail Safety Statistics Report, which focuses attention on safety issues in the public rail transit industry, and is intended to support FTA's mission to identify safety priorities and strategies to address industry concerns, includes unreliable data. This affects the accuracy of the statistics regarding rail transit safety, including those used for producing information on trends over time, which can be safety indicators to help guide FTA's safety oversight efforts. For example, data discrepancies may under or overstate accidents, injuries, fatalities, and property damage totals, among other things. FTA has implemented some changes to the data collection process and has begun to validate data and correct discrepancies contained in the SSO Rail Accident Database. However, FTA's validation proposal does not contain specific efforts to establish procedures that would improve data reporting in the future. We are making recommendations to DOT to help FTA improve internal control over the process used to obtain data from SSO agencies and ensure the accuracy and reliability of the SSO Rail Accident Database.
Recommendations for Executive Action
Status: Closed - Implemented
Comments: In January 2011, we found numerous inaccuracies in the Department of Transportation's Federal Transit Administration (FTA) State Safety Oversight (SSO) Rail Accident Database. After assessing the quality of data that FTA collects from states, such as data on types of accidents and causes, we determined that these data were not sufficiently reliable for reporting on safety trends, such as those presented, for example, in the agency's 2009 Rail Safety Statistics Report. Specifically, we found data discrepancies such as unverified and duplicative entries, as well as insufficient internal control over data manually entered by states each year into a template submitted to FTA. Such data discrepancies may under or overstate accidents, injuries, fatalities, and property damage totals affecting the accuracy of transit safety statistics and, potentially, FTA's ability to help guide safety oversight efforts. Therefore, we recommended that FTA develop and implement appropriate internal control activities, such as data entry design features, to ensure that data entered into SSO agency reporting templates are accurate. In June 2015, we confirmed that FTA had redesigned its SSO data collection mechanism, adding substantial internal controls to the annual reporting template to prevent data reporting errors and to further standardize data collection. These new internal control measures included data completion requirements to ensure users do not submit incomplete entries, data entry restrictions using drop-down menus, and defined data formats and ranges to ensure data are correctly formatted and within proper ranges. By implementing these internal controls, the quality of data contained within the SSO Rail Accident Database should improve, helping the agency with its safety oversight efforts.
Recommendation: The Secretary of Transportation should direct the Administrator of FTA to develop and implement appropriate internal control activities to ensure that the data entered into SSO agency reporting templates are accurate. To accomplish this, the Administrator should consider data entry design features to ensure consistency in reporting across rail transit agencies.
Agency Affected: Department of Transportation
Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
Recommendation: The Secretary of Transportation should direct the Administrator of FTA to incorporate appropriate internal control over the method used to review and reconcile SSO agency data with other data sources to better ensure accuracy and reliability of the SSO Rail Accident Database.
Agency Affected: Department of Transportation