What GAO Found
DOT based estimates of the business benefits of electronically controlled pneumatic (ECP) brakes on limited data, in part, because railroads that have used ECP brakes to date have shared limited data on their use. ECP brakes provide an electronic brake signal instantaneously throughout the train, allowing train cars to brake faster than with conventional air brakes. In supporting the May 2015 rule requiring the use of ECP brakes on certain trains hauling flammable liquid, the Department of Transportation (DOT) estimated the potential business benefits of ECP brakes, including reduced fuel consumption, reduced wear on wheels, and improved operational efficiencies. Industry stakeholders claim that DOT overestimated benefits. Seven of 10 experts GAO interviewed who commented on such benefit estimates said that DOT's estimates of business benefits, such as reduced fuel consumption, were based on experiences that may not be representative. DOT also estimated benefits to railroads from improved operational efficiency (e.g., the ability to return to speed sooner after braking), while industry stakeholders stated that poor reliability of ECP brakes will greatly limit any such benefits; however, only two out of five railroads provided GAO extensive quantifiable data to support these claims. DOT's use of limited data adds uncertainty to the estimates that DOT did not always acknowledge in the rule and its supporting analysis. By acknowledging uncertainties and in the future requiring railroads to collect and provide DOT more data on ECP brake use, DOT could improve its estimates and public confidence in those estimates, and use the data to determine the extent to which the ECP brake rule is meeting its objectives.
DOT and an industry association each conducted computer-based modeling and additional analysis to estimate the potential safety benefits of ECP brakes, but took different approaches based in part on different assumptions of how the brakes affect what happens in a derailment. DOT's analysis supporting its final rule found that the improved braking performance of ECP brakes can reduce the number of cars in a derailment that puncture and release their contents by almost 20 percent compared to other braking technologies. DOT published two reports and explanatory details in the final rule to document this approach. The industry association's analysis and modeling, using a different approach and assumptions, found ECP brakes provide a “marginal” safety benefit. GAO found DOT's modeling lacked transparency as the information published may not be sufficient to enable an independent third party to replicate it. For example, DOT did not report complete details on specific inputs, such as how the model applied the brake force to tank cars. One researcher attempted to replicate the analysis and told GAO he was unable to do so, citing limited information. Best practices identified by the Office of Management and Budget state that modeling results published by federal agencies should be supported by transparent data to facilitate third-party review. By providing more information about the modeling, DOT could help stakeholders and the public better understand the analysis and the extent to which the model's results hinged on DOT's choices and assumptions. This increased understanding could in turn increase confidence in the ECP brake requirement.
Why GAO Did This Study
In May 2015, DOT issued a final rule requiring certain trains hauling flammable liquids to equip with ECP brakes. This rule has met opposition from many industry stakeholders. The Fixing America's Surface Transportation Act requires DOT to test ECP brakes and reevaluate the economic analysis supporting the ECP brake requirement and includes a provision for GAO to review the potential costs and benefits of ECP. This report examines views on costs railroads may realize in meeting the ECP brake rule, the potential business benefits, and how DOT and the railroad industry estimated safety benefits.
GAO reviewed rulemaking documents; interviewed 13 rail experts selected based on published work and suggestions from the National Academies of Sciences; interviewed DOT officials and representatives of the seven largest railroads in North America; interviewed industry stakeholders, including the Association of American Railroads and compared DOT's estimates and modeling efforts against federal criteria and GAO standards for internal control.
GAO recommends that DOT acknowledge uncertainty in its revised economic analysis of ECP brakes, collect data from railroads on their use of ECP brakes, and publish additional information about ECP brake modeling. DOT disagreed with the recommendations, stating that GAO did not provide sufficient evidence. GAO believes it had sufficient evidence and stands by the recommendations, as discussed in this report.
Recommendations for Executive Action
|Department of Transportation||
Priority Rec.As DOT, in response to the Fixing America's Surface Transportation (FAST) Act, conducts additional evaluation and analysis of ECP brakes and updates the regulatory impact analysis, the Secretary of Transportation should direct the Federal Railroad Administration (FRA) and Pipeline and Hazardous Materials Safety Administration (PHMSA) to take into account, in the updated regulatory impact analysis conducted in response to the FAST Act, potential uncertainty in key variables and assumptions, such as, but not limited to, fuel prices and future rail traffic of crude oil and ethanol, discuss this uncertainty, and present ranges of possible scenarios.
|Department of Transportation||As DOT, in response to the FAST Act, conducts additional evaluation and analysis of ECP brakes and updates the regulatory impact analysis, the Secretary of Transportation should direct FRA and PHMSA to create a plan to collect data from railroads' ongoing and future operational experiences using ECP brakes. The plan should include details on how the agency will work with railroads to collect this data, ensure that such data are reliable, and analyze these data to conduct a retrospective analysis of the ECP brakes requirement that could help inform any potential future actions regarding ECP brakes.|
|Department of Transportation||As DOT, in response to the FAST Act, conducts additional evaluation and analysis of ECP brakes and updates the regulatory impact analysis, if, based on its updated analysis, DOT promulgates a new rule on the applicable ECP brake system requirements, the Secretary of Transportation should direct FRA and PHMSA to require that freight railroads, once they equip with ECP brakes in response to the requirement, collect and provide data to FRA on their ongoing operational experience with ECP brakes.|
|Department of Transportation||As DOT, in response to the FAST Act, conducts additional evaluation and analysis of ECP brakes and updates the regulatory impact analysis, the Secretary of Transportation should direct FRA and PHMSA to publish information--including data inputs, formulas, and results of all simulations and assumptions regarding DOT's use of the LS-DYNA model used and related analyses to support the 2015 final rule--that would allow a third party to fully assess and replicate the analysis.|