Train Braking: DOT's Rulemaking on Electronically Controlled Pneumatic Brakes Could Benefit from Additional Data and Transparency
Highlights
What GAO Found
DOT based estimates of the business benefits of electronically controlled pneumatic (ECP) brakes on limited data, in part, because railroads that have used ECP brakes to date have shared limited data on their use. ECP brakes provide an electronic brake signal instantaneously throughout the train, allowing train cars to brake faster than with conventional air brakes. In supporting the May 2015 rule requiring the use of ECP brakes on certain trains hauling flammable liquid, the Department of Transportation (DOT) estimated the potential business benefits of ECP brakes, including reduced fuel consumption, reduced wear on wheels, and improved operational efficiencies. Industry stakeholders claim that DOT overestimated benefits. Seven of 10 experts GAO interviewed who commented on such benefit estimates said that DOT's estimates of business benefits, such as reduced fuel consumption, were based on experiences that may not be representative. DOT also estimated benefits to railroads from improved operational efficiency (e.g., the ability to return to speed sooner after braking), while industry stakeholders stated that poor reliability of ECP brakes will greatly limit any such benefits; however, only two out of five railroads provided GAO extensive quantifiable data to support these claims. DOT's use of limited data adds uncertainty to the estimates that DOT did not always acknowledge in the rule and its supporting analysis. By acknowledging uncertainties and in the future requiring railroads to collect and provide DOT more data on ECP brake use, DOT could improve its estimates and public confidence in those estimates, and use the data to determine the extent to which the ECP brake rule is meeting its objectives.
DOT and an industry association each conducted computer-based modeling and additional analysis to estimate the potential safety benefits of ECP brakes, but took different approaches based in part on different assumptions of how the brakes affect what happens in a derailment. DOT's analysis supporting its final rule found that the improved braking performance of ECP brakes can reduce the number of cars in a derailment that puncture and release their contents by almost 20 percent compared to other braking technologies. DOT published two reports and explanatory details in the final rule to document this approach. The industry association's analysis and modeling, using a different approach and assumptions, found ECP brakes provide a “marginal” safety benefit. GAO found DOT's modeling lacked transparency as the information published may not be sufficient to enable an independent third party to replicate it. For example, DOT did not report complete details on specific inputs, such as how the model applied the brake force to tank cars. One researcher attempted to replicate the analysis and told GAO he was unable to do so, citing limited information. Best practices identified by the Office of Management and Budget state that modeling results published by federal agencies should be supported by transparent data to facilitate third-party review. By providing more information about the modeling, DOT could help stakeholders and the public better understand the analysis and the extent to which the model's results hinged on DOT's choices and assumptions. This increased understanding could in turn increase confidence in the ECP brake requirement.
Why GAO Did This Study
In May 2015, DOT issued a final rule requiring certain trains hauling flammable liquids to equip with ECP brakes. This rule has met opposition from many industry stakeholders. The Fixing America's Surface Transportation Act requires DOT to test ECP brakes and reevaluate the economic analysis supporting the ECP brake requirement and includes a provision for GAO to review the potential costs and benefits of ECP. This report examines views on costs railroads may realize in meeting the ECP brake rule, the potential business benefits, and how DOT and the railroad industry estimated safety benefits.
GAO reviewed rulemaking documents; interviewed 13 rail experts selected based on published work and suggestions from the National Academies of Sciences; interviewed DOT officials and representatives of the seven largest railroads in North America; interviewed industry stakeholders, including the Association of American Railroads and compared DOT's estimates and modeling efforts against federal criteria and GAO standards for internal control.
Recommendations
GAO recommends that DOT acknowledge uncertainty in its revised economic analysis of ECP brakes, collect data from railroads on their use of ECP brakes, and publish additional information about ECP brake modeling. DOT disagreed with the recommendations, stating that GAO did not provide sufficient evidence. GAO believes it had sufficient evidence and stands by the recommendations, as discussed in this report.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Transportation |
Priority Rec.
As DOT, in response to the Fixing America's Surface Transportation (FAST) Act, conducts additional evaluation and analysis of ECP brakes and updates the regulatory impact analysis, the Secretary of Transportation should direct the Federal Railroad Administration (FRA) and Pipeline and Hazardous Materials Safety Administration (PHMSA) to take into account, in the updated regulatory impact analysis conducted in response to the FAST Act, potential uncertainty in key variables and assumptions, such as, but not limited to, fuel prices and future rail traffic of crude oil and ethanol, discuss this uncertainty, and present ranges of possible scenarios. |
Closed – Implemented
In May 2015, DOT issued a final rule requiring certain trains hauling flammable liquids to be equipped with electronically controlled pneumatic (ECP) brakes. This rule met opposition from many industry stakeholders. The Fixing America's Surface Transportation Act (FAST Act), enacted in December 2015, requires DOT to test ECP brakes and reevaluate the economic analysis supporting the ECP brake requirement. In 2016, we reported that the DOT did not appropriately account for uncertainty when estimating the costs and benefits of its 2015 rule. To support the rule, DOT conducted computer-based modeling of the potential safety benefits of ECP brakes and estimated potential costs and benefits of the ECP brake requirement in a regulatory impact analysis (RIA). Industry stakeholders claimed that DOT overestimated benefits and underestimated the costs of ECP brakes. We reported that DOT based estimates of benefits associated with ECP brakes on limited data, in part, because railroads that have used ECP brakes to date have shared limited data on their use. DOT also made single point estimates of potential costs and benefits instead of a range of possible scenarios. For example, DOT's estimate for the potential safety benefits of ECP brakes was based on a single forecast of future oil by rail traffic over the next 20 years and not a range of future traffic levels. As a result, it introduced an additional level of uncertainty into many of its estimates and assumptions beyond the uncertainty inherent in any estimate. Guidance from the Office of Management and Budget suggests that federal agencies could address such uncertainty by providing a range of possible scenarios in estimates. We noted that DOT's acknowledgment of uncertainty in certain estimates in the RIA could help increase confidence in those estimates and address industry stakeholder concerns. Therefore, we recommended that DOT take into account, in an updated RIA that the FAST Act required the agency to conduct, potential uncertainty in key variables and assumptions, such as, but not limited to, fuel prices and future rail traffic of crude oil and ethanol, discuss this uncertainty, and present ranges of possible scenarios. In October 2017, DOT issued an updated RIA. This updated analysis accounted for uncertainty in a number of key variables, such as by providing high and low estimates the forecasted volume of traffic of crude oil and ethanol by rail, discussing the uncertainty of forecasted fuel prices, and by providing a range of the estimated costs and benefits of the rule. As a result, industry stakeholders and the public can have increased confidence in the updated cost and benefit estimates provided in DOT's updated RIA supporting the 2015 rule.
|
| Department of Transportation | As DOT, in response to the FAST Act, conducts additional evaluation and analysis of ECP brakes and updates the regulatory impact analysis, the Secretary of Transportation should direct FRA and PHMSA to create a plan to collect data from railroads' ongoing and future operational experiences using ECP brakes. The plan should include details on how the agency will work with railroads to collect this data, ensure that such data are reliable, and analyze these data to conduct a retrospective analysis of the ECP brakes requirement that could help inform any potential future actions regarding ECP brakes. |
Closed – Not Implemented
Our report, issued in October 2016, examined DOT's rulemaking on ECP brakes and determined that DOT needed additional data and greater transparency in estimating potential safety benefits of ECP brakes. After conducting additional analyses, as required by the FAST Act, DOT determined that the ECP brake requirements for railroads were not economically justified, and issued a final rule in September 2018 removing these requirements. Given that DOT's ECP brake requirements are no longer in effect, we are closing these recommendations as not implemented.
|
| Department of Transportation | As DOT, in response to the FAST Act, conducts additional evaluation and analysis of ECP brakes and updates the regulatory impact analysis, if, based on its updated analysis, DOT promulgates a new rule on the applicable ECP brake system requirements, the Secretary of Transportation should direct FRA and PHMSA to require that freight railroads, once they equip with ECP brakes in response to the requirement, collect and provide data to FRA on their ongoing operational experience with ECP brakes. |
Closed – Not Implemented
Our report, issued in October 2016, examined DOT's rulemaking on ECP brakes and determined that DOT needed additional data and greater transparency in estimating potential safety benefits of ECP brakes. After conducting additional analyses, as required by the FAST Act, DOT determined that the ECP brake requirements for railroads were not economically justified, and issued a final rule in September 2018 removing these requirements. Given that DOT's ECP brake requirements are no longer in effect, we are closing these recommendations as not implemented.
|
| Department of Transportation | As DOT, in response to the FAST Act, conducts additional evaluation and analysis of ECP brakes and updates the regulatory impact analysis, the Secretary of Transportation should direct FRA and PHMSA to publish information--including data inputs, formulas, and results of all simulations and assumptions regarding DOT's use of the LS-DYNA model used and related analyses to support the 2015 final rule--that would allow a third party to fully assess and replicate the analysis. |
Closed – Implemented
In May 2015, the Department of Transportation (DOT) issued a final rule requiring certain trains hauling certain flammable liquids such as crude oil to be equipped with electronically controlled pneumatic (ECP) brakes. The Fixing America's Surface Transportation Act (FAST Act), enacted in December 2015, required DOT to test ECP brakes and reevaluate the analysis supporting the ECP brake requirement. In 2016, we reported that to support the rule, DOT conducted computer-based modeling involving simulations of derailments to estimate the potential safety benefits of ECP brakes. DOT published two reports and explanatory details in the final rule to document its modeling approach. However, we found that DOT's modeling lacked transparency as the information published was not necessarily sufficient to enable an independent third party to replicate it. For example, DOT did not report complete details on specific inputs. Best practices identified by the Office of Management and Budget state that modeling results published by federal agencies should be supported by transparent data to facilitate third-party review. Because DOT did not provide some of the specific data and information underlying its modeling efforts and analysis, the public may not have had reasonable assurance as to DOT's projected safety benefits of ECP brakes, limiting industry stakeholder and public confidence in DOT's overall findings. As a result, we recommended that as DOT, in response to the FAST Act, conducts additional evaluation and analysis of ECP brakes, it should publish information--including data inputs, formulas, and results of all simulations and assumptions regarding DOT's use of the model and related analyses to support the 2015 final rule--that would allow a third party to fully assess and replicate the analysis. In September 2017, DOT finalized an updated report on its computer-based modeling of ECP brakes. This report included data on additional testing and modeling conducted. It also included information on additional validation conducted of the model. It included information on data inputs, formulas, and simulation results that would allow a third party to replicate the analysis. As a result, stakeholders and the public are in a better position to understand the analysis and the extent to which the model's results hinged on DOT's choices and assumptions that could in turn increase confidence in DOT's findings.
|