Motor Carriers:

Better Information Needed to Assess Effectiveness and Efficiency of Safety Interventions

GAO-17-49: Published: Oct 27, 2016. Publicly Released: Oct 27, 2016.

Additional Materials:


Susan Fleming
(202) 512-2834


Office of Public Affairs
(202) 512-4800

What GAO Found

In July 2010, the Federal Motor Carrier Safety Administration (FMCSA) chose to delay nationwide implementation of two of the eight interventions that FMCSA uses to address motor carrier safety concerns under its Compliance, Safety, Accountability (CSA) program. This delay is linked to continuing delays in developing software needed to support the two interventions, offsite investigations and the use of cooperative safety plans. The software under development is intended to help FMCSA overcome some of the information challenges it faces due to its reliance on legacy information systems. FMCSA estimates that the software development project will be completed by April 2017.

FMCSA has conducted evaluations of the effectiveness and efficiency outcomes it established for the CSA program. However, GAO identified several limitations in FMCSA's approaches that impact the usefulness of the evaluations:

Intervention effectiveness: FMCSA has developed a statistical model to annually evaluate the combined effectiveness of interventions. Although the model has some key strengths, such as accounting for a broad range of external factors, GAO identified a number of design and methodology limitations that reduce the usefulness of its results. For example, the model does not include an assessment of individual intervention types. Without this type of specific information, FMCSA is hampered in its ability to identify the circumstances under which different types of interventions are effective. Similarly, these types of limitations affect FMCSA's ability to accurately draw conclusions about intervention effectiveness across all intervention types.

Intervention efficiency: To assess the efficiency of CSA interventions, FMCSA has relied on a study that it sponsored and that was published in 2011. This study estimated the average cost of conducting interventions in four states from October 2008 through May 2009. However, FMCSA has not taken steps to update its cost estimates for interventions since the 2011 evaluation, despite changes since that time in the resources needed to conduct CSA interventions; nor has it taken steps to develop additional information that is representative of the costs in other states. Without current cost estimates that are representative of all states, FMCSA cannot appropriately assess the efficiency of its interventions.

FMCSA has taken some actions to improve the effectiveness and efficiency of CSA interventions, but lacks measures to monitor progress. In April 2014, FMCSA established a working group to assess CSA interventions and make recommendations for improvement. As of April 2016, the group had made 20 recommendations, of which 12 had been implemented. However, GAO found that while FMCSA has established some performance measures for its effectiveness outcome that are appropriate, it has not established similar measures for its efficiency outcome. FMCSA headquarters officials told GAO that effectiveness and efficiency are complementary outcomes that FMCSA strives to balance. Without a complete set of measures for both outcomes, FMCSA lacks benchmarks needed to regularly measure progress to achieve these outcomes.

Why GAO Did This Study

As part of its mission to reduce crashes and fatalities involving large commercial trucks and buses, FMCSA seeks to use a data-driven approach to identify the highest-risk motor carriers and address safety problems by applying a range of eight CSA program enforcement tools, called interventions, ranging from warning letters to placing carriers out of service.

A provision in a Senate report requires GAO to periodically assess FMCSA's implementation of the CSA program. This report examines the extent to which FMCSA has (1) implemented CSA interventions, (2) evaluated the effectiveness and efficiency of CSA interventions, and (3) monitored progress toward achieving outcomes.

GAO reviewed FMCSA data and documentation on all eight CSA intervention types from fiscal years 2010–2015, including FMCSA's strategic planning documents, guidance, and program evaluations. GAO interviewed industry stakeholders and FMCSA officials in headquarters, in each of FMCSA's service centers, and in eight states selected for their participation in FMCSA's CSA pilot test, location, and program size, among other factors.

What GAO Recommends

GAO recommends that FMCSA evaluate the effectiveness of individual intervention types, update cost estimates so that they are current and representative of all states, and establish complete performance measures.

The Department of Transportation concurred with all of GAO's recommendations.

For more information, contact Susan Fleming at (202) 512-2834 or

Recommendations for Executive Action

  1. Status: Open

    Comments: The Federal Motor Carrier Safety Administration (FMCSA) has begun a review of accepted practices for designing program effectiveness evaluations. Based on this review, FMCSA plans to assess current models and studies to determine the agency's capability to measure the effectiveness of CSA interventions in improving motor carriers' safety behavior. Upon completion of this assessment, FMCSA intends to explore the feasibility of implementing new methods to measure the effectiveness of individual intervention types. The planned completion for these actions is in early 2018.

    Recommendation: To determine whether CSA interventions influence motor carrier safety performance, the Secretary of Transportation should direct the FMCSA Administrator to identify and implement, as appropriate, methods to evaluate the effectiveness of individual intervention types or common intervention patterns to obtain more complete, appropriate, and accurate information on the effectiveness of interventions in improving motor carrier safety performance. In identifying and implementing appropriate methods, FMCSA should incorporate accepted practices for designing program effectiveness evaluations, including practices that would enable FMCSA to more confidently attribute changes in carriers' safety behavior to CSA interventions.

    Agency Affected: Department of Transportation

  2. Status: Open

    Comments: The Federal Motor Carrier Safety Administration (FMCSA) plans to update intervention cost information and to take representative state information--such as varying travel costs--into consideration. Due to a decline in the number of offsite investigations conducted, FMCSA was not able to collect sufficient data to update its cost estimates for all intervention types in calendar year 2017. FMCSA plans to collect data on the resources used to conduct each intervention type in spring 2018 and to complete its analysis of data collection results by June 30, 2018.

    Recommendation: To understand the efficiency of CSA interventions the Secretary of Transportation should direct the FMCSA Administrator to update FMCSA's cost estimates to determine the resources currently used to conduct individual intervention types and ensure FMCSA has cost information that is representative of all states.

    Agency Affected: Department of Transportation

  3. Status: Open

    Comments: Based on completion of recommendations 1 and 2 from GAO's report, the Federal Motor Carrier Safety Administration (FMCSA) plans to establish an inventory of effectiveness and efficiency measures and monitor performance on an ongoing basis. Its planned implementation date for these performance measures is January 31, 2019.

    Recommendation: To enable FMCSA management to monitor the agency's progress in achieving its effectiveness and efficiency outcomes for CSA interventions and balance priorities, the Secretary of Transportation should direct the FMCSA Administrator to establish and use performance measures to regularly monitor progress toward both FMCSA's effectiveness outcome and its efficiency outcome.

    Agency Affected: Department of Transportation


Explore the full database of GAO's Open Recommendations »

Feb 28, 2018

Jan 30, 2018

Jan 16, 2018

Dec 21, 2017

Dec 14, 2017

Dec 4, 2017

Nov 30, 2017

Nov 15, 2017

Nov 2, 2017

Oct 31, 2017

Looking for more? Browse all our products here