Safety scoring system “Conceptually Sound,” but needs improvement, report says

From Transport Topics:

The federal Compliance, Safety, Accountability program’s controversial Safety Measurement System to identify motor carriers at high risk for future crashes is “conceptually sound” but several features need improvement, according to a new congressionally mandated report by a panel of the National Academies of Sciences, Engineering and Medicine.

“SMS is structured in a reasonable way, and its method of identifying motor carriers for alert status is defensible,” the study, made public June 27, concluded. “However, much of what is now done is ad hoc and based on subject-matter expertise that has not been sufficiently empirically validated.”

The 132-page study was conducted over the past 15 months by a 12-member elite panel of academics, several who specialize in statistics and transportation safety policy. It was funded by the Federal Motor Carrier Safety Administration and mandated by the FAST Act of 2015.

The approximate cost of the study is just under $1 million, said an academies spokeswoman.

The academies’ panel was charged with analyzing the ability of FMCSA’s SMS to discriminate between low- and high-risk carriers, assess the public usage of SMS, review the data and methodology used to calculate the measures and provide advice on additional data collection and safety assessment methodologies.

In March, FMCSA withdrew a January 2016 proposed motor carrier safety fitness rule saying it was awaiting the results of the study.

While the panel’s report was complimentary of the agency for many of its ideas and efforts, it recommended that over the next two years regulators develop a more “statistically principled approach” based on an “item response theory” — that is, a more detailed data-oriented approach that digs deeper and measures the performance of individual trucks and buses, not just at the motor carrier level.

See the full story from Transport Topics online.