Can we rely on the DfT's in-year provisional estimates?
As promised, the DfT have today published their update on reported road casualties in 2015, letting us know what the situation is until the end of September 2015. Lots of road safety groups are interested in these results as it may give them an indication of recent trends. At RSA we have spent over a decade analysing these in-year figures and usually treat them with a large dose of caution. This time, though, there is a lot more information in there about how and why the numbers may have changed, which made interesting reading. First of all though, a word on how useful these numbers can ever be, even if they are 100% correct.
Road User Types
The DfT have also reviewed road user types (car users, pedestrians, pedal cyclists and motorcyclists) using the rolling-year methodology but have not carried out significance tests, probably because they wouldn’t stand a chance of returning a significant. Here KSI casualties in all groups have fallen with pedal cyclist casualties dropping most (5%) in the year ending 30th September 2015. Child casualties have fallen significantly if you compare tends against the 2010-14 average with a 16% reduction in KSI casualties. You can of course review the individual quarter’s performance but the numbers get smaller, percentage changes get higher, and there’s no hope of calculating significance.
Pedal Cycle Casualties -5%
Why have casualties reduced?
Whenever changes in casualty number are reported the natural question to ask is ‘why’ and this is difficult to answer. Road traffic is held up as the biggest influencing factor as exposure would surely help to determine the number of those injured on the roads. This is in turn linked to economic performance which could actually be what influences traffic levels in the first place. Other big factors that come into play are the weather (see our article from last year on this) and changes in licensure patterns (fewer young drivers). The DfT themselves say of the recent changes that, ‘there are a number of factors that have combined together to improve some aspects of safety on Britain’s roads’.
The DfT have for the first time applied an adjustment allowing for weather effects which, rather unexcitedly, shows that weather for the year ending September 30th 2015 was remarkably normal with barely any correction up or down. The previous year’s data did receive more of a correction however which virtually wipes out all of the KSI reductions mentioned at the start of this article. This begs the question then; if the KSI reductions are statistically significant, but can largely be attributed to a change to normal weather patterns, does that mean that it’s the weather that has significantly changed and not the safety levels on our roads?
The big problem with all of this is…
Uncertainty in the provisional estimates are at an all-time-high, at least the author can’t remember a time when the provisional data release was subject to such a large caveat about the submission of data from ‘some police forces’. The DfT notes that they have had to change what they originally estimated for January – June as there were more casualties in the first two quarters of 2015 than previously estimated due to late returns (my phrase, not theirs).
A quote directly from today’s release nicely clarifies the situation, “In 2014 roughly 97 per cent of all accidents for the second quarter of the year had been submitted at the time of publishing the provisional figures for that quarter. Therefore we uprate the 2015 returns by 3.09 per cent to take into account the expected number of records yet to be submitted. The amount the figures are uprated by will be too low if there are significantly more than the expected 3 per cent of records missing. This is what has happened this year.”
The reasons for under- or late-reporting are far too full of complexity and conjecture to cover here and there is still the expectation that everything will be alright in the end i.e. when we get the official 2015 results in June.
What seems clear though is that the DfT are much more open about data collection and the factors that influence national trends and this is to be warmly welcomed. Simply counting casualties to determine whether a change is significant is poor practice and an understanding of why changes take place should be made wherever possible. It may sound difficult but there are lots of techniques local authorities and police forces can use to review trends and if you would like to talk to one of the team about how to do this, please get in touch.
Our own MAST Provisional system will be updated in the coming weeks and we will be in touch via our newsletter to let you know when that happens.