Visualising Road Safety Data with MapsJanuary 7, 2016
Analysing Northamptonshire’s Switched-Off Speed CamerasMarch 10, 2016
Can we rely on the DfT's in-year provisional estimates?
As promised, the DfT have today published their update on reported road casualties in 2015, letting us know what the situation is until the end of September 2015. Lots of road safety groups are interested in these results as it may give them an indication of recent trends. At RSA we have spent over a decade analysing these in-year figures and usually treat them with a large dose of caution. This time, though, there is a lot more information in there about how and why the numbers may have changed, which made interesting reading. First of all though, a word on how useful these numbers can ever be, even if they are 100% correct.
Statistically significant resultsWithin the road casualties dataset for GB we sadly have a large amount of information that makes robust analysis quite easy. When the full-year results are announced there are hundreds of thousands of casualties and vehicles to work with and comparing change over time can tell us when meaningful changes have taken place. The word ‘meaningful’ can depend on your position but for most analysts we would want to find out if the change is statistically significant, which would hopefully rule out any blips and random fluctuations in the data (technically classified as natural variation).
DfT DefinitionThe 95% confidence level is the standard against which statistics are typically tested. It means that in 100 years with the same risk of fatalities (or injury), 95 of those years will result in a number of fatalities (or injuries) between a given range. If the actual change falls outside of this range then we can be 95% confident that the change is as a result of a genuine trend rather than a product of chance.
Example from the 2014 dataWhen RRCGB was published in September 2015 there was a section on the reported changes in casualty numbers and whether they were statistically significant. You can see the results in the table below which create an expected range (central forecast) against which the results can be tested. In this case the increases in fatalities and KSI casualties as not significant but the increase in casualties of all severities was.
Two ways of looking at in-year dataLooking at the provisional data for July – September 2015, the raw data appears to show some variations versus the previous years. Rather than just looking at three months’ worth of data the DfT have created ‘rolling years’ that effectively change the year end date – in this case to 30th September rather than 31st December. The results for road deaths seem to be bad news with a 3% increase, although this figure is in line with the average for the preceding five years. When reviewing KSI casualties there has been a 3% drop, set against a rise in traffic levels of 2.2% which may indicate our roads are becoming relatively safer. Using the DfT’s significance tests they say that, ‘There has been a statistically significant decrease in the number of people injured (but not killed) in road traffic accidents between the years ending September 2014 and 2015’, but that ‘Although the number of people killed in road traffic accidents has increased between years, this change is small enough that it can be explained by the natural variation in deaths over time’. They have also published figures for the third quarter on their own, none of which pass the 95% confidence level, despite showing similar percentage change reductions for KSI casualties. This demonstrates why you need more data to have confidence that the changes you are seeing are significant. Local authorities and police forces often take an interest in in-year data and hope to reveal patterns to help understand what’s happening on the roads. Given the small numbers involved here it will be a very difficult job to truly say whether local changes are significant or not. Filtering data to individual user groups is almost meaningless on a quarter-by-quarter basis.
Road User TypesThe DfT have also reviewed road user types (car users, pedestrians, pedal cyclists and motorcyclists) using the rolling-year methodology but have not carried out significance tests, probably because they wouldn’t stand a chance of returning a significant. Here KSI casualties in all groups have fallen with pedal cyclist casualties dropping most (5%) in the year ending 30th September 2015. Child casualties have fallen significantly if you compare tends against the 2010-14 average with a 16% reduction in KSI casualties. You can of course review the individual quarter’s performance but the numbers get smaller, percentage changes get higher, and there’s no hope of calculating significance.
Pedal Cycle Casualties -5%
Motorbike Casualties -4%