Monday, 16 March 2009

Bicycle overtaking and rebuttals

Something I've become slightly infamous for is my 2007 work on drivers overtaking bicyclists. A couple of weeks ago I was alerted to a US website where somebody called Dan Gutierrez posted a surprisingly angry critique of my findings as well as some data from his own replication of parts of the study (the main document is here [pdf]). Dan found some different results to me, which is great as I've long expected there would be differences in driver behaviour between the UK and the US, particularly because of differences in road design between the two countries. However, rather than simply conclude our countries are different, Dan seems to conclude I'm either a big numpty who can't do research, or a deliberate liar. Either way: ouch, Dan.

Two weeks ago I emailed Dan to try to clear things up, but haven't had a reply, so I thought I'd reproduce my email here. Given he's been quite so stinging about my work in a public forum I feel I should have some right to reply. And, more critically, I spent ages writing this email and at least by posting it here the effort is less wasted. Again: ouch, Dan.

Dear Dan,

Hello! Someone recently pointed me to your online article discussing the findings of the bicycle overtaking study I conducted a couple of years ago. I had a look at your document and I have to say, I was slightly surprised by the general tone, and use of words like 'deceptive' when referring to my presenting findings. But hey! I've been called worse things than that. I hope you don't mind my writing to try and clear up one or two things, as having read that document I almost feel I've offended you somehow?

First, the graphs. I'm a big fan of How to Lie with Statistics too, but I really wasn't trying to hide anything with those graphs. They were intended primarily for use by my colleagues, who regularly use graphs of this sort and who would be totally familiar with the practice of truncating the y-axis. It's done simply to make the differences that exist clearer, to facilitate discussion, not to hide the overall magnitude of an effect - I'd fully expect people to look at the bottom of the axis and see it doesn't start at zero. I'm also satisfied I didn't build up any insignificant differences into significant ones by plotting the graphs this way - I can give you more details to explain this if you're interested.

Moreover, you're dead right that the overall mean passing distance is about 4 feet, but I think by focusing only on the average passing distances you overlook the really important thing, which is all the variation in the data. As you'd expect, the gaps drivers leave when passing cyclists vary a great deal. The distribution of gaps wasn't far off being a Gaussian distribution - a bell curve - which means most drivers left an amount of space somewhere near the average, a few left a massive amount of space and, critically, a few down in the left-hand tail-end of the distribution left very little space indeed (in fact, two of the drivers I encountered left less than zero space).

This last point, about the small number of drivers who leave very little space, is probably important. Every day, many cyclists are passed by motor vehicles. And we know that some of these events end with the cyclists being hit, sadly. Given there are drivers who leave very little space indeed (to the extent some leave less than none), I'd say that it probably does matter if the average gap left by drivers gets smaller. If the average gap gets smaller, this very likely means the whole distribution is shifting along to the left. It's probable - but by no means proven, certainly - that if the average gap declines by one inch, the very near misses shift by an inch too, so all the vehicles that would have just missed the bicycle by a whisker (which we know happens fairly often) instead become vehicles that just hit by a whisker. And the bigger the shift in average passing distance, the more near-misses might become hits. Noone can prove this, but given there are so many near-misses already every day, I really wouldn't want to see drivers doing anything to decrease the gaps they leave, even by only a centimetre on average. I don't know what your thoughts are on this tail-end issue?

You went on to mention that I didn't also look at riding in a position to command the lane. There is a cultural misunderstanding here! British roads are quite small compared to yours (typically an urban lane is about 2.5m wide; often half that size in the countryside). The 1.25m riding position is pretty much in the centre of the lane, so you can take those 1.25m data as being the centre-of-the-lane commanding data you were looking for. Also, you're totally correct to say that drivers did change their behaviour in response to changes in bicycle lane position - I think I said something to that effect in the paper's discussion - but the key point remains: as the bicycle moved further towards the centre of the road, the gap between it and passing vehicles tended to decline. That's why I said 'to a first approximation' vehicles don't respond to changes in the bike's position: I know they do respond, but they don't respond enough! As you say, a 1 foot move by the bike led to a 0.75 foot response in my data; the further out the bike was, the smaller the gap between it and the passing vehicles. Hence my saying 'to a first approximation': that statement was worded to convey my surprise at this finding, not to ignore it. (Incidentally, I suspect the strange disappearance of the helmet effect at the 1m riding position in my data is something to do with this position forcing motorists to approach the centreline of the road; at 1.25m they definitely have to cross it, but at 1m I suspect they had just enough space that they tried to stay entirely within the lane. Or something like that. You have to remember this was the first study looking at such things, and it wasn't clear in advance that the centreline would be an issue. Research builds over time.)

The difference between our countries' road systems, which I mentioned above, is the key to the final part of your paper. I'm honestly really impressed by the lengths you've gone to in collecting in those data. I'm on record saying that I'd expect other countries (especially in North America) to see different results to those I found in the UK, and that I'd love it if people were able to test this. That was why I was quite surprised to see that when somebody finally did do this test, it was in a document that seems distinctly hostile to me! It's great that you found something different to what I found - we now have concrete data showing there's a difference between our countries. But might it not have been fairer simply to describe this as what it is - a difference between our countries - rather than suggest I don't know what I'm talking about?

Anyway, this was only going to be a short email and it's grown into a lengthy one. I write in a genuine spirit of friendship, rather than to moan, although I fear it won't come across that way. I doubt either of us likes the idea of cyclists being struck from behind by passing cars (which, in the UK's accident data at least, has a really high probability of killing the cyclist). Any information we can gather which might make this less likely is incredibly valuable. I hope in future we might work together, rather than in opposition, towards this goal.

With good wishes,

Ian

Quite reasonable, I hope you'll agree. It's a shame we cyclists can't get along more. Goodness knows, we should be united against the common enemy.

17 comments:

Anonymous said...

Poor Ian, I think his feelings are hurt. If I had intended to call him “a big numpty who can't do research, or a deliberate liar”, I would have done so. Surely an academician can handle direct, even pointed criticism? I do have problems with both his methods and conclusions, and we will be discussing this (at least I hope he will want to discuss these issues with me) in the future.

For the record, I've been overwhelmed with work (doing two jobs to compensate for an ill senior colleague and mentor, who I’m also visiting the hospital nearly daily, and family (aged parents in law) and advocacy issues (too numerous to itemize here), so replying to Ian’s message has not been at the top of my priority queue. His e-mail to me deserves a thoughtful response; one that I haven't yet completed. It also wouldn't have occurred to me to post his private message to a blog, but since he has chosen to do so, my response will be similarly public; probably on the CABO blog in addition to Ian directly.

I will get back to posting a response...but probably not today.

- Dan Gutierrez -
www.CyclistView.com

Dave McCraw said...

Dear Dan,

What do you expect when you make an outright (and repeated) claim of deliberate deception to exaggerate the findings?

Frankly I'm amazed that something as routine as a truncated axis is considered underhand.

In a party political broadcast, perhaps. But in a serious piece of research, I don't think so.

On the positive side I think it's encouraging that attention is being focused on this area - well done.

Kind regards,

Dave McCraw

Tom said...

Dan,

While we are at war, could you make England pay for all the buildings they burned in 1812.

Bye the way, I sorry about the health of your loved ones. If that isn't too private of an issue.

Appologies to Ian for using his blog to "take the mick".

Anonymous said...

oooh.. i'm a gonna watch this space!!

researchers slugging it out!! In public!! This should be fun...

Scientific research by definition means that one is trying to find out things that is generally not known. In science, research is normally conducted carefully and diligently, and the results are described accurately, along with a discussion of the results.. which may be the section where the researcher's biases or 'gut' feelings can be manifested.

Also, the result of a research study is always important, whichever way the findings lean towards.

Finally, research findings are meant to be studied and more often than not, not taken at face vale. So there are rebuttals and counter arguments all the time in scientific research, but for the most part, it is civil.

Dr. Walker conducted his research with the full spirit of science and published his findings.

Gutierrez (Phd?) slammed the results, while calling him names at the same time... and not in a scientific arena, but something as commonplace as a webpage.

While some of Gutierrez's arguments may hold water, his approach was abrasive and unprofessional.

It is impressive that Gutierrez conducted his own data collection as well, in an attempt to replicate Walker.. but the sheer venom aimed in his rebuttal almost nullifies the credibility of the US data.

Gutierrez has some explaining to do, and perhaps out of this interaction there will emerge a better understanding of driver behavior around cyclists.

.
.
.

Like I said, I'll be watching this space.

-AKP

Anonymous said...

There is a difference between Walker's and Gutierrez's experiments which neither have pointed out: Walker rode alone. Gutierrez rode with a 2nd bicyclist.

I have noted many times that when I ride alone I often have those one or two too-close drivers. When I ride in a group those too-close moments don't happen. Same roads, same time of day. Just a group of 6+ riders instead of myself alone.

I don't know why, and I don't know whether 2 riders is different enough, but it should noted that the two experiments are not equivalent.

didrik said...

Also, wasn't Ian looking at helmet/no helmet and gender perception effect on passing distance? Did he vary road position? It seems like Dan was testing the effect of lane position on passing distance. The studies don't seem the same to me.

Anonymous said...

Ian, I'd like to pick your brains on the question on the status of Cyclists in the eyes of motorists. I think you were talking about this recently in Holland.

I write from Sydney Australia where the relationship between the two is poisonous, in part because as many have noted, the fewer people cycle, the worse it is. Our bike commute rate is a woeful .8%
Cycling here is primarily a weekend leisure activity.

But I'm wondering about other factors as determinants of bike status, respect, and thus safety.

Few people seem to have twigged to the fact that here in Aust, most cyclists are on the wrong sort of bikes for transport. They are hunched over flat handlebars or even over the turned down bull bars.

Very few people sit up perfectly straight as do the Dutch. You'd find it hard to get a bike shop here to sell you such a bike.

They are seen as sissy although the overt excuse is that they hit headwinds.

My feeling is that you could do a study which might show that rider sitting up straight, able to look drivers in the eye, either directly, or via a rear vision mirror, is safer, treated with more respect, etc.

Have you thought of using your Equipment on the hunched over/up straight split?

Secondly, I suspect that the Electric bike, now coming into vogue, will be a safer bike in terms of the status it enjoys, even though it might not be always apparent that one is riding one. Certainly its easier to be sitting up straight on the E bike since wind is not a factor.

Secondly, my personal observations is that one feels different on the road, more as one vehicle with others.

This might engender a false sense of safety, I know.

Anyway, my core interest is much larger, whether the E bike can help create a thinking shift in a country very hostile to bike commuting by bringing a different cachet to commuter cycling.

Are you coming to speak in Australia by any chance?

Mike Rubbo rubbo@aapt.net.au

Ian Walker said...

Hi Mike,

Thanks for the comments. Yes, I just got back from the Netherlands today. I haven't ever cycled there before, but this weekend did 240 km to and from the Ligfiets meeting/World Human Powered Vehicle championships. Anyone with an interest in cycling as a means of transport really needs to visit the Netherlands immediately, to see what can be done, and how great it feels to ride long distances without being bothered by cars. Not that the Dutch will admit they have a good system: they seem to share same tendency we have in Britain whereby if anyone from outside offers a compliment, we immediately say "Oh, clearly you haven't seen all the problems..."!

Anyway, with regard to different riding styles/different types of bike, I think we might now be at a stage where we know a bicyclist's appearance can alter a driver's behaviour, at least across lots of people. As such, probably the simplest way of getting an idea of how drivers would respond to upright commuter bikes versus lower sports bikes would be simply to survey some drivers, showing them pictures of riders on different types of bike and seeing how the drivers categorize them (skilled, cautious, fast, slow, etc.). If you've bought my claim that such beliefs can translate into driving behaviour, then this might give you some idea of how drivers are likely to respond to those different types of bike out on the road. You could do the same with electric bikes as well.

I am planning to visit Australia next year, to attend The ICAP conference, which has a traffic psychology section. Perhaps I'll see you there?

Gareth Rees said...

The truncated y-axis has a prominent place in Darrell Huff's How to Lie with Statistics but it doesn't deserve to be elevated to some kind of iron law. You can use y-axis truncation to mislead (by exaggerating insignificant differences), but you can also use y-axis extension to mislead (by minimising significant differences). The important thing is to make the scale clear.

Edward Tufte: "In general, in a time-series, use a baseline that shows the data not the zero point. If the zero point reasonably occurs in plotting the data, fine. But don't spend a lot of empty vertical space trying to reach down to the zero point at the cost of hiding what is going on in the data line itself. (The book, How to Lie With Statistics, is wrong on this point.)"

Theo van Soest said...

Hi Ian

Recently I stumbled over your very interesting finding again. I still want to replicate these findings here in the Netherlands.
The comments of Dan are a bit overdone in my opinion. However, the discussion points to some intersting findings: cultural differences in traffic behavior (in the Netherlands you would be attacked when occupying ("controlling") a complete lane. Also the findings of Dan leave me with some methodic questions:
-Clearly, there is a secend cyclist holding the camera. This surely influences the experiment.
-In a dual lane road with only moderate traffic there is a lot of opportunity to switch lanes and therefore will lead to more space when overtaking.

You also point out that not only the average overtaking distance is relevant but the distribution in these data is also. I think this is good to clarify.

Ian Walker said...

Theo,

I've cycled in the Netherlands and so have some idea of what it's like there. If you want to do a study that would be great - do let me know if I can help in any way.

Ian

KelseyMusic said...

I've been driven at by a small number of psycho drivers over a matter of years - and wasn't wearing a helmet - some people have no morals - they don't care who they try to bully or intimidate - they are just plain nasty people - good job I can report them to the police for a start.

pigrao said...

I'm not a statistician (I'm a mere mathematician) but I hear from some colleagues that double-blind is fundamental in collecting such data. The suggestion here is to conduct both Ian's and Dan's data collection activity on a much larger scale. Possibly without the knowledge of the cyclists. It will take some creativity to design this experiment, but I believe that is the only way to detach emotions from cold statistical data gathering.

pigrao said...

I'm not a statistician (I'm a mere mathematician) but I hear from some colleagues that double-blind is fundamental in collecting such data. The suggestion here is to conduct both Ian's and Dan's data collection activity on a much larger scale. Possibly without the knowledge of the cyclists. It will take some creativity to design this experiment, but I believe that is the only way to detach emotions from cold statistical data gathering.

Peter said...

Ian,
some people have recently expressed a need for your overtaking research to be replicated here in Australia where the helmet debate is heated and constant and your research is quoted quite a bit.
Would you be able to give us an idea how to go about it and what is involved technically.
Peter. (robinson(at)nex.net.au)

Milady said...

re: Peter's comment I'd like to know what equipment was used and how to replicate it. It would be interesting to see if different parts of say, London, had different data (collected over a long time perhaps). and perhaps in relation to weather too. Anyway, just a vague idea. Love the blog, just found it and will wack it on my google reader. love the very english use the word 'moan'

Sam said...

Hi Ian,

I'm sure you've heard about the Florida DOT study that reconfirmed your findings?
http://www.dot.state.fl.us/research-center/Completed_Proj/Summary_RD/FDOT_BDK82%20977-01_rpt.pdf

I'm pretty happy given that I'm female who rides in casual clothing :)