Sunday, 1 March 2009

Science stories: the devil IS in the detail but you've got to look for it

One of my colleagues, Chris Ashwin, made a bit of splash in the news this last week from being involved in a study on the genetics of optimism and pessimism. Looking at the Guardian article, which the link takes you to, I saw something in the readers' comments which was painfully familiar:

100 people from God knows where isn`t really representative of humanity. Was is a nice day? How healthy were the volunteers? How old? What were they being paid? Its all in the detail folks, or is that my cynicism gene kicking in?

Oh goodness me, but aren't you clever? I saw an awful lot of this sort of thing when my bicycle overtaking work was being reported a couple of years ago: again and again people would write comments which essentially said "This three-paragraph newspaper report I've just read doesn't mention X. I can't believe this researcher didn't consider X! The whole thing is clearly bollocks!" An entire study is dismissed because some detail isn't immediately in front of the reader's nose. You can see this behaviour whenever science is reported in the media.

Of course these details matter in scientific research, which is why researchers write up their procedures with painstaking care in journal articles so the details are there for everyone to see, examine and judge. Typically, this sort of information runs to several closely-typed pages for any given study (I've looked at Chris's paper and the procedure and results run to over 1200 words). So why on earth do people expect to find the same level of reporting in a newspaper story or blog post? Do people think scientists spend years doing research and then, faced with the challenge of reporting their findings, dash off a quick press release in five minutes before moving on to the the next project? Or is it that they believe the details don't exist, and that teams of professional researchers manage routinely to overlook undergraduate-level design issues when doing their jobs?

It's great people are thinking about what they read and showing some critical thought, but can't they take the logical step and actually look for answers to their questions rather than simply assuming the answers don't exist because they're not in a newspaper report? Am I wrong to find this dismissal of people's efforts so irritating?


Anonymous said...

I'm glad you (a proper academic in a science discipline) said that.

You'll know that the direction of causality in human affairs is not always clear. For example do we see something and get indignant about it - or are we in a generalised mood of indignation (possibly with some cause of which we are
unconcious) and then fire that indignation at some fairly random thing which happens to catch our attention?

There's a great deal of *totally justified* indignation and anger about "bad science" (Homepathy and whatnot) on the web at the mo'. Ben Goldacre, for example, does a great job of talking intelligently about this.

But it's obvious from the comments and forums on Goldacre’s site that there are people who are not especially scientifically knowledgeable but whose comments are aping the legitimate indignation of the genuinely knowledgeable scientists and doctors who also comment on that site.

There can be a great deal of pleasure in getting angry and pissed – especially if you feel it’s totally justified and (Wow! Bonus!) shows how much cleverer you are than other people.

Ian Walker said...

Interesting point. Although I suspect the exact findings also matter: people will overlook glaring questions when they like the sound of the headline finding and will get picky when they don't.