Pandemic, Infodemic: New Scientist on how to spot suspect science #2

If the covid-19 outbreak has taught us anything, it is the fearful vulnerability of our cyberspace to the postings of every crank, fool and sinister manipulator who has a keyboard, and the difficulty of separating their lies and hidden agendas from honest error, rightful emotion- and tiny fragments of genuine truth. It will not go away after covid-19. It is also a problem in politics, economics. health and other fields which have a direct bearing on human life.

We do not advocate the abolition of a universal Right to Post, nor do we call to the restriction of discourse to those with proven expertise in a given field. But advocates of the Free Speech Defence should recall that the freedom granted by a driving licence is not the same as the freedom to drive fast down the wrong carriageway of major trunk roads, nor to ignore red traffic lights.

Until a happier day when these dilemmas might be resolved, we have pleasure in advocating this checklist compiled by Graham Lawton of New Scientist on how you can recognise studies that might be poor, misleading-or worse.

Study is published on a blog, preprint server, or social media

It hasn’t been peer reviewed by other experts yet. Two heads are better than one .All the best science goes through a long process whereby other experts in the field look at it. In science this is called peer review. In marriage, it is called health and safety. How would you get on if you blew the family savings on a new Rolls Royce without telling your wife first?

Study only has one author

It’s probably very early research, and very tentative. Once again, two heads are better than one.

The researchers are from a surprising field of study

Perhaps they are geologists talking about viruses. Why would they suddenly do that, who’s paying them? Would you take advice from a cricketer about how to play poker? Maybe- but a poker player might be better.

The analysis is very fast

It’s never a good idea to go too fast in science and engineering. Older readers will remember the Comet airliner, which should have been a world beater, but kept crashing because it was rushed into service too quickly

The study is very small

This is common sense- but Graham’s rule of thumb is that any medical study with less than 50 participants is highly questionable-and we agree. Any football team can win one match. Champions are proved by a whole season

The trial has no placebo group

If you test a drug you give samples of it to a group of 100 people. You also give pills that look and taste exactly like the drug to a second group called the placebo group. Only this way can you be sure your drug is doing anything at all. If there is no placebo group, then the researchers are on very shaky ground indeed

The study reports a correlation or association

We did this ! Correlation is not proof-remember the sharks and ice cream?(Learning, Science and Society 23 April) It’s often telling you something, but it may not be what you think it is. So be advised.

#newscientist #grahamlawton

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s