Facts are hares, but truth is a tortoise

Don’t believe every ‘study’ that comes your wayLast year, researchers at King’s College London reported that theobromine, a key component of cocoa, might have anti-ageing properties. In July this year, sports scientists at University of Malaya, Kuala Lumpur, said merely sniffing chocolate before a workout could help you do more reps. Might, could – nothing definite there, yet we lap up such “research”.It doesn’t matter when all you’re doing is nibbling on a harmless piece of chocolate, but what if the research or advice happens to be medical? Did you get a scare when Trump’s FDA last year said taking paracetamol while pregnant could increase your child’s risk of autism?We are drowning in research today, journalist Helen Pearson writes in her book Beyond Belief: How Evidence Shows What Really Works. In 2025 alone, some 7mn scholarly articles were published worldwide – over 19,000 a day, Pearson says. There could be hundreds of papers on the same subject, with differing conclusions. For example, some research shows an occasional glass of wine is good for you, while others deem even a drop of alcohol harmful. How do we arrive at the truth?The old and easy way was to count the research. If four papers said wine is unhealthy and five said it isn’t, you could read it as a licence to drink. But this was problematic. What if the five pro reports had sampled only five subjects, whereas the four anti reports had sampled 5,000? And often, even this “vote counting” wasn’t considered necessary.Pearson mentions a mistake in the most popular baby and child care book of all time that might have resulted in up to 50,000 deaths in US, Europe and Australasia. In the 1957 edition of the guide, she says, its author, Dr Benjamin Spock, changed his advice on putting babies to bed.They ought to sleep face down, rather than on their backs, he wrote. Why? Because another famous doctor, Paul Woolley Jr, had said so in a study published in 1945. What followed was a spate of sudden infant death syndrome or Sids. It wasn’t until 1991, after doing “studies of studies”, that govts strongly advised making babies sleep on their backs.Those studies of studies, or systematic reviews, Pearson says, are the basis of what we now call evidence-based medicine. They come in various forms – meta-analyses, living reviews (where the conclusion can change with emerging evidence), etc – and, surprise, they are only 35 years old.They’ve become so important that in 2019, there were 29,000 systematic reviews in the field of medicine alone. Thanks to them, we know that there’s no “clear link” between maternal paracetamol use during pregnancy and autism or ADHD in offspring.Systematic reviews are the ultimate test of any claim, be it in medicine or environmental science, or even management theory. For example, they show that “a lot of standard management practices – like performance appraisals – are not supported by strong evidence showing they help”.So, how do flimsy, unproven theories – like chocolate’s anti-ageing effect – go viral? It’s because systematic reviews take three years, on average. They’re costly too, and AI isn’t much help yet, because it can hallucinate and lead us to the wrong conclusions. So myths and fads continue flying. What are you reading this with, protein coffee or a turmeric-ginger shot?You use AI every day. Now get your AI Quotient. Take the AIQ test.