Are shark attacks and ice cream sales linked? Do drugs work? Being able to distinguish cause and effect is crucial. Now we have the maths to do it reliably
IN THE mid-1990s, an algorithm trained on hospital admission data made a surprising prediction. It said that people who presented with pneumonia were more likely to survive if they also had asthma. This flew in the face of all medical knowledge, which said that asthmatic patients were at increased risk from the disease. Yet the data gathered from multiple hospitals was indisputable: if you had asthma, your chances were better. What was going on?
It turned out that the algorithm had missed a crucial piece of the puzzle. Doctors treating pneumonia patients with asthma were passing them straight to the intensive care unit, where the aggressive treatment significantly reduced their risk of dying from pneumonia. It was a case of cause and effect being hopelessly entangled. Fortunately, no changes were rolled out on the basis of the algorithm.
Unweaving the true connection between cause and effect is crucial for modern-day science. It underpins everything from the development of medication to the design of infrastructure and even our understanding of the laws of physics. But for well over a century, scientists have lacked the tools to get it right. Not only has the difference between cause and effect often been impossible to work out from data alone, but we have struggled to reliably distinguish causal links from coincidence.
Now, mathematical work could fix that for good, giving science the causal language that it desperately needs. This has far-ranging applications in our data-rich age, from drug discovery to medical diagnosis, and may be the essential tool to resolve this fatal flaw.
A mantra most scientists can recite in their sleep …