How to read health news without being misled

One week, a study links coffee to a longer life; the next, another links it to heart trouble. Red wine protects, then harms. A new blood test "detects cancer years early". Most of these headlines are not lies — they accurately report something a study found. The problem is what the study could and could not show. A handful of questions, asked consistently, separate the findings worth acting on from the ones that are merely interesting.

Association is not causation

Most health headlines come from observational studies: researchers follow large groups and look for links between what people do and what happens to them. Such studies are invaluable for generating ideas and for detecting rare harms. But they cannot easily separate the effect of the exposure from everything else that differs between the groups — a problem called confounding. People who take vitamins, eat salads or drink moderately also tend to be wealthier, better educated and more health-conscious.

The cautionary tale is hormone therapy after menopause. A large body of observational studies suggested that women taking hormones had a 40–50% lower risk of coronary heart disease. Then the Women's Health Initiative randomly assigned 16,608 women to oestrogen plus progestin or placebo. Hormones did not prevent heart disease; the risk was higher (hazard ratio 1.29). Per 10,000 women per year, there were 7 more coronary events, 8 more strokes, 8 more pulmonary embolisms and 8 more invasive breast cancers — against 6 fewer colorectal cancers and 5 fewer hip fractures.

Only a randomised controlled trial (RCT), in which chance alone decides who gets the treatment, reliably balances both known and unknown differences between groups. In news reports, words such as "linked to", "associated with" or "tied to" are the signal that you are reading about an association.

Surrogate endpoints: when the number improves and the patient doesn't

Many studies measure a surrogate endpoint — a lab value or measurement assumed to predict health — rather than a hard endpoint such as death, heart attack, stroke, fracture or quality of life. Surrogates are faster and cheaper to study. Sometimes they mislead badly.

• CAST (1989). After a heart attack, frequent extra heartbeats predict sudden death. The drugs encainide and flecainide suppressed these beats very effectively. In a randomised trial, total mortality over an average of 10 months was 7.7% with the drugs versus 3.0% with placebo (56 of 730 vs 22 of 725 patients). The drug arm was stopped.
• Torcetrapib (2007). Low HDL ("good") cholesterol predicts heart disease, so raising it seemed logical. In the ILLUMINATE trial of 15,067 high-risk patients, torcetrapib raised HDL by about 34 mg/dL and lowered LDL by about 21 mg/dL — and increased major cardiovascular events (6.2% vs 5.0%) and deaths (1.2% vs 0.8%). It also raised blood pressure.
• Rosiglitazone. This diabetes drug lowered blood sugar, the surrogate on which diabetes drugs were approved. A 2007 meta-analysis of 42 trials with about 28,000 patients found a higher risk of heart attack (odds ratio 1.43). In 2010 the European Medicines Agency recommended suspending it, concluding that its benefits no longer outweighed its risks.

A better number is not the same as a better life. Ask what the study actually measured.

Mice, cells and test tubes

"Compound kills cancer cells" usually means it did so in a dish or a mouse. Such work is essential basic science, but it is a long way from a treatment. When researchers followed 76 highly cited animal studies, only 37% were later confirmed in human randomised trials; 18% were contradicted, and 45% had never been tested in humans at all. Many substances kill cancer cells in a dish — including ones that are also toxic to healthy cells.

Preprints, small samples and single studies

A preprint is a manuscript posted online before peer review. Preprints speed up science, but their conclusions can change substantially — or the work may never be published. News reports should say clearly when a study is a preprint.

Small studies are more vulnerable to chance and tend to produce exaggerated effects that shrink when larger studies follow. Trials stopped early because of an unexpectedly good result can overstate benefit for the same reason.

Above all, one study is rarely decisive. The most reliable answer usually comes from a systematic review that searches for and appraises all relevant trials, as in the Cochrane reviews discussed in our article on general health checks. A single striking result that contradicts a large body of evidence is more likely to be wrong than revolutionary.

Who paid, and how are the numbers framed?

Industry funding does not make a study wrong, and many essential trials are industry-funded. But a Cochrane methodology review found that industry-sponsored studies more often reported favourable efficacy results (risk ratio 1.27) and favourable conclusions (risk ratio 1.34) than independently funded ones — a bias not explained by standard risk-of-bias assessments. Look for the funding and conflict-of-interest statements.

Finally, check how results are expressed. A "50% reduction" can mean 2 in 1,000 instead of 4 in 1,000; analyses of press releases have found that many report relative risks without absolute numbers. See our guide to absolute and relative risk. And when screening is promoted with improved survival rates, remember lead-time bias and overdiagnosis.

A checklist: eight questions for any health story

1. What kind of study is it? Cell or animal study, observational study, randomised trial or systematic review?
2. Association or cause? If observational, what else could explain the link?
3. What was measured? A hard outcome (death, heart attack, fracture, symptoms) or a lab value?
4. How big is the effect in absolute terms? Out of how many people?
5. How many people were studied, and for how long?
6. Was it peer-reviewed? Or is it a preprint or a conference abstract?
7. Who funded it, and do the authors have conflicts of interest?
8. Does it fit the wider evidence? What do systematic reviews and guidelines say?

Reliable sources in German

For readers in Germany, Austria and Switzerland, three independent sources stand out:

• gesundheitsinformation.de is published by IQWiG, the Institute for Quality and Efficiency in Health Care, an independent scientific institute financed by statutory health insurance. Providing understandable health information is part of its statutory mandate, and the site carries no advertising.
• Medizin transparent is a project of Cochrane Austria at the University for Continuing Education Krems. It checks health claims from media and advertising against the evidence, and readers can submit questions free of charge.
• The Harding-Zentrum für Risikokompetenz (Faculty of Health Sciences Brandenburg, University of Potsdam) publishes fact boxes that show benefits and harms of medical measures in absolute numbers.

This article is general information and not individual medical advice. Decisions about tests and treatment belong in a conversation with your own physician.