In support of his view that overconfidence could be endemic in medicine, Nobel laureate Daniel Kahneman wrote: ‘Experts who acknowledge the full extent of their ignorance may expect to be replaced by more confident competitors who are better able to gain the trust of clients’ (Kahneman, 2011: 263).
Let us consider how statistics might be used to gain the trust of some … at the possible cost of losing the trust of others. Take, for example, Professor Sir David Spiegelhalter, who addressed a claim made in 2015 by the International Agency for Research in Cancer (IARC) that consuming 50 g of processed meat daily was associated with an 18% increased risk of bowel cancer (Spiegelhalter, 2019: 31).
The 18% refers to a relative risk (RR), representing the increased risk of getting bowel cancer between a group of people who consume 50 g of processed meat daily, and a group who don't. But it ignores absolute risk (AR), ‘which means the change in the actual proportion in each group who would be expected to suffer the adverse event’ (Spiegelhalter, 2019: 32). Thus, normally around six in 100 people (6%) who don't eat processed meat would contract bowel cancer. However, according to the IARC, if 100 similar people ate 50 g of processed meat daily, 18% more would get bowel cancer. An 18% relative increase over 6% is 6% x 1.18 = 7.08%; i.e. one extra case of bowel cancer in 100 lifetime processed meat eaters. This one extra case, observes Spiegelhalter (2019), ‘does not sound so impressive as the relative risk …’
In the context of statins, the title of a paper by Diamond and Ravnskov (2015) aims to show ‘How statistical deception created the appearance that statins are safe and effective in primary and secondary prevention of cardiovascular disease.’ They hypothesise a 5-year trial of 2000 healthy, middle-aged men to see if a statin can prevent heart disease. Half of the participants receive the statin and half receive a placebo.
Typically, during a 5-year period some 2% of healthy, middle-aged men experience a nonfatal myocardial infarction (MI). So, at the end of the hypothetical trial, 2% of the placebo-treated men and 1% of the statin-treated men suffered an MI.
Statin treatment, therefore, has benefited 1% of the treated participants. Thus, with statin treatment the absolute risk reduction (ARR) was one percentage point, and the number needed to treat was 100, with 1 in 100 benefiting. Alternatively, the chance of NOT having an MI during the 5-year period without a statin was 98% and 99% with a statin. However, when it comes to presenting this hypothetical trial's results to the public, instead of using the ARR to show a 1% improvement, the researchers use a relative risk reduction (RRR) since ‘statin treatment reduced the incidence of heart disease by 50%, because 1 is 50% of 2’ (Diamond and Ravnskov, 2015).
As a meta-analysis undertaken by Byrne et al (2022) concluded, ‘the absolute risk reductions of treatment with statins in terms of all-cause mortality, myocardial infarction, and stroke are modest compared with the relative risk reductions, and the presence of significant heterogeneity reduces the certainty of the evidence.’
While Gigerenzer et al (2010) observed that reporting RRs without baseline risks is favoured by journalists who prefer bigger numbers for bigger headlines, they also concede that the source ‘seems to be medical journals, from which figures spread to press releases, health pamphlets, and the media.’ This headline-grabbing type of approach to the presentation of data, however, is not perpetrated on an occasional basis. Diamond and Leaverton (2023) have recently shown that, ‘[t]he practice of expressing the RR without the AR has become routinely deployed in the reporting of findings in many different areas of clinical research.’
The authors have provided a historical perspective on the commonplace nature of such data presentation of the results of randomised controlled trials (RCTs) on coronary heart disease event monitoring and prevention over some 40 years. Indeed, they explain that an undue emphasis on RR combined with far from full disclosure of AR when reporting RCT outcomes ‘has led healthcare providers and the public to overestimate concerns about high cholesterol and to be misled as to the magnitude of the benefits of cholesterol-lowering therapy.’
As Mark Twain warned: ‘Facts are stubborn things, but statistics are pliable.’