Lies, damned lies, and statistics.

Attributed to Mark Twain, about Disraeli, this phrase has received considerable use, in decrying statistics as a means of bolstering an argument, and indeed, sometimes in the support of one. However, when statistics are used, in the support of a drug or treatment in Medicine, or a lifestyle change or recommendation, we are into a different 'ballgame' altogether. It is best policy then to treat them with healthy scepticism.

The The Jobbing Doctor recently referred to the increase in oesophageal cancer, from an anecdote about his own practise. This is in fact borne out by the statistics. In the period between 1995 and 2008 there has been an increase of nearly 50%! But that only means, an absolute risk of 14.4 persons per 100,000 as opposed to the previous 8.8 persons per 100,000. That's no consolation to those with adenocarcinoma of that organ, but the risk is still quite low. And that brings me to the crux, of my railing against, studies that prove very little, but grab headlines and frighten and confuse the populace by mixing up relative and absolute risk, often to sell us a drug, a lifestyle change or a treatment protocol that has little to no effect on survival, or the progress of an illness.

There has been much publicity, about the trials of various drugs recently, highlighting the pro's and the con's (literally) of their worth. The use of statin's, for example, was initially believed to be a panacea for low risk (should that be no risk?) patients in the prevention, of future heart disease. But this has been clearly shown to be of little benefit, whilst at the same time as increasing risks for other diseases such as diabetes. The use of statin's in primary prevention is now pretty much a 'dead parrot', but will GP's stop prescribing them for the achievement of a highly dubious surrogate end point? Well, err, no, because they are paid to achieve them by the QOF (Quality Outcomes Framework). And let not the science, get in the way of a nice little earner!

Trials, generally have been the basis of evidenced based Medicine since 1946, when Austin Hill designed the 'randomised control trial', for tuberculosis treatment with antibiotics, which were in their infancy. I suppose it was easier then, because there was not much in the way of groundbreaking science or drugs around, but as 'Pharma' became bigger and richer, the stakes got much higher, and the opportunities for 'massaging' the results became ever easier, especially once the baffling science of statistics, became a science in itself. So we are today, beset by statistics, that are 'mangled', to produce the right outcome, for a drug, a lifestyle or dietary change, or the cessation of something, that may be vaguely enjoyable.

Until only relatively recently, there was no compulsion to reveal all the results of a particular trial, but now 'Pharma' is compelled to do so, bringing to light a number, that showed particular drugs, to be less effective than previously thought. 'Pharma' had been guilty then, of 'hiding' from public (and other scientists) view, those trials that showed some drugs, had little to no effect, or did harm. This is termed 'selection bias' and has long been used to influence outcomes, in favour of that which was needed to justify, a particular protocol or drug's use. But, random chance, has a part to play, and as most RCT's are designed to ensure at least a 5% success ratio, then 1 in 20 completely useless one's will register as positive. These and others, are those they did not wish us to see, but no longer, they are now compelled to show all.

Another trick, often used, especially when, you can hide the data behind a 'paywall' such as the BMJ, or 'The Lancet' or many of the other journals, is to mix up relative and absolute risk within the 'abstract'. For example; I can double your chance of winning the Lottery. How; buy another ticket! Your relative chance has doubled, but only to 2 in 14 million, your absolute chance. If one extrapolates that into scientific study, as is often done, a somewhat distorted view, is given to the unsuspecting reader (journalist usually). As I have also said, many times, in my posts, correlation does not prove causation. And, as many trials are observational they simply correlate facts about the cohort under observation. The whole of the vegetarian revered book, 'The China Study', was no more than a complex and detailed correlation of observed data. Denise Minger wrote an extremely well researched and devastating critique of that testament to a vegan diet, that blew it out of the water, in my view.

This co-relationship (correlation) between several factors can be co-incidental. For example; many fat people are diabetic, but not all are, and in fact quite a few thin people are also diabetic. But it is not an absolute fact that all diabetics are fat; unless of course you ply them with drugs from 'Pharma', when they almost certainly, will get fat. That was a hypothesis, by the way, borne out by a considerable amount of evidence, and more importantly, the proof implicit, in the biological mechanism of insulin, which is generally enhanced by hypoglycemic drugs, of being an agent of fat storage.


An example of how society has been hoodwinked by both 'Big Pharma' and the GP's is the ever lowering of blood pressure targets, both for those at any risk, and those at 'allegedly' known risk, ( people with a history of CHD/CVD), who are prescribed anti-hypertensives of one kind or another. This is yet another 'surrogate end point', viewed as a possible marker for heart disease, but not actual heart disease. Well, the QOF for prevention, is for GP's to prescribe these appalling drugs, to lower blood pressure, in the targeted cohort, for which they get paid, on a 'payment by results' system. However, it turns out that the QOF for this had no discernible effect at all, on outcomes for the lowering of BP in patients being treated in primary care. There is even an implication, if you click on the whole study (amazingly free), that other 'payment by results' treatment protocols, for other diseases of society are also not effective, except for the pay of GP's. Some Doctors say, that they were already achieving high levels of compliance prior to QOF's institution, so these rewards are simply retrospective. I think that's a cynical manipulation of the facts to suit events. Either way, it's hardly value for money, or indeed of any real use to the patient, who is taking medication to fulfill a 'tick' list, often at cost to their well being.

Have then Doctors, taken the money, despite their efforts having produced no tangible result, or is the whole exercise pointless anyway? Because, further study will show that although the various drugs do lower BP to an extent, in some, but not all patients, the outcomes for the cohort taking them remains unchanged. Exactly the same number die, whether they take drugs or not. This review from Cochran highlights the inadequacies of both the studies and interventions in use, and is pretty damning, in its conclusions.

So, long suffering reader, I would urge scepticism in all data, that is presented to you as 'proof' of anything, especially a drug or treatment, that has been provided as a preventative measure by your 'hard pressed' GP. Was the study an RCT? Was it reproduced in a number of trials ( at least three), with a significant time scale (at least a year), for each? Was it interventional (the drug or device/protocol being designed to change an outcome)? Was it 'blinded' (the cohort and the trial personnel have no knowledge of who is getting what)? Was it longitudinal (over a long time) and if so the time must be long and the cohort large, such as the Framingham Study (60 years). This latter study, interestingly, has been criticised by many as both observational, self reported for many of its facts (by the cohort) and that much of the adverse data was suppressed. However, Michael Eades found that it was reported, just not very prominently.

Finally, be a sceptic, and believe only that, which is proven by real science, not Daily Mail headline drivel, or advocacy research, or perhaps worse, what your Doctor tells you (if it's QOF'ed).