“Lies, damned lies, and statistics.” Benjamin Disraeli

20 01 2009

This variously attributed quote was most probably made originally by Benjamin Disraeli. His point was to highlight the use of numbers in an argument, the further use of statistics to bolster weak arguments and then the ultimate disparagement of statistics themselves, particularly if those statistics don’t fit the correct side of the argument.

The debate over the MMR vaccine sprung from one single scientific paper that spawned a million column inches, months of debating hours and possibly even the death of some children. Everyone has an opinion. Yet eight years later the study has been totally discreditied and Dr Wakefield, the first author has been struck off for research “issues”.

Have you actually read the paper? Did you know there were only 12 apparently random children in the study? And that only 8 had the histological change that allegedly was the “cause” of autism itself becoming clinically apparent within 6 days of the vaccination. Now, what was your opinion on MMR vaccination again?

Currently, the hot topic on the lips and legal pads of medical managers everywhere is “The Surgical Checklist.” Everyone has heard about it particularly how a simple list is hugely reducing surgical complication and even death rates across the world.

Have you actually read the paper?

What do you think about the potential for bias in the data collection (see results, not method)?

How many patients actually had the checklist applied?

What does the “checklist” actually comprise of?

Save for the list, are there any other particular changes to care that may have affected the outcome?

How do you think a simple surgical checklist on its own actually reduces the rate of complications such as myocardial infarction, pulmonary embolus and ventilator requirement? Or is it perhaps something else affecting results?

Importantly for hospitals trying to prevent “wrong side surgery”, what evidence does the paper provide for reduction in this problem?

Is the reduction in death rate statistically significant?

What is the Hawthorne effect?

(I’m not sure it’s clear but what economic area does it suggest UK, Canada and New Zealand are in?)

The newspapers are particularly hot for this topic not the least because of the highlighted resistance of surgeons to this change. The fire of this debate is fuelled additionally by the age old antagonisms and prejudices particularly of managers and patient groups.

A small sample of that debate can be found in the comment section here. It is not supportive of the medical profession. Perhaps we could hand out some ladders of inference to help calm things?

Now read just about any news article or pronouncement from management on the issue. Are the changes to be instituted  justified by the vigour and results of the study? Can you understand why there may be some antagonism between the two groups? If you were (are) a manager, how would you approach the situation?

Now I’m not against improvements in clinical care or checklists or debate, I’d just like it a bit better informed.

Lies, damned lies or statistics, you decide.

%d bloggers like this: