Sunday 10 January 2016

Identifying Dodgy Research - part 1

Lets attack this problem in pieces. Firstly, are there particular specialties that are more likely to be involved? Yes. I am not the first to notice that anaesthesia is grossly over-represented in the list of culprits.

The Scientific American in 2009 referred to anesthesiologist Scott Reuben as "A Medical Madoff", who faked data in 21 studies.
As noted by Retraction Watch, this number of withdrawn studies was eclipsed by Joachm Boldt. The current record holder is Yoshitaka Fujii, who is reported by Retraction Watch to have fabricated his results in at least 172 published studies.

Anesthesiologists "have an absolutely horrifying track record in terms of retractions," according to a researcher who studies ethics quoted by Retraction Watch. You would think that the profession would get its house in order after the first scandal, but no, that didn't happen.
Also of note is the extreme persistence of the complainants in these cases, in the face of a serious reluctance by anaesthetists to criticise their colleagues. It is likely that many complainants have not been so doggedly persistent and have let dodgy research remain on the books for the sake of their careers.
Anaesthetists comprise only a small percentage of medical specialists, yet the retractions from just these three are far more than proportional.

Is the problem now under control? I think drugs in sport would be an appropriate metaphor here. The bureaucrats say they have testing and policies against it, yet the truth is quite different.

Noted anaesthesia academic Dan Sessler, with whom I have discussed research, is more concerned with widespread minor misconduct, rather than outright fraud. Some of his particular concerns include unreported data selection, and outcomes and hypotheses which are not specified in advance.
He believes that the likelihood that these are more common than fabrication means they probably contribute more to scientific error.

There is indisputable evidence that the problems noted by Sessler are still occurring here, and that Australian research regulators are doing little to stamp out the practice. The RACP has declined to comment on the evidence. (Yes, RACP. It is not just an anaesthesia problem.)

Next time you read a scientific paper, check whether there is a statement that all data has been reported on, or an explanation given for why it was not.