Friday, 5 February 2016

How do you expect to manage diabetes well if you don't reduce errors?

Q: Why is Type 2 error called Type 2 error?
A: Because it is an error, a mistake, it is the wrong thing to do.
Doctors keep talking about "good" evidence. There is no such thing, when asking what is the best way to manage diabetes. It only makes sense in a situation where there are only 2 options. Evidence can have a low probability of Type 1 error, or a low risk of Type 2 error. Receiver-operator theory says it cannot simultaneously minimise both.

 It also says that, given a fixed amount of information, decreasing Type 1 error will inevitably increase Type 2 error. Receiver-operator theory is one of the fundamental mathematical laws that governs sufficiently complex situations in the universe, which certainly applies to optimization of diabetes care.

It is not hard to understand in qualitative terms. I could base my diabetes care on all the data available. Or I could chose to base it on randomized trial evidence. If I chose the latter I would reduce Type 1 error. I would not be on useless and quite possibly harmful treatments. But on the other hand, I would increase Type 2 error. I would not be on treatments that were actually effective. I would have rejected useful treatments because my evidence filter was too restrictive.

The default medical practice of minimising Type 1 error is entirely appropriate if you are considering a treatment that is difficult, dangerous, expensive or time consuming.
On the other hand, insisting on an RCT before you go to, for example, yoga, is just silly. Look at all the successful T1s who have done yoga. Try to find anyone who says that yoga made their diabetes worse.

Now consider the alternative. How much will your diabetes suffer because you choose not to do yoga? If you are a Type 1 error reduction zealot you cannot answer this question because you reject all practical evidence that could be used to estimate the magnitude of the effect. A more sensible person may be quite happy with an answer based on an easily performed case control study.

Of course yoga is just one example of something that may help diabetes control. What are others? If you are the sort of person who bases their treatment only on randomised trials, you won't even have any idea what they are, because most potentially useful strategies will never be the subject of an RCT in your lifetime.

If you understand ROC, you could ask your doctor how they reduce Type 2 error. They may answer that they look at the sample size of RCTs, because that is a definition they learned from one of their textbooks to pass their exams.  Memorising is not understanding. Doctors can pass exams by learning a definition of study type, but show them an actual study, and many can't tell you what type it is.
Laugh at them by all means, but I wouldn't look for a new doctor on that basis because the next one will be no better. Learn enough to make your own decisions.