EDITORIAL, Ganga Ram Journal, March-April 2013
Dicey clinical decisions and biostatistics
A. INDRAYAN
Department of Surgical Gastroenterology and Liver Transplant
Sir Ganga Ram Hospital, New Delhi
Flipping a coin and rolling a die are statistical favourites to generate uncertainty. All dicey events are inherently uncertain. But can some clinical decisions be just as uncertain? On the face of it, seems oxymoronic as the term ‘clinical’ also means ‘efficient and unemotional’1. If that is so, clinical decisions should not have much tolerance for uncertainties. In practice, clinical decisions and uncertainties go together.
Admittedly, dicey is not the word of choice in medical research literature. There are some examples, though. Path to lyme disease diagnosis is dicey2 and the role of Dicer expressions in RNAi therapy in cancer is dicey3. Nasal valve compromise can be dicey4 and neonatal withdrawal syndrome is clinically a dicey call5. This is just about all that I could locate in PubMed. It seems that most clinical researchers realize that clinical decisions are not so dicey after all although they are afflicted with uncertainties. Outside medicine, George Cockcroft’s work on making life decisions by rolling a die6 is a popular intellectual gossip.
Medical implications of dicey clinical decisions may be obscure but consider two patients with the same APACHE-II score, say 18 – one survives and the other not. Certainly the survival is not determined by flipping a coin as survival–death probabilities are not equal at APACHE-II = 18, neither it is determined by rolling a die with face probabilities equal to P(survival/APACHE-II = 18). Chance of death among operated cases of acute necrotizing pancreatitis is 1/6 and same as any particular number from 1 to 6 showing up on rolling a die. Fortunately, death is not as dicey. It surely depends on host factors to begin with and on the care one receives at the end of it all. Ironically, notwithstanding the respect it commands, APACHE-II is used to predict survival of a critical patient on the basis of physiological measurements at admission7 as though hospital facilities and critical care are irrelevant! Yet, it has great value in comparing the severity of cases in two or more groups or over time. APACHE-II is actually meant to compare the efficacy of two or more therapies for the patients of same degree of severity but has come to be widely used to prognosticate mortality. Incidentally, this score is the by-product of a popular biostatistical method – logistic regression. The associated uncertainties just illustrated underscore the need to be extra careful with such a method. Statistical methods may look innocuous but they are dangerous tools in the hands of inexpert8. In fact experts too falter when the underlying requirements9 are overlooked and residual uncertainties are discounted.
Biostatistics methods claim to be equipped to handle chance as this science uses sampling errors and randomness as their backbone. Many do not realize that this constitutes only one component of uncertainty syndrome. While aleatory part can be reasonably quantified and handled through statistical methods, my worry has been epistemic uncertainties8. Belonging to unknown domain, epistemic uncertainty lends us into real dicey zone so that some decisions remain no better than rolling a die. The only difficulty is that many of us do not realize that this so.
Biostatisticians are notorious for extracting information from even random numbers. Random numbers also follow a pattern, called uniform distribution – thus amenable to all kinds of statistical manipulations and inferences. Howsoever random, measurement errors, we all know, mostly follow a Gaussian distribution, and capable of giving us valid conclusions. Random after all are not as random as they seem. On the other hand, numbers can always be assigned, and justified, to any graded phenomenon, and they may not contain any useful information as stand alone. For example, risk is perceptionally useful quantity yet may not provide exciting news. If a procedure has a success rate of 28%, is it good or bad? Isolation can be self-defeating. If you know that an alternative procedure has a rate of 35%, suddenly an inferential value is attached to 28% success in the other method10.
Coming back to popular scoring systems such as APACHE and Glasgow, how much do they help in removing uncertainty? For the type of critical cases covered by these scores, the chance of survival is just about 80 to 85%. Based on this information, if I blindly predict as a non-expert and without the help of any scoring system that a critical case has a chance of 80% of survival, I would not be far wrong even in the long run where such probabilities are supposed to work. In that sense, acclaimed APACHE, for that matter any such scoring system, adds just about 5% to the accuracy of prediction. Imagine the cost in terms of obtaining accurate measurements and feeding them in to scoring system for this paltry gain. This probably is true for many medical tests. Clinical picture may already make you 70% confident of the diagnosis. Tests add just about 5 to 10%. But they are still preferred as they are objective and free of bias – and also help to shield from legal hassles. Perhaps the value of most biostatistical methods also lies in this protection. They assign scientific probabilities to the dicey clinical decisions. But statistical decisions can also be falsely positive or falsely negative just as are medical tests. A Type-I error rate of 5% is enough for ducking when a decision turns out …. If a statistical decision does not appeal to your conscience, re-examine it and the chances are that you will find your conscience was right8.
REFERENCES
1. Oxford Dictionaries (http://oxforddictionaries.com/definition/american_english/clinical, last retrieved 27 March 2013).
2. Pfeiffer MB. Exclusive: Dicey path to lyme disease diagnosis – Poughkeepsie Journal, 22 October 2012. (http://www.poughkeepsiejournal.com/article/20121022/NEWS06/112170001/EXCLUSIVE-DICEY-PATH-LYME-DISEASE-DIAGNOSIS, last retrieved March 28, 2013)
3. Merritt WM, Bar-Eli M, Sood AK. The dicey role of Dicer: Implications for RNAi therapy. Cancer Res 70: 2010:2571-4. (http://cancerres.aacrjournals.org/content/70/7/2571.long, last retrieved 15 March 2013).
4. Collins TR. A clinical challenge: Nasal valve compromise can be a dicey problem, panelists say. ENT Today, February 2011 (http://www.enttoday.org/details/article/1006491/A_Clinical_Challenge_Nasal_valve_compromise_can_be_a_dicey_problem_panelists_say.html, last retrieved 28 March 2013).
5. Cohen LS. Neonatal withdrawal syndrome and SSRIs. Healthy Place – Mental illness overview (http://www.healthyplace.com/other-info/mental-illness-overview/neonatal-withdrawal-syndrome-and-ssris/, last retrieved 29 March 2013).
6. Coverdale K. Dicey Decisions (http://www.vice.com/read/dicey-decisions-0000191-v19n6, last retrieved 29 March 2013).
7. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: A severity of disease classification system. Crit Care Med 1985; 13:818-29 ( http://xa.yimg.com/kq/groups/16749867/1433507048/name/Critical+Care+Medicine+1985+Knaus.pdf, last retrieved 15 march 2013).
8. Indrayan A. Medical Biostatistics, Third Edition. CRC Press, New York, 2012 (http://www.crcpress.com/product/isbn/9781439884140, last retrieved 29 March 2013).
9. Kumar R, Indrayan A, Chhabra P. Reporting quality of multivariable logistic regression in selected Indian medical journals. 2012; 58:123-6 (http://www.jpgmonline.com/article.asp?issn=0022-3859;year=2012;volume=58;issue=2;spage=123;epage=126;aulast=Kumar, last retrieved 29 March 2013).
10. Ubel PA. Beyond costs and benefits: Understanding how patients make health care decisions. The Oncologist 2010; 15 (Suppl 1):5-10 (http://theoncologist.alphamedpress.org/content/15/suppl_1/5.full, last retrieved 29 March 2013).
Dicey clinical decisions and biostatistics
A. INDRAYAN
Department of Surgical Gastroenterology and Liver Transplant
Sir Ganga Ram Hospital, New Delhi
Flipping a coin and rolling a die are statistical favourites to generate uncertainty. All dicey events are inherently uncertain. But can some clinical decisions be just as uncertain? On the face of it, seems oxymoronic as the term ‘clinical’ also means ‘efficient and unemotional’1. If that is so, clinical decisions should not have much tolerance for uncertainties. In practice, clinical decisions and uncertainties go together.
Admittedly, dicey is not the word of choice in medical research literature. There are some examples, though. Path to lyme disease diagnosis is dicey2 and the role of Dicer expressions in RNAi therapy in cancer is dicey3. Nasal valve compromise can be dicey4 and neonatal withdrawal syndrome is clinically a dicey call5. This is just about all that I could locate in PubMed. It seems that most clinical researchers realize that clinical decisions are not so dicey after all although they are afflicted with uncertainties. Outside medicine, George Cockcroft’s work on making life decisions by rolling a die6 is a popular intellectual gossip.
Medical implications of dicey clinical decisions may be obscure but consider two patients with the same APACHE-II score, say 18 – one survives and the other not. Certainly the survival is not determined by flipping a coin as survival–death probabilities are not equal at APACHE-II = 18, neither it is determined by rolling a die with face probabilities equal to P(survival/APACHE-II = 18). Chance of death among operated cases of acute necrotizing pancreatitis is 1/6 and same as any particular number from 1 to 6 showing up on rolling a die. Fortunately, death is not as dicey. It surely depends on host factors to begin with and on the care one receives at the end of it all. Ironically, notwithstanding the respect it commands, APACHE-II is used to predict survival of a critical patient on the basis of physiological measurements at admission7 as though hospital facilities and critical care are irrelevant! Yet, it has great value in comparing the severity of cases in two or more groups or over time. APACHE-II is actually meant to compare the efficacy of two or more therapies for the patients of same degree of severity but has come to be widely used to prognosticate mortality. Incidentally, this score is the by-product of a popular biostatistical method – logistic regression. The associated uncertainties just illustrated underscore the need to be extra careful with such a method. Statistical methods may look innocuous but they are dangerous tools in the hands of inexpert8. In fact experts too falter when the underlying requirements9 are overlooked and residual uncertainties are discounted.
Biostatistics methods claim to be equipped to handle chance as this science uses sampling errors and randomness as their backbone. Many do not realize that this constitutes only one component of uncertainty syndrome. While aleatory part can be reasonably quantified and handled through statistical methods, my worry has been epistemic uncertainties8. Belonging to unknown domain, epistemic uncertainty lends us into real dicey zone so that some decisions remain no better than rolling a die. The only difficulty is that many of us do not realize that this so.
Biostatisticians are notorious for extracting information from even random numbers. Random numbers also follow a pattern, called uniform distribution – thus amenable to all kinds of statistical manipulations and inferences. Howsoever random, measurement errors, we all know, mostly follow a Gaussian distribution, and capable of giving us valid conclusions. Random after all are not as random as they seem. On the other hand, numbers can always be assigned, and justified, to any graded phenomenon, and they may not contain any useful information as stand alone. For example, risk is perceptionally useful quantity yet may not provide exciting news. If a procedure has a success rate of 28%, is it good or bad? Isolation can be self-defeating. If you know that an alternative procedure has a rate of 35%, suddenly an inferential value is attached to 28% success in the other method10.
Coming back to popular scoring systems such as APACHE and Glasgow, how much do they help in removing uncertainty? For the type of critical cases covered by these scores, the chance of survival is just about 80 to 85%. Based on this information, if I blindly predict as a non-expert and without the help of any scoring system that a critical case has a chance of 80% of survival, I would not be far wrong even in the long run where such probabilities are supposed to work. In that sense, acclaimed APACHE, for that matter any such scoring system, adds just about 5% to the accuracy of prediction. Imagine the cost in terms of obtaining accurate measurements and feeding them in to scoring system for this paltry gain. This probably is true for many medical tests. Clinical picture may already make you 70% confident of the diagnosis. Tests add just about 5 to 10%. But they are still preferred as they are objective and free of bias – and also help to shield from legal hassles. Perhaps the value of most biostatistical methods also lies in this protection. They assign scientific probabilities to the dicey clinical decisions. But statistical decisions can also be falsely positive or falsely negative just as are medical tests. A Type-I error rate of 5% is enough for ducking when a decision turns out …. If a statistical decision does not appeal to your conscience, re-examine it and the chances are that you will find your conscience was right8.
REFERENCES
1. Oxford Dictionaries (http://oxforddictionaries.com/definition/american_english/clinical, last retrieved 27 March 2013).
2. Pfeiffer MB. Exclusive: Dicey path to lyme disease diagnosis – Poughkeepsie Journal, 22 October 2012. (http://www.poughkeepsiejournal.com/article/20121022/NEWS06/112170001/EXCLUSIVE-DICEY-PATH-LYME-DISEASE-DIAGNOSIS, last retrieved March 28, 2013)
3. Merritt WM, Bar-Eli M, Sood AK. The dicey role of Dicer: Implications for RNAi therapy. Cancer Res 70: 2010:2571-4. (http://cancerres.aacrjournals.org/content/70/7/2571.long, last retrieved 15 March 2013).
4. Collins TR. A clinical challenge: Nasal valve compromise can be a dicey problem, panelists say. ENT Today, February 2011 (http://www.enttoday.org/details/article/1006491/A_Clinical_Challenge_Nasal_valve_compromise_can_be_a_dicey_problem_panelists_say.html, last retrieved 28 March 2013).
5. Cohen LS. Neonatal withdrawal syndrome and SSRIs. Healthy Place – Mental illness overview (http://www.healthyplace.com/other-info/mental-illness-overview/neonatal-withdrawal-syndrome-and-ssris/, last retrieved 29 March 2013).
6. Coverdale K. Dicey Decisions (http://www.vice.com/read/dicey-decisions-0000191-v19n6, last retrieved 29 March 2013).
7. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: A severity of disease classification system. Crit Care Med 1985; 13:818-29 ( http://xa.yimg.com/kq/groups/16749867/1433507048/name/Critical+Care+Medicine+1985+Knaus.pdf, last retrieved 15 march 2013).
8. Indrayan A. Medical Biostatistics, Third Edition. CRC Press, New York, 2012 (http://www.crcpress.com/product/isbn/9781439884140, last retrieved 29 March 2013).
9. Kumar R, Indrayan A, Chhabra P. Reporting quality of multivariable logistic regression in selected Indian medical journals. 2012; 58:123-6 (http://www.jpgmonline.com/article.asp?issn=0022-3859;year=2012;volume=58;issue=2;spage=123;epage=126;aulast=Kumar, last retrieved 29 March 2013).
10. Ubel PA. Beyond costs and benefits: Understanding how patients make health care decisions. The Oncologist 2010; 15 (Suppl 1):5-10 (http://theoncologist.alphamedpress.org/content/15/suppl_1/5.full, last retrieved 29 March 2013).