Misuse of ROC Curves in Prediction
Dr Abhaya Indrayan,
MSc, MS, PhD (Ohio State), FAMS, FRSS, FSMS, FASc
Former Professor & Head of Biostatistics at Delhi University College of Medical Sciences, Delhi
Biostatistics Consultant at Max Healthcare, New Delhi
Almost all papers on prediction models use the area under the ROC curve (C-index) as the gold standard for assessing the predictive accuracy of their models. However, this is the wrong index for this purpose. C-index is based on sensitivity and specificity which are measures of discrimination and classification of the KNOWN outcomes and the results could be very different when used for prediction of UNKNOWN future outcomes. Thus, the use of C-index for assessing prediction accuracy must be discontinued. Instead, positive and negative predictive values (PPV and NPV) based ROC curve should be used. PPV and NPV fully consider prevalence that sensitivity and specificity do not. See https://journals.lww.com/jopm/abstract/2024/70020/use_of_roc_curve_analysis_for_prediction_gives.8.aspx .
MSc, MS, PhD (Ohio State), FAMS, FRSS, FSMS, FASc
Former Professor & Head of Biostatistics at Delhi University College of Medical Sciences, Delhi
Biostatistics Consultant at Max Healthcare, New Delhi
Almost all papers on prediction models use the area under the ROC curve (C-index) as the gold standard for assessing the predictive accuracy of their models. However, this is the wrong index for this purpose. C-index is based on sensitivity and specificity which are measures of discrimination and classification of the KNOWN outcomes and the results could be very different when used for prediction of UNKNOWN future outcomes. Thus, the use of C-index for assessing prediction accuracy must be discontinued. Instead, positive and negative predictive values (PPV and NPV) based ROC curve should be used. PPV and NPV fully consider prevalence that sensitivity and specificity do not. See https://journals.lww.com/jopm/abstract/2024/70020/use_of_roc_curve_analysis_for_prediction_gives.8.aspx .