On Fair Performance Comparison between Random Survival Forest and Cox Regression: An Example of Colorectal Cancer Study

Random Survival Forest Cox Regression Machine Learning Brier’s Score Discordance Index Colorectal Cancer.

Authors

  • Sirin Cetin Department of Biostatistics, Faculty of Medicine, Tokat GaziosmanPasa University,, Turkey
  • Ayse Ulgen
    ayshe.ulgen@global.t-bird.edu
    Department of Biostatistics, Faculty of Medicine, Girne American University, Karmi,, Cyprus
  • Isa Dede Medical Oncology, Faculty of Medicine, Mustafa Kemal University, Antakya,, Turkey
  • Wentian Li The Robert S. Boas Center for Genomics and Human Genetics, The Feinstein Institutes for Medical Research, Northwell Health, Manhasset, NY,, United States

Downloads

Random Forest (RF), a mostly model-free and robust machine learning method, has been successfully applied to right-censored survival data, under the name of Random Survival Forest (RSF). However, RF/RSF has its distinct strategies in classification and prediction. First, it is an ensemble classifier and its performance is an average of multiple rounds of data fitting. Second, the training set is a bootstrap (sampling with replacement) generated set with repeated used of roughly 2/3 of all samples and testing set consists of those not used (out of bag samples). Both features are not intrinsic to Cox regression or other single classifiers. Not considering these two features could potentially lead to a partial comparison between the performance of the two methods. By using a colorectal survival dataset, we illustrate the problems of using k-fold cross-validation, using only one resampling without an ensemble average, and using the whole dataset for both fitting and testing, in Cox regression, when comparing with RSF. We provide a more accessible R code for simple calculation of discordance index (D-index) and unweighted integrated Brier score (IBS) for Cox regression, and unweighted IBS for RSF.

 

Doi: 10.28991/SciMedJ-2021-0301-9

Full Text: PDF