Application of machine learning models for survival data with non-proportional hazard and case study_Chinese Journal of Evidence-Based Medicine

Authors：

CHEN Haoran ^1,3 , LIU Xiayang ² , WANG Min ¹ , YANG Lin ¹ , WANG Jiayang ^1,3 , SUN Haixia ¹ , DUAN Yongheng ² , WU Xusheng ² , SHANG Li ² , QIAN Qing ^1,3 ,  HE Xiaofeng ² ,  LI Jiao ^1,3

1. Institute of Medical Information, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing 100020, P. R. China;
2. Shenzhen Health Development Research and Data Management Center, Shenzhen 518028, P. R. China;
3. National Population Health Data Center, Chinese Academy of Medical Sciences, Beijing 100730, P. R. China;

Corresponding author：

HE Xiaofeng, Email: 0912hexf@163.com; LI Jiao, Email: li.jiao@imicams.ac.cn

Keywords：

Survival analysis; Non-proportional hazards; Machine learning; Statistical methods

DOI：

10.7507/1672-2531.202401190

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To summarize and explore the application of machine learning models to survival data with non-proportional hazards (NPH), and to provide a methodological reference for large-scale, high-dimensional survival data. Method Firstly, the concept of NPH and related testing methods were outlined; then the advantages and disadvantages of machine learning algorithm-based NPH survival analysis methods were summarized based on the relevant literature; finally, using real-world clinical data, a case study was conducted with two ensemble machine learning models and two deep learning models in survival data with NPH: a study of the risk of death within 30 days in stroke patients in the ICU. Results 8 commonly used machine learning model-based NPH survival analyses were identified, including five traditional machine learning models such as random survival forest and three deep learning models based on artificial neural networks (e.g., DeepHit); the case study found that the random survival forest models performed the best (C-index=0.773, IBS=0.151), and the permutation importance-based algorithm found that age was the most important characteristic affecting the risk of death in stroke patients. Conclusion Survival big data in the era of precision medicine presenting NPH is common, and machine learning model-based survival analysis can be used when faced with more complex survival data and higher survival analysis needs.

1.	王佩佩, 双卫兵. 生存分析概述及模型应用. 中国医学工程, 2023, 31(11): 62-68.
2.	黄丽红, 言方荣, 买亚兵, 等. 肿瘤临床研究中非比例风险生存资料的统计分析. 中国循证医学杂志, 2023, 23(7): 826-833.
3.	严若华, 李卫. Cox回归模型比例风险假定的检验方法研究. 中国卫生统计, 2016, 33(2): 345-349.
4.	Park SY, Park JE, Kim H, et al. Review of statistical methods for evaluating the performance of survival or other time-to-event prediction models (from conventional to deep learning approaches). Korean J Radiol, 2021, 22(10): 1697-1707.
5.	Mukhopadhyay P, Ye J, Anderson KM, et al. Log-rank test vs maxcombo and difference in restricted mean survival time tests for comparing survival under nonproportional hazards in immuno-oncology trials: a systematic review and meta-analysis. JAMA Oncol, 2022, 8(9): 1294-1300.
6.	Salerno S, Li Y. High-dimensional survival analysis: methods and applications. Annu Rev Stat Appl, 2023, 10(1): 25-49.
7.	Rajpurkar P, Chen E, Banerjee O, et al. AI in health and medicine. Nat Med, 2022, 28(1): 31-38.
8.	Shehab M, Abualigah L, Shambour Q, et al. Machine learning in medical applications: a review of state-of-the-art methods. Comput Biol Med, 2022, 145: 105458.
9.	Huang Y, Li J, Li M, et al. Application of machine learning in predicting survival outcomes involving real-world data: a scoping review. BMC Med Res Methodol, 2023, 23(1): 268.
10.	Katzman JL, Shaham U, Cloninger A, et al. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Med Res Methodol, 2018, 18(1): 24.
11.	Lei J, Xu X, Xu J, et al. The predictive value of modified-DeepSurv in overall survivals of patients with lung cancer. iScience, 2023, 26(11): 108200.
12.	Godoy LC, Ko DT, Farkouh ME, et al. Dealing with nonproportional hazards in coronary revascularisation studies. Can J Cardiol, 2023, 39(11): 1651-1660.
13.	Gregson J, Sharples L, Stone GW, et al. Nonproportional hazards for time-to-event outcomes in clinical trials: JACC review topic of the week. J Am Coll Cardiol, 2019, 74(16): 2102-2112.
14.	Chen L, Shi B, Feng S. Where medical statistics meets artificial intelligence. N Engl J Med, 2023, 389(25): 2403.
15.	Ishwaran HKU, Blackstone EH, LAUER MS. Random survival forests. Ann Appl Stat, 2008, 2: 841-860.
16.	陈哲, 许恒敏, 李哲轩, 等. 随机生存森林: 基于机器学习算法的生存分析模型. 中华预防医学杂志, 2021, 55(1): 104-109.
17.	李淼, 罗天娥, 郭强, 等. 随机生存森林模型在肺癌患者预后分析中的应用. 中国卫生统计, 2021, 38(3): 327-331.
18.	Van Belle V, Pelckmans K, Suykens JAK, et al. Learning transformation models for ranking and survival analysis. J Mach Learn Res, 2011, 12: 819-862.
19.	Van Belle V, Pelckmans K, Van Huffel S, et al. Support vector methods for survival analysis: a comparison between ranking and regression approaches. Artif Intell Med, 2011, 53(2): 107-118.
20.	Shivaswamy PK, Chu W, Jansche M. A support vector approach to censored targets; proceedings of the seventh ieee international conference on data mining (ICDM 2007). 2007.
21.	Chen Y, Jia Z, Mercola D, et al. A gradient boosting algorithm for survival analysis via direct optimization of concordance index. Comput Math Methods Med, 2013, 2013: 873595.
22.	Hothorn T, Bühlmann P, Dudoit S, et al. Survival ensembles. Biostatistics, 2006, 7(3): 355-373.
23.	Chen D, Liang S, Chen J, et al. Machine learning-based overall and cancer-specific survival prediction of M0 penile squamous cell carcinoma: a population-based retrospective study. Heliyon, 2023, 10(1): e23442.
24.	Yu CN, Greiner R, Lin HC, et al. Learning patient-specific cancer survival distributions as a sequence of dependent regressors, proceedings of the neural information processing systems. 2011.
25.	Gu W, Zhang Z, Xie X, et al. An improved muti-task learning algorithm for analyzing cancer survival data. IEEE/ACM Trans Comput Biol Bioinform, 2021, 18(2): 500-511.
26.	Kvamme H, Borgan Ø. Continuous and discrete-time survival prediction with neural networks. Lifetime Data Anal, 2021, 27(4): 710-736.
27.	Yang X, Qiu H, Wang L, et al. Predicting colorectal cancer survival using time-to-event machine learning: retrospective cohort study. J Med Internet Res, 2023, 25: e44417.
28.	Adeoye J, Koohi-Moghadam M, Lo AWI, et al. Deep learning predicts the malignant-transformation-free survival of oral potentially malignant disorders. Cancers (Basel), 2021, 13(23): 6054.
29.	Fotso S. Deep neural networks for survival analysis based on a multi-task framework. 2018.
30.	Kiessling J, Brunnberg A, Holte G, et al. Artificial intelligence outperforms kaplan-meier analyses estimating survival after elective treatment of abdominal aortic aneurysms. Eur J Vasc Endovasc Surg, 2023, 65(4): 600-607.
31.	Ananthakrishnan R, Green S, Previtali A, et al. Critical review of oncology clinical trial design under non-proportional hazards. Crit Rev Oncol Hematol, 2021, 162: 103350.
32.	Wang M, Greenberg M, Forkert ND, et al. Dementia risk prediction in individuals with mild cognitive impairment: a comparison of Cox regression and machine learning models. BMC Med Res Methodol, 2022, 22(1): 284.
33.	Kvamme H, Borgan Ø, Scheel I. Time-to-event prediction with neural networks and cox regression. 2019.
34.	Lee C, Zame WR, Yoon J, et al. DeepHit: a deep learning approach to survival analysis with competing risks. New Orleans: AAAI Press, 2018.
35.	Antolini L, Boracchi P, Biganzoli E. A time-dependent discrimination index for survival data. Stat Med, 2005, 24(24): 3927-3944.
36.	Gerds TA, Schumacher M. Consistent estimation of the expected Brier score in general survival models with right-censored event times. Biom J, 2006, 48(6): 1029-1040.
37.	Cottin A, Zulian M, Pécuchet N, et al. MS-CPFI: a model-agnostic counterfactual perturbation feature importance algorithm for interpreting black-box multi-state models. Artif Intell Med, 2024, 147: 102741.
38.	Hu W, Jin T, Pan Z, et al. An interpretable ensemble learning model facilitates early risk stratification of ischemic stroke in intensive care unit: development and external validation of ICU-ISPM. Comput Biol Med, 2023, 166: 107577.
39.	Saxena M, Young P, Pilcher D, et al. Early temperature and mortality in critically ill patients with acute neurological diseases: trauma and stroke differ from infection. Intensive Care Med, 2015, 41(5): 823-832.
40.	Freidlin B, Korn EL. Methods for accommodating nonproportional hazards in clinical trials: ready for the primary analysis. J Clin Oncol, 2019, 37(35): 3455-3459.
41.	US Food and Drug Administration. Public workshop: oncology clinical trials in the presence of non-proportional hazards. 2018.
42.	Kattan MW, Gerds TA. A framework for the evaluation of statistical prediction models. Chest, 2020, 158(1S): S29-S38.
43.	杨丰春, 郑思, 李姣. 可解释机器学习方法在疾病预测中的应用: 脓毒血症患者死亡风险研究. 首都医科大学学报, 2022, 43(4): 610-617.
44.	Infante G, Miceli R, Ambrogi F. Sample size and predictive performance of machine learning methods with survival data: a simulation study. Stat Med, 2023, 42(30): 5657-5675.
45.	Cho H, She J, De Marchi D, et al. Machine learning and health science research: tutorial. J Med Internet Res, 2024, 26: e50890.
46.	Andaur Navarro CL, Damen JAA, Takada T, et al. Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review. BMJ, 202, 375: n2281.
47.	Liu Y, Chen PC, Krause J, et al. How to read articles that use machine learning: users' guides to the medical literature. JAMA, 2019, 322(18): 1806-1816.
48.	Andaur Navarro CL, Damen JAA, van Smeden M, et al. Systematic review identifies the design and methodological conduct of studies on machine learning-based prediction models. J Clin Epidemiol, 2023, 154: 8-22.
49.	Dargan S, Kumar M, Ayyagari MR, et al. A survey of deep learning and its applications: a new paradigm to machine learning. Arc Compu Metho Engi, 2020, 27(4): 1071-1092.
50.	Cui P, Athey S. Stable learning establishes some common ground between causal inference and machine learning. Nature Mach Intel, 2022, 4(2): 110-115.

1. 王佩佩, 双卫兵. 生存分析概述及模型应用. 中国医学工程, 2023, 31(11): 62-68.
2. 黄丽红, 言方荣, 买亚兵, 等. 肿瘤临床研究中非比例风险生存资料的统计分析. 中国循证医学杂志, 2023, 23(7): 826-833.
3. 严若华, 李卫. Cox回归模型比例风险假定的检验方法研究. 中国卫生统计, 2016, 33(2): 345-349.
4. Park SY, Park JE, Kim H, et al. Review of statistical methods for evaluating the performance of survival or other time-to-event prediction models (from conventional to deep learning approaches). Korean J Radiol, 2021, 22(10): 1697-1707.
5. Mukhopadhyay P, Ye J, Anderson KM, et al. Log-rank test vs maxcombo and difference in restricted mean survival time tests for comparing survival under nonproportional hazards in immuno-oncology trials: a systematic review and meta-analysis. JAMA Oncol, 2022, 8(9): 1294-1300.
6. Salerno S, Li Y. High-dimensional survival analysis: methods and applications. Annu Rev Stat Appl, 2023, 10(1): 25-49.
7. Rajpurkar P, Chen E, Banerjee O, et al. AI in health and medicine. Nat Med, 2022, 28(1): 31-38.
8. Shehab M, Abualigah L, Shambour Q, et al. Machine learning in medical applications: a review of state-of-the-art methods. Comput Biol Med, 2022, 145: 105458.
9. Huang Y, Li J, Li M, et al. Application of machine learning in predicting survival outcomes involving real-world data: a scoping review. BMC Med Res Methodol, 2023, 23(1): 268.
10. Katzman JL, Shaham U, Cloninger A, et al. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Med Res Methodol, 2018, 18(1): 24.
11. Lei J, Xu X, Xu J, et al. The predictive value of modified-DeepSurv in overall survivals of patients with lung cancer. iScience, 2023, 26(11): 108200.
12. Godoy LC, Ko DT, Farkouh ME, et al. Dealing with nonproportional hazards in coronary revascularisation studies. Can J Cardiol, 2023, 39(11): 1651-1660.
13. Gregson J, Sharples L, Stone GW, et al. Nonproportional hazards for time-to-event outcomes in clinical trials: JACC review topic of the week. J Am Coll Cardiol, 2019, 74(16): 2102-2112.
14. Chen L, Shi B, Feng S. Where medical statistics meets artificial intelligence. N Engl J Med, 2023, 389(25): 2403.
15. Ishwaran HKU, Blackstone EH, LAUER MS. Random survival forests. Ann Appl Stat, 2008, 2: 841-860.
16. 陈哲, 许恒敏, 李哲轩, 等. 随机生存森林: 基于机器学习算法的生存分析模型. 中华预防医学杂志, 2021, 55(1): 104-109.
17. 李淼, 罗天娥, 郭强, 等. 随机生存森林模型在肺癌患者预后分析中的应用. 中国卫生统计, 2021, 38(3): 327-331.
18. Van Belle V, Pelckmans K, Suykens JAK, et al. Learning transformation models for ranking and survival analysis. J Mach Learn Res, 2011, 12: 819-862.
19. Van Belle V, Pelckmans K, Van Huffel S, et al. Support vector methods for survival analysis: a comparison between ranking and regression approaches. Artif Intell Med, 2011, 53(2): 107-118.
20. Shivaswamy PK, Chu W, Jansche M. A support vector approach to censored targets; proceedings of the seventh ieee international conference on data mining (ICDM 2007). 2007.
21. Chen Y, Jia Z, Mercola D, et al. A gradient boosting algorithm for survival analysis via direct optimization of concordance index. Comput Math Methods Med, 2013, 2013: 873595.
22. Hothorn T, Bühlmann P, Dudoit S, et al. Survival ensembles. Biostatistics, 2006, 7(3): 355-373.
23. Chen D, Liang S, Chen J, et al. Machine learning-based overall and cancer-specific survival prediction of M0 penile squamous cell carcinoma: a population-based retrospective study. Heliyon, 2023, 10(1): e23442.
24. Yu CN, Greiner R, Lin HC, et al. Learning patient-specific cancer survival distributions as a sequence of dependent regressors, proceedings of the neural information processing systems. 2011.
25. Gu W, Zhang Z, Xie X, et al. An improved muti-task learning algorithm for analyzing cancer survival data. IEEE/ACM Trans Comput Biol Bioinform, 2021, 18(2): 500-511.
26. Kvamme H, Borgan Ø. Continuous and discrete-time survival prediction with neural networks. Lifetime Data Anal, 2021, 27(4): 710-736.
27. Yang X, Qiu H, Wang L, et al. Predicting colorectal cancer survival using time-to-event machine learning: retrospective cohort study. J Med Internet Res, 2023, 25: e44417.
28. Adeoye J, Koohi-Moghadam M, Lo AWI, et al. Deep learning predicts the malignant-transformation-free survival of oral potentially malignant disorders. Cancers (Basel), 2021, 13(23): 6054.
29. Fotso S. Deep neural networks for survival analysis based on a multi-task framework. 2018.
30. Kiessling J, Brunnberg A, Holte G, et al. Artificial intelligence outperforms kaplan-meier analyses estimating survival after elective treatment of abdominal aortic aneurysms. Eur J Vasc Endovasc Surg, 2023, 65(4): 600-607.
31. Ananthakrishnan R, Green S, Previtali A, et al. Critical review of oncology clinical trial design under non-proportional hazards. Crit Rev Oncol Hematol, 2021, 162: 103350.
32. Wang M, Greenberg M, Forkert ND, et al. Dementia risk prediction in individuals with mild cognitive impairment: a comparison of Cox regression and machine learning models. BMC Med Res Methodol, 2022, 22(1): 284.
33. Kvamme H, Borgan Ø, Scheel I. Time-to-event prediction with neural networks and cox regression. 2019.
34. Lee C, Zame WR, Yoon J, et al. DeepHit: a deep learning approach to survival analysis with competing risks. New Orleans: AAAI Press, 2018.
35. Antolini L, Boracchi P, Biganzoli E. A time-dependent discrimination index for survival data. Stat Med, 2005, 24(24): 3927-3944.
36. Gerds TA, Schumacher M. Consistent estimation of the expected Brier score in general survival models with right-censored event times. Biom J, 2006, 48(6): 1029-1040.
37. Cottin A, Zulian M, Pécuchet N, et al. MS-CPFI: a model-agnostic counterfactual perturbation feature importance algorithm for interpreting black-box multi-state models. Artif Intell Med, 2024, 147: 102741.
38. Hu W, Jin T, Pan Z, et al. An interpretable ensemble learning model facilitates early risk stratification of ischemic stroke in intensive care unit: development and external validation of ICU-ISPM. Comput Biol Med, 2023, 166: 107577.
39. Saxena M, Young P, Pilcher D, et al. Early temperature and mortality in critically ill patients with acute neurological diseases: trauma and stroke differ from infection. Intensive Care Med, 2015, 41(5): 823-832.
40. Freidlin B, Korn EL. Methods for accommodating nonproportional hazards in clinical trials: ready for the primary analysis. J Clin Oncol, 2019, 37(35): 3455-3459.
41. US Food and Drug Administration. Public workshop: oncology clinical trials in the presence of non-proportional hazards. 2018.
42. Kattan MW, Gerds TA. A framework for the evaluation of statistical prediction models. Chest, 2020, 158(1S): S29-S38.
43. 杨丰春, 郑思, 李姣. 可解释机器学习方法在疾病预测中的应用: 脓毒血症患者死亡风险研究. 首都医科大学学报, 2022, 43(4): 610-617.
44. Infante G, Miceli R, Ambrogi F. Sample size and predictive performance of machine learning methods with survival data: a simulation study. Stat Med, 2023, 42(30): 5657-5675.
45. Cho H, She J, De Marchi D, et al. Machine learning and health science research: tutorial. J Med Internet Res, 2024, 26: e50890.
46. Andaur Navarro CL, Damen JAA, Takada T, et al. Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review. BMJ, 202, 375: n2281.
47. Liu Y, Chen PC, Krause J, et al. How to read articles that use machine learning: users' guides to the medical literature. JAMA, 2019, 322(18): 1806-1816.
48. Andaur Navarro CL, Damen JAA, van Smeden M, et al. Systematic review identifies the design and methodological conduct of studies on machine learning-based prediction models. J Clin Epidemiol, 2023, 154: 8-22.
49. Dargan S, Kumar M, Ayyagari MR, et al. A survey of deep learning and its applications: a new paradigm to machine learning. Arc Compu Metho Engi, 2020, 27(4): 1071-1092.
50. Cui P, Athey S. Stable learning establishes some common ground between causal inference and machine learning. Nature Mach Intel, 2022, 4(2): 110-115.

Chinese Journal of Evidence-Based Medicine

Latest ArticlesApplication of machine learning models for survival data with non-proportional hazard and case study

Abstract Full text Figures/Tables Video References Cited by

Format

Content