Interpretable machine learning-based prognostic model for severe chronic obstructive pulmonary disease_Chinese Journal of Respiratory and Critical Care Medicine

Authors：

GENG Qikun ¹ , LI Jili ¹ , HU Yundi ² , LI Runhong ³ , XIANG Tao ¹ , ZHANG Shuangyi ¹ ,  LI Lei ⁴

1. West China School of Medicine, Sichuan University, Chengdu, Sichuan 610041, P. R. China;
2. School of Data Science, Fudan University, Shanghai 200433, P. R. China;
3. State Key Laboratory of Biotherapy, Sichuan University, Chengdu, Sichuan 610041, P. R. China;
4. Department of Respiratory and Critical Care Medicine, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, P. R. China;

Corresponding author：

LI Lei, Email: lilei41lilei@163.com

Keywords：

Chronic obstructive pulmonary disease; machine learning; eICU Collaborative Research Database; mortality; SHAP (SHapley Additive exPlanation) method

DOI：

10.7507/1671-6205.202311003

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To develop a machine learning (ML) model to predict the risk of death in intensive care unit (ICU) patients with chronic obstructive pulmonary disease (COPD), explain the factors related to the risk of death in COPD patients, and solve the "black box" problem of ML model. Methods A total of 8088 patients with severe COPD were selected from the eICU Collaborative Research Database (eICU-CRD). Data within the initial 24 hours of each ICU stay were extracted and randomly divided, with 70% for model training and 30% for model validation. The LASSO regression was deployed for predictor variable selection to avoid overfitting. Five ML models were employed to predict in-hospital mortality. The prediction performance of the ML models was compared with alternative models using the area under curve (AUC), while SHAP (SHapley Additive exPlanations) method was used to explain this random forest (RF) model. Results The RF model performed best among the APACHE IVa scoring system and five ML models with the AUC of 0.830 (95%CI 0.806 - 0.855). The SHAP method detects the top 10 predictors according to the importance ranking and the minimum of non-invasive systolic blood pressure was recognized as the most significant predictor variable. Conclusion Leveraging ML model to capture risk factors and using the SHAP method to interpret the prediction outcome can predict the risk of death of patients early, which helps clinicians make accurate treatment plans and allocate medical resources rationally.

Citation： GENG Qikun, LI Jili, HU Yundi, LI Runhong, XIANG Tao, ZHANG Shuangyi, LI Lei. Interpretable machine learning-based prognostic model for severe chronic obstructive pulmonary disease. Chinese Journal of Respiratory and Critical Care Medicine, 2024, 23(3): 153-159. doi: 10.7507/1671-6205.202311003 Copy

1.	GBD 2015 Chronic Respiratory Disease Collaborators. Global, regional, and national deaths, prevalence, disability-adjusted life years, and years lived with disability for chronic obstructive pulmonary disease and asthma, 1990-2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet Respir Med, 2017, 5(9): 691-706.
2.	Sandelowsky H, Weinreich UM, Aarli BB, et al. COPD - do the right thing. BMC Fam Pract, 2021, 22(1): 244.
3.	Guarascio AJ, Ray SM, Finch CK, et al. The clinical and economic burden of chronic obstructive pulmonary disease in the USA. Clinicoecon Outcomes Res, 2013, 5(1): 235-245.
4.	Ongel EA, Karakurt Z, Salturk C, et al. How do COPD comorbidities affect ICU outcomes? Int J Chron Obstruct Pulmon Dis, 2014, 9: 1187-1196.
5.	Soriano JB, Polverino F, Cosio BG. What is early COPD and why is it important? Eur Respir J, 2018, 52(6): 1801448.
6.	Moll M, Qiao D, Regan EA, et al. Machine learning and prediction of all-cause mortality in COPD. Chest, 2020, 158(3): 952-964.
7.	卢玲, 王长征. 慢性阻塞性肺疾病过去一年有一次加重住院史真的是未来加重的高风险因素吗? 中国呼吸与危重监护杂志, 2023, 22(4): 229-231.
8.	Celli BR, Cote CG, Marin JM, et al. The body-mass index, airflow obstruction, dyspnea, and exercise capacity index in chronic obstructive pulmonary disease. N Engl J Med, 2004, 350(10): 1005-1112.
9.	Godinjak A, Iglica A, Rama A, et al. Predictive value of SAPS II and APACHE II scoring systems for patient outcome in a medical intensive care unit. Acta Med Acad, 2016, 45(2): 97-103.
10.	Zeng Z, Yao S, Zheng J, et al. Development and validation of a novel blending machine learning model for hospital mortality prediction in ICU patients with Sepsis. BioData Min, 2021, 14(1): 40.
11.	Zhang Z, Ho KM, Hong Y. Machine learning for the prediction of volume responsiveness in patients with oliguric acute kidney injury in critical care. Crit Care, 2019, 23(1): 112.
12.	Chekroud AM, Zotti RJ, Shehzad Z, et al. Cross-trial prediction of treatment outcome in depression: a machine learning approach. Lancet Psychiatry, 2016, 3(3): 243-250.
13.	李梦琪, 贺丰泰, 吴雅楠, 等. 基于机器学习方法建立慢性阻塞性肺疾病合并肺癌风险的预测模型. 中国呼吸与危重监护杂志, 2022, 21(11): 782-789.
14.	Shillan D, Sterne JaC, Champneys A, et al. Use of machine learning to analyse routinely collected intensive care unit data: a systematic review. Crit Care, 2019, 23(1): 284.
15.	Luo C, Zhu Y, Zhu Z, et al. A machine learning-based risk stratification tool for in-hospital mortality of intensive care unit patients with heart failure. J Transl Med, 2022, 20(1): 136.
16.	Stenwig E, Salvi G, Rossi PS, et al. Comparative analysis of explainable machine learning prediction models for hospital mortality. BMC Med Res Methodol, 2022, 22(1): 53.
17.	Hou N, Li M, He L, et al. Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost. J Transl Med, 2020, 18(1): 462.
18.	Breiman L. Random forests. Mach Learn, 2001, 45(1): 5-32.
19.	Dai P, Chang W, Xin Z, et al. Retrospective study on the influencing factors and prediction of hospitalization expenses for chronic renal failure in China based on random forest and LASSO regression. Front Public Health, 2021, 9: 678276.
20.	Peng SY, Chuang YC, Kang TW, et al. Random forest can predict 30-day mortality of spontaneous intracerebral hemorrhage with remarkable discrimination. Eur J Neurol, 2010, 17(7): 945-950.
21.	Tseng PY, Chen YT, Wang CH, et al. Prediction of the development of acute kidney injury following cardiac surgery by machine learning. Crit Care, 2020, 24(1): 478.
22.	Xue B, Li D, Lu C, et al. Use of machine learning to develop and evaluate models using preoperative and intraoperative data to identify risks of postoperative complications. JAMA Netw Open, 2021, 4(3): e212240.
23.	Pollard TJ, Johnson AEW, Raffa JD, et al. The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Scientific Data, 2018, 5: 180178.
24.	Tsugawa Y, Jha AK, Newhouse JP, et al. Variation in physician spending and association with patient outcomes. JAMA Intern Med, 2017, 177(5): 675-682.
25.	Yang B, Zhu Y, Lu X, et al. A novel composite indicator of predicting mortality risk for heart failure patients with diabetes admitted to intensive care unit based on machine learning. Front Endocrinol (Lausanne), 2022, 13: 917838.
26.	Groenewegen KH, Schols AMWJ, Wouters EFM. Mortality and mortality-related factors after hospitalization for acute exacerbation of COPD. Chest, 2003, 124(2): 459-467.
27.	Kelly E, Owen CA, Pinto-Plata V, et al. The role of systemic inflammatory biomarkers to predict mortality in chronic obstructive pulmonary disease. Expert Rev Respir Med, 2013, 7(1): 57-64.
28.	Tabak YP, Sun X, Johannes RS, et al. Development and validation of a mortality risk-adjustment model for patients hospitalized for exacerbations of chronic obstructive pulmonary disease. Med Care, 2013, 51(7): 597-605.
29.	Tibshirani R. Regression shrinkage and selection via the Lasso. J R Statist Soc B, 1996, 58(1): 267-288.
30.	Muthukrishnan R, Rohini R. LASSO: a feature selection technique in predictive modeling for machine learning. IEEE International Conference on Advances in Computer Applications (ICACA), 2016: 18-20.
31.	Emmanuel T, Maupong T, Mpoeleng D, et al. A survey on missing data in machine learning. J Big Data, 2021, 8(1): 140.
32.	Stekhoven DJ, Bühlmann P. MissForest-non-parametric missing value imputation for mixed-type data. Bioinformatics, 2012, 28(1): 112-118.
33.	Li YM, Li ZL, Chen F, et al. A LASSO-derived risk model for long-term mortality in Chinese patients with acute coronary syndrome. J Transl Med, 2020, 18(1): 157.
34.	Lohner V, Pehlivan G, Sanroma G, et al. Relation between sex, menopause, and white matter hyperintensities: The Rhineland Study. Neurology, 2022, 99(9): e935-e43.
35.	Buergel T, Steinfeldt J, Ruyoga G, et al. Metabolomic profiles predict individual multidisease outcomes. Nat Med, 2022, 28(11): 2309-2320.
36.	Lundberg SM, Lee SI. A unified approach to interpreting model predictions. 31st Annual Conference on Neural Information Processing Systems (NIPS), 2017, 30.
37.	Zimmerman JE, Kramer AA, Mcnair DS, et al. Acute physiology and chronic health evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med, 2006, 34(5): 1297-1310.
38.	Matheson AM, Parraga G. Machine learning predictions of COPD mortality: computational hide and seek. Chest, 2020, 158(3): 846-847.
39.	Lanclus M, Clukers J, Van Holsbeke C, et al. Machine learning algorithms utilizing functional respiratory imaging may predict COPD exacerbations. Acad Radiol, 2019, 26(9): 1191-1199.
40.	Kaplan A, Cao H, Fitzgerald JM, et al. Artificial intelligence/machine learning in respiratory medicine and potential role in asthma and COPD diagnosis. J Allergy Clin Immunol Pract, 2021, 9(6): 2255-2261.
41.	Vega L, Viñes F and Neyman KM. Unravelling Morphological and Topological Energy Contributions of Metal Nanoparticles. Nanomaterials (Basel), 2021, 12(1): 17.
42.	Richards G, Rayward-Smith VJ, Sönksen PH, et al. Data mining for indicators of early mortality in a database of clinical records. Artif Intell Med, 2001, 22(3): 215-231.
43.	Mirzakhani F, Sadoughi F, Hatami M, et al. Which model is superior in predicting ICU survival: artificial intelligence versus conventional approaches. BMC Med Inform Decis Mak, 2022, 22(1): 167.
44.	Xu J, Chen X, Zheng X. Acinetobacter baumannii complex-caused bloodstream infection in ICU during a 12-year period: Predicting fulminant sepsis by interpretable machine learning. Front Microbiol, 2022, 13: 1037735.
45.	Li J, Liu S, Hu Y, et al. Predicting mortality in intensive care unit patients with heart failure using an interpretable machine learning model: retrospective cohort study. J Med Internet Res, 2022, 24(8): e38082.
46.	Huang D, He D, Gong L, et al. A prediction model for hospital mortality in patients with severe community-acquired pneumonia and chronic obstructive pulmonary disease. Respir Res, 2022, 23(1): 250.
47.	Franco Palacios CR, Haugen EN, Thompson AM, et al. Clinical outcomes with a systolic blood pressure lower than 120 mmHg in older patients with high disease burden. Ren Fail, 2016, 38(9): 1364-1369.
48.	Ashley C, Burton D, Sverrisdottir YB, et al. Firing probability and mean firing rates of human muscle vasoconstrictor neurones are elevated during chronic asphyxia. J Physiol, 2010, 588(Pt 4): 701-712.

1. GBD 2015 Chronic Respiratory Disease Collaborators. Global, regional, and national deaths, prevalence, disability-adjusted life years, and years lived with disability for chronic obstructive pulmonary disease and asthma, 1990-2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet Respir Med, 2017, 5(9): 691-706.
2. Sandelowsky H, Weinreich UM, Aarli BB, et al. COPD - do the right thing. BMC Fam Pract, 2021, 22(1): 244.
3. Guarascio AJ, Ray SM, Finch CK, et al. The clinical and economic burden of chronic obstructive pulmonary disease in the USA. Clinicoecon Outcomes Res, 2013, 5(1): 235-245.
4. Ongel EA, Karakurt Z, Salturk C, et al. How do COPD comorbidities affect ICU outcomes? Int J Chron Obstruct Pulmon Dis, 2014, 9: 1187-1196.
5. Soriano JB, Polverino F, Cosio BG. What is early COPD and why is it important? Eur Respir J, 2018, 52(6): 1801448.
6. Moll M, Qiao D, Regan EA, et al. Machine learning and prediction of all-cause mortality in COPD. Chest, 2020, 158(3): 952-964.
7. 卢玲, 王长征. 慢性阻塞性肺疾病过去一年有一次加重住院史真的是未来加重的高风险因素吗? 中国呼吸与危重监护杂志, 2023, 22(4): 229-231.
8. Celli BR, Cote CG, Marin JM, et al. The body-mass index, airflow obstruction, dyspnea, and exercise capacity index in chronic obstructive pulmonary disease. N Engl J Med, 2004, 350(10): 1005-1112.
9. Godinjak A, Iglica A, Rama A, et al. Predictive value of SAPS II and APACHE II scoring systems for patient outcome in a medical intensive care unit. Acta Med Acad, 2016, 45(2): 97-103.
10. Zeng Z, Yao S, Zheng J, et al. Development and validation of a novel blending machine learning model for hospital mortality prediction in ICU patients with Sepsis. BioData Min, 2021, 14(1): 40.
11. Zhang Z, Ho KM, Hong Y. Machine learning for the prediction of volume responsiveness in patients with oliguric acute kidney injury in critical care. Crit Care, 2019, 23(1): 112.
12. Chekroud AM, Zotti RJ, Shehzad Z, et al. Cross-trial prediction of treatment outcome in depression: a machine learning approach. Lancet Psychiatry, 2016, 3(3): 243-250.
13. 李梦琪, 贺丰泰, 吴雅楠, 等. 基于机器学习方法建立慢性阻塞性肺疾病合并肺癌风险的预测模型. 中国呼吸与危重监护杂志, 2022, 21(11): 782-789.
14. Shillan D, Sterne JaC, Champneys A, et al. Use of machine learning to analyse routinely collected intensive care unit data: a systematic review. Crit Care, 2019, 23(1): 284.
15. Luo C, Zhu Y, Zhu Z, et al. A machine learning-based risk stratification tool for in-hospital mortality of intensive care unit patients with heart failure. J Transl Med, 2022, 20(1): 136.
16. Stenwig E, Salvi G, Rossi PS, et al. Comparative analysis of explainable machine learning prediction models for hospital mortality. BMC Med Res Methodol, 2022, 22(1): 53.
17. Hou N, Li M, He L, et al. Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost. J Transl Med, 2020, 18(1): 462.
18. Breiman L. Random forests. Mach Learn, 2001, 45(1): 5-32.
19. Dai P, Chang W, Xin Z, et al. Retrospective study on the influencing factors and prediction of hospitalization expenses for chronic renal failure in China based on random forest and LASSO regression. Front Public Health, 2021, 9: 678276.
20. Peng SY, Chuang YC, Kang TW, et al. Random forest can predict 30-day mortality of spontaneous intracerebral hemorrhage with remarkable discrimination. Eur J Neurol, 2010, 17(7): 945-950.
21. Tseng PY, Chen YT, Wang CH, et al. Prediction of the development of acute kidney injury following cardiac surgery by machine learning. Crit Care, 2020, 24(1): 478.
22. Xue B, Li D, Lu C, et al. Use of machine learning to develop and evaluate models using preoperative and intraoperative data to identify risks of postoperative complications. JAMA Netw Open, 2021, 4(3): e212240.
23. Pollard TJ, Johnson AEW, Raffa JD, et al. The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Scientific Data, 2018, 5: 180178.
24. Tsugawa Y, Jha AK, Newhouse JP, et al. Variation in physician spending and association with patient outcomes. JAMA Intern Med, 2017, 177(5): 675-682.
25. Yang B, Zhu Y, Lu X, et al. A novel composite indicator of predicting mortality risk for heart failure patients with diabetes admitted to intensive care unit based on machine learning. Front Endocrinol (Lausanne), 2022, 13: 917838.
26. Groenewegen KH, Schols AMWJ, Wouters EFM. Mortality and mortality-related factors after hospitalization for acute exacerbation of COPD. Chest, 2003, 124(2): 459-467.
27. Kelly E, Owen CA, Pinto-Plata V, et al. The role of systemic inflammatory biomarkers to predict mortality in chronic obstructive pulmonary disease. Expert Rev Respir Med, 2013, 7(1): 57-64.
28. Tabak YP, Sun X, Johannes RS, et al. Development and validation of a mortality risk-adjustment model for patients hospitalized for exacerbations of chronic obstructive pulmonary disease. Med Care, 2013, 51(7): 597-605.
29. Tibshirani R. Regression shrinkage and selection via the Lasso. J R Statist Soc B, 1996, 58(1): 267-288.
30. Muthukrishnan R, Rohini R. LASSO: a feature selection technique in predictive modeling for machine learning. IEEE International Conference on Advances in Computer Applications (ICACA), 2016: 18-20.
31. Emmanuel T, Maupong T, Mpoeleng D, et al. A survey on missing data in machine learning. J Big Data, 2021, 8(1): 140.
32. Stekhoven DJ, Bühlmann P. MissForest-non-parametric missing value imputation for mixed-type data. Bioinformatics, 2012, 28(1): 112-118.
33. Li YM, Li ZL, Chen F, et al. A LASSO-derived risk model for long-term mortality in Chinese patients with acute coronary syndrome. J Transl Med, 2020, 18(1): 157.
34. Lohner V, Pehlivan G, Sanroma G, et al. Relation between sex, menopause, and white matter hyperintensities: The Rhineland Study. Neurology, 2022, 99(9): e935-e43.
35. Buergel T, Steinfeldt J, Ruyoga G, et al. Metabolomic profiles predict individual multidisease outcomes. Nat Med, 2022, 28(11): 2309-2320.
36. Lundberg SM, Lee SI. A unified approach to interpreting model predictions. 31st Annual Conference on Neural Information Processing Systems (NIPS), 2017, 30.
37. Zimmerman JE, Kramer AA, Mcnair DS, et al. Acute physiology and chronic health evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med, 2006, 34(5): 1297-1310.
38. Matheson AM, Parraga G. Machine learning predictions of COPD mortality: computational hide and seek. Chest, 2020, 158(3): 846-847.
39. Lanclus M, Clukers J, Van Holsbeke C, et al. Machine learning algorithms utilizing functional respiratory imaging may predict COPD exacerbations. Acad Radiol, 2019, 26(9): 1191-1199.
40. Kaplan A, Cao H, Fitzgerald JM, et al. Artificial intelligence/machine learning in respiratory medicine and potential role in asthma and COPD diagnosis. J Allergy Clin Immunol Pract, 2021, 9(6): 2255-2261.
41. Vega L, Viñes F and Neyman KM. Unravelling Morphological and Topological Energy Contributions of Metal Nanoparticles. Nanomaterials (Basel), 2021, 12(1): 17.
42. Richards G, Rayward-Smith VJ, Sönksen PH, et al. Data mining for indicators of early mortality in a database of clinical records. Artif Intell Med, 2001, 22(3): 215-231.
43. Mirzakhani F, Sadoughi F, Hatami M, et al. Which model is superior in predicting ICU survival: artificial intelligence versus conventional approaches. BMC Med Inform Decis Mak, 2022, 22(1): 167.
44. Xu J, Chen X, Zheng X. Acinetobacter baumannii complex-caused bloodstream infection in ICU during a 12-year period: Predicting fulminant sepsis by interpretable machine learning. Front Microbiol, 2022, 13: 1037735.
45. Li J, Liu S, Hu Y, et al. Predicting mortality in intensive care unit patients with heart failure using an interpretable machine learning model: retrospective cohort study. J Med Internet Res, 2022, 24(8): e38082.
46. Huang D, He D, Gong L, et al. A prediction model for hospital mortality in patients with severe community-acquired pneumonia and chronic obstructive pulmonary disease. Respir Res, 2022, 23(1): 250.
47. Franco Palacios CR, Haugen EN, Thompson AM, et al. Clinical outcomes with a systolic blood pressure lower than 120 mmHg in older patients with high disease burden. Ren Fail, 2016, 38(9): 1364-1369.
48. Ashley C, Burton D, Sverrisdottir YB, et al. Firing probability and mean firing rates of human muscle vasoconstrictor neurones are elevated during chronic asphyxia. J Physiol, 2010, 588(Pt 4): 701-712.

Next Article
The impact of prognostic nutritional index on short-term prognosis in critically chronic obstructive pulmonary disease patients

Chinese Journal of Respiratory and Critical Care Medicine

Interpretable machine learning-based prognostic model for severe chronic obstructive pulmonary disease

Abstract Full text Figures/Tables Video References Cited by

Next Article

Format

Content