Prognostic model of small sample critical diseases based on transfer learning_Journal of Biomedical Engineering

Authors：

XIA Jing ¹ , PAN Su ¹ , YAN Molei ² , CAI Guolong ² , YAN Jing ² ,  NING Gangmin ¹

1. College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou 310027, P.R.China;
2. Department of ICU, Zhejiang Hospital, Hangzhou 310013, P.R.China;

Corresponding author：

NING Gangmin, Email: gmning@zju.edu.cn

Keywords：

critical disease; prognostic model; small samples; long short-term memory; transfer learning

DOI：

10.7507/1001-5515.201905074

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Aiming at the problem that the small samples of critical disease in clinic may lead to prognostic models with poor performance of overfitting, large prediction error and instability, the long short-term memory transferring algorithm (transLSTM) was proposed. Based on the idea of transfer learning, the algorithm leverages the correlation between diseases to transfer information of different disease prognostic models, constructs the effictive model of target disease of small samples with the aid of large data of related diseases, hence improves the prediction performance and reduces the requirement for target training sample quantity. The transLSTM algorithm firstly uses the related disease samples to pretrain partial model parameters, and then further adjusts the whole network with the target training samples. The testing results on MIMIC-Ⅲ database showed that compared with traditional LSTM classification algorithm, the transLSTM algorithm had 0.02-0.07 higher AUROC and 0.05-0.14 larger AUPRC, while its number of training iterations was only 39%-64% of the traditional algorithm. The results of application on sepsis revealed that the transLSTM model of only 100 training samples had comparable mortality prediction performance to the traditional model of 250 training samples. In small sample situations, the transLSTM algorithm has significant advantages with higher prediciton accuracy and faster training speed. It realizes the application of transfer learning in the prognostic model of critical disease with small samples.

Citation： XIA Jing, PAN Su, YAN Molei, CAI Guolong, YAN Jing, NING Gangmin. Prognostic model of small sample critical diseases based on transfer learning. Journal of Biomedical Engineering, 2020, 37(1): 1-9. doi: 10.7507/1001-5515.201905074 Copy

1.	Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput, 1997, 9(8): 1735-1780.
2.	Miotto R, Wang Fei, Wang Shuang, et al. Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform, 2018, 19(6): 1236-1246.
3.	Lipton Z C, Kale D C, Elkan C, et al. Learning to diagnose with LSTM recurrent neural networks. arXiv: 1511.03677.
4.	Che Z, Purushotham S, Khemani R, et al. Interpretable deep models for ICU outcome prediction//AMIA Annual Symposium Proceedings. Chicago: American Medical Informatics Association, 2016: 371-380.
5.	Jo Y, Lee L, Palaskar S. Combining LSTM and latent topic modeling for mortality prediction. arXiv: 1709.02842.
6.	Harutyunyan H, Khachatrian H, Kale D C, et al. Multitask learning and benchmarking with clinical time series data. arXiv: 1703.07771.
7.	Purushotham S, Meng C, Che Zhengping, et al. Benchmark of deep learning models on large healthcare MIMIC datasets. arXiv: 1710.08531.
8.	Pham T, Tran T, Phung D, et al. Deepcare: A deep dynamic memory model for predictive medicine//Bailey J, Khan L, Washio T, et al. Advances in knowledge discovery and data mining. PAKDD 2016. Cham: Springer, 2016, 9652: 30-41.
9.	Baytas I M, Xiao Cao, Zhang Xi, et al. Patient subtyping via time-aware LSTM networks//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Canada: ACM, 2017: 65-74.
10.	Van Steenkiste T, Ruyssinck J, De Baets L, et al. Accurate prediction of blood culture outcome in the intensive care unit using long short-term memory neural networks. Artif Intell Med, 2019, 97: 38-43.
11.	Suresh H, Gong J J, Guttag J V. Learning tasks for multitask learning: Heterogenous patient populations in the ICU// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. London: ACM, 2018: 802-810.
12.	Reddy B K, Delen D. Predicting hospital readmission for lupus patients: An RNN-LSTM-based deep-learning methodology. Comput Biol Med, 2018, 101: 199-209.
13.	Pan S J, Yang Qiang. A survey on transfer learning. IEEE Trans Knowl Data Eng, 2010, 22(10): 1345-1359.
14.	Weiss K, Khoshgoftaar T M, Wang D. A survey of transfer learning. J Big Data, 2016, 3(1): 9.
15.	庄福振. 迁移学习中文本分类算法研究. 北京: 中国科学院大学, 2011.
16.	戴文渊. 基于实例和特征的迁移学习算法研究. 上海: 上海交通大学, 2009.
17.	Shao Ling, Zhu Fan, Li Xuelong. Transfer learning for visual categorization: a survey. IEEE Trans Neural Netw Learn Syst, 2015, 26(5): 1019-1034.
18.	Cheplygina V, Pena I P, Pedersen J H, et al. Transfer learning for multi-center classification of chronic obstructive pulmonary disease. IEEE J Biomed Health Inform, 2018, 22(5): 1486-1496.
19.	Paul R, Hawkins S H, Balagurunathan Y, et al. Deep feature transfer learning in combination with traditional features predicts survival among patients with lung adenocarcinoma. Tomography, 2016, 2(4): 388-395.
20.	Pan Weike. A survey of transfer learning for collaborative recommendation with auxiliary data. Neurocomputing, 2016, 177: 447-453.
21.	Bianchi A, Raimondo Vendra M, Protopapas P, et al. Improving image classification robustness through selective CNN-filters fine-tuning. arXiv: 1904.03949.
22.	Zhu Y, Zhuang F, Yang J, et al. Adaptively transfer category-classifier for handwritten chinese character recognition//Advances in Knowledge Discovery and Data Mining. Cham: Springer International Publishing, 2019: 110-122.
23.	龙明盛. 迁移学习问题与方法研究. 北京: 清华大学, 2014.
24.	Yosinski J, Clune J, Bengio Y, et al. How transferable are features in deep neural networks?//Ghahramani Z, Welling profile M, Cortes C, et al. Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2014, 2: 3320-3328.
25.	Jozefowicz R, Zaremba W, Sutskever I. An empirical exploration of recurrent network architectures. Proceedings of Machine Learning Research, 2015, 37: 2342-2350.
26.	Johnson A E W, Pollard T J, Shen Lu, et al. MIMIC-Ⅲ, a freely accessible critical care database. Scientific Data, 2016, 3: 160035.
27.	World Health Organization. ICD-10: International Statistical Classification of Diseases and Related Health Problems. Geneva: World Health Organization, 2004, 1.
28.	Zhou Jianfang, Qian Chuanyun, Zhao Mingyan, et al. Epidemiology and outcome of severe sepsis and septic shock in intensive care units in mainland China. PLoS One, 2014, 9(9): e107181.
29.	Dellinger R P, Levy M M, Rhodes A, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med, 2013, 41(2): 580-637.
30.	Cho K, Van Merrienboer B, Gulcehre C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv: 1406.1078.
31.	Burges C C. A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov, 1998, 2(2): 121-167.
32.	Lateh M A, Kamilah Muda A, Yusof Z I M, et al. Handling a small dataset problem in prediction model by employ artificial data generation approach: A review. Journal of Physics: Conference Series, 2017, 892(1): 1-10.
33.	Chao G Y, Tsai T I, Lu T J, et al. A new approach to prediction of radiotherapy of bladder cancer cells in small dataset analysis. Expert Syst Appl, 2011, 38(7): 7963-7969.
34.	辛宪会, 叶秋果, 滕惠忠, 等. 小样本机器学习算法的特性分析与应用. 海洋测绘, 2007, 27(3): 16-19.

1. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput, 1997, 9(8): 1735-1780.
2. Miotto R, Wang Fei, Wang Shuang, et al. Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform, 2018, 19(6): 1236-1246.
3. Lipton Z C, Kale D C, Elkan C, et al. Learning to diagnose with LSTM recurrent neural networks. arXiv: 1511.03677.
4. Che Z, Purushotham S, Khemani R, et al. Interpretable deep models for ICU outcome prediction//AMIA Annual Symposium Proceedings. Chicago: American Medical Informatics Association, 2016: 371-380.
5. Jo Y, Lee L, Palaskar S. Combining LSTM and latent topic modeling for mortality prediction. arXiv: 1709.02842.
6. Harutyunyan H, Khachatrian H, Kale D C, et al. Multitask learning and benchmarking with clinical time series data. arXiv: 1703.07771.
7. Purushotham S, Meng C, Che Zhengping, et al. Benchmark of deep learning models on large healthcare MIMIC datasets. arXiv: 1710.08531.
8. Pham T, Tran T, Phung D, et al. Deepcare: A deep dynamic memory model for predictive medicine//Bailey J, Khan L, Washio T, et al. Advances in knowledge discovery and data mining. PAKDD 2016. Cham: Springer, 2016, 9652: 30-41.
9. Baytas I M, Xiao Cao, Zhang Xi, et al. Patient subtyping via time-aware LSTM networks//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Canada: ACM, 2017: 65-74.
10. Van Steenkiste T, Ruyssinck J, De Baets L, et al. Accurate prediction of blood culture outcome in the intensive care unit using long short-term memory neural networks. Artif Intell Med, 2019, 97: 38-43.
11. Suresh H, Gong J J, Guttag J V. Learning tasks for multitask learning: Heterogenous patient populations in the ICU// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. London: ACM, 2018: 802-810.
12. Reddy B K, Delen D. Predicting hospital readmission for lupus patients: An RNN-LSTM-based deep-learning methodology. Comput Biol Med, 2018, 101: 199-209.
13. Pan S J, Yang Qiang. A survey on transfer learning. IEEE Trans Knowl Data Eng, 2010, 22(10): 1345-1359.
14. Weiss K, Khoshgoftaar T M, Wang D. A survey of transfer learning. J Big Data, 2016, 3(1): 9.
15. 庄福振. 迁移学习中文本分类算法研究. 北京: 中国科学院大学, 2011.
16. 戴文渊. 基于实例和特征的迁移学习算法研究. 上海: 上海交通大学, 2009.
17. Shao Ling, Zhu Fan, Li Xuelong. Transfer learning for visual categorization: a survey. IEEE Trans Neural Netw Learn Syst, 2015, 26(5): 1019-1034.
18. Cheplygina V, Pena I P, Pedersen J H, et al. Transfer learning for multi-center classification of chronic obstructive pulmonary disease. IEEE J Biomed Health Inform, 2018, 22(5): 1486-1496.
19. Paul R, Hawkins S H, Balagurunathan Y, et al. Deep feature transfer learning in combination with traditional features predicts survival among patients with lung adenocarcinoma. Tomography, 2016, 2(4): 388-395.
20. Pan Weike. A survey of transfer learning for collaborative recommendation with auxiliary data. Neurocomputing, 2016, 177: 447-453.
21. Bianchi A, Raimondo Vendra M, Protopapas P, et al. Improving image classification robustness through selective CNN-filters fine-tuning. arXiv: 1904.03949.
22. Zhu Y, Zhuang F, Yang J, et al. Adaptively transfer category-classifier for handwritten chinese character recognition//Advances in Knowledge Discovery and Data Mining. Cham: Springer International Publishing, 2019: 110-122.
23. 龙明盛. 迁移学习问题与方法研究. 北京: 清华大学, 2014.
24. Yosinski J, Clune J, Bengio Y, et al. How transferable are features in deep neural networks?//Ghahramani Z, Welling profile M, Cortes C, et al. Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2014, 2: 3320-3328.
25. Jozefowicz R, Zaremba W, Sutskever I. An empirical exploration of recurrent network architectures. Proceedings of Machine Learning Research, 2015, 37: 2342-2350.
26. Johnson A E W, Pollard T J, Shen Lu, et al. MIMIC-Ⅲ, a freely accessible critical care database. Scientific Data, 2016, 3: 160035.
27. World Health Organization. ICD-10: International Statistical Classification of Diseases and Related Health Problems. Geneva: World Health Organization, 2004, 1.
28. Zhou Jianfang, Qian Chuanyun, Zhao Mingyan, et al. Epidemiology and outcome of severe sepsis and septic shock in intensive care units in mainland China. PLoS One, 2014, 9(9): e107181.
29. Dellinger R P, Levy M M, Rhodes A, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med, 2013, 41(2): 580-637.
30. Cho K, Van Merrienboer B, Gulcehre C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv: 1406.1078.
31. Burges C C. A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov, 1998, 2(2): 121-167.
32. Lateh M A, Kamilah Muda A, Yusof Z I M, et al. Handling a small dataset problem in prediction model by employ artificial data generation approach: A review. Journal of Physics: Conference Series, 2017, 892(1): 1-10.
33. Chao G Y, Tsai T I, Lu T J, et al. A new approach to prediction of radiotherapy of bladder cancer cells in small dataset analysis. Expert Syst Appl, 2011, 38(7): 7963-7969.
34. 辛宪会, 叶秋果, 滕惠忠, 等. 小样本机器学习算法的特性分析与应用. 海洋测绘, 2007, 27(3): 16-19.

Next Article
Prediction of gene mutation in lung cancer based on deep learning and histomorphology analysis

Journal of Biomedical Engineering

Prognostic model of small sample critical diseases based on transfer learning

Abstract Full text Figures/Tables Video References Cited by

Next Article

Format

Content