A review of deep learning methods for the detection and classification of pulmonary nodules_Journal of Biomedical Engineering

Authors：

ZHAO Qingyi ^1,2 ,  KONG Ping ² , MIN Jianzhong ² , ZHOU Yanli ² , LIANG Zhuangzhuang ³ , CHEN Sheng ³ , LI Maoju ²

1. School of Medical Instrument and Food Engineering, University of Shanghai for Science & Technology, Shanghai 200093, P.R.China;
2. Shanghai Key Laboratory of Molecular Imaging, Shanghai University of Medicine & Health Sciences, Shanghai 201318, P.R.China;
3. School of Optical Electrical and Computer Engineering, University of Shanghai for Science & Technology, Shanghai 200093, P.R.China;

Corresponding author：

KONG Ping, Email: kongp@sumhs.edu.cn

Keywords：

deep learning; computer-aided diagnosis; pulmonary nodules; convolutional neural network; medical image

DOI：

10.7507/1001-5515.201903027

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Lung cancer has the highest mortality rate among all malignant tumors. The key to reducing lung cancer mortality is the accurate diagnosis of pulmonary nodules in early-stage lung cancer. Computer-aided diagnostic techniques are considered to have potential beyond human experts for accurate diagnosis of early pulmonary nodules. The detection and classification of pulmonary nodules based on deep learning technology can continuously improve the accuracy of diagnosis through self-learning, and is an important means to achieve computer-aided diagnosis. First, we systematically introduced the application of two dimension convolutional neural network (2D-CNN), three dimension convolutional neural network (3D-CNN) and faster regions convolutional neural network (Faster R-CNN) techniques in the detection of pulmonary nodules. Then we introduced the application of 2D-CNN, 3D-CNN, multi-stream multi-scale convolutional neural network (MMCNN), deep convolutional generative adversarial networks (DCGAN) and transfer learning technology in classification of pulmonary nodules. Finally, we conducted a comprehensive comparative analysis of different deep learning methods in the detection and classification of pulmonary nodules.

Citation： ZHAO Qingyi, KONG Ping, MIN Jianzhong, ZHOU Yanli, LIANG Zhuangzhuang, CHEN Sheng, LI Maoju. A review of deep learning methods for the detection and classification of pulmonary nodules. Journal of Biomedical Engineering, 2019, 36(6): 1060-1068. doi: 10.7507/1001-5515.201903027 Copy

1.	Torre L A, Siegel R L, Jemal A. Lung cancer statistics. Adv Exp Med Biol, 2016, 893: 1-19.
2.	Chen Wanqing, Zheng Rongshou, Baade P D, et al. Cancer statistics in China, 2015. CA Cancer J Clin, 2016, 66(2): 115-132.
3.	Cao Maomao, Chen Wanqing. Epidemiology of lung cancer in China. Thoracic Cancer, 2019, 10(1): 3-7.
4.	Kermany D S, Goldbaum M, Cai Wenjia, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell, 2018, 172(5): 1122-1131.
5.	Lecun Y, Boser B, Denker J S, et al. Backpropagation applied to handwritten zip code recognition. Neural Comput, 1989, 1(4): 541-551.
6.	Lecun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
7.	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Commun ACM, 2017, 60(6): 84-90.
8.	Donahue J, Hendricks L A, Rohrbach M A, et al. Long-term recurrent convolutional networks for visual recognition and description. IEEE Trans Pattern Anal Mach Intell, 2017, 39(4): 677-691.
9.	Havaei M, Davy A, Warde-Farley D, et al. Brain tumor segmentation with deep neural networks. Med Image Anal, 2017, 35: 18-31.
10.	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 770-778.
11.	Szegedy C, Liu W, Jia Y Q, et al. Going deeper with convolutions// 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2015: 1-9.
12.	Setio A A A, Ciompi F, Litjens G, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging, 2016, 35(5): 1160-1169.
13.	Setio A A A, Traverso A, De Bel T, et al. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge. Med Image Anal, 2017, 42: 1-13.
14.	苗光, 李朝锋. 二维和 3 维卷积神经网络相结合的 CT 图像肺结节检测方法. 激光与光电子学进展, 2018, 55(5): 135-143.
15.	Han Y, Ye J C. Framing U-net via deep convolutional framelets: application to sparse-view CT. IEEE Trans Med Imaging, 2018, 37(6): 1418-1429.
16.	Ghafoorian M, Karssemeijer N, Heskes T, et al. Deep multi-scale location-aware 3D convolutional neural networks for automated detection of lacunes of presumed vascular origin. Neuroimage Clin, 2017, 14: 391-399.
17.	Dou Qi, Chen Hao, Yu Lequan, et al. Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans Biomed Eng, 2017, 64(7): 1558-1567.
18.	Pezeshk A, Hamidian S, Petrick N, et al. 3D convolutional neural networks for automatic detection of pulmonary nodules in chest CT. IEEE J Biomed Health Inform, 2018: 2168-2194.
19.	Winkels M, Cohen T S. 3D G-CNNs for pulmonary nodule detection. arXiv preprint arXiv: 2018, 1804.04656.
20.	Huang X J, Shan J J, Vaidya V, et al. Lung nodule detection in CT using 3D convolutional neural networks. New York: IEEE, 2017: 379-383.
21.	Anirudh R, Thiagarajan J J, Bremer T, et al. Lung nodule detection using 3D convolutional neural networks trained on weakly labeled data// Medical Imaging 2016: Computer-Aided Diagnosis. San Diego: International Society for Optics and Photonics, 2016, 9785: 978532.
22.	Hamidian S, Sahiner B, Petrick N, et al. 3D convolutional neural network for automatic detection of lung nodules in chest CT// Proceedings of SPIE. Orlando: The International Society for Optical Engineering, 2017, 10134: 1013409.
23.	Roth H R, Oda H, Zhou Xiangrong, et al. An application of cascaded 3D fully convolutional networks for medical image segmentation. Comput Med Imaging Graph, 2018, 66: 90-99.
24.	Gu Yu, Lu Xiaoqi, Yang Lidong, et al. Automatic lung nodule detection using a 3D deep convolutional neural network combined with a multi-scale prediction strategy in chest CTs. Comput Biol Med, 2018, 103: 220-231.
25.	Ren Shaoqing, He Kaiming, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell, 2017, 39(6): 1137-1149.
26.	Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation// 2014 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2014: 580-587.
27.	He K, Gkioxari G, Dollar P, et al. Mask R-CNN// 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2961-2969.
28.	Xie Hongtao, Yang Dongbao, Sun Nannan, et al. Automated pulmonary nodule detection in CT images using deep convolutional neural networks. Pattern Recognit, 2019, 85: 109-119.
29.	Ding J, Li A, Hu Z, et al. Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks// International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City: ICCAI, 2017: 559-567.
30.	Hua K L, Hsu C H, Hidayati H C, et al. Computer-aided classification of lung nodules on computed tomography images via deep learning technique. Onco Targets Ther, 2015, 8: 2015-2022.
31.	Liu Shuang, Xie Yiting, Jirapatnakul A, et al. Pulmonary nodule classification in lung cancer screening with three-dimensional convolutional neural networks. J Med Imaging, 2017, 4(4): 041308.
32.	Ciompi F, Chung K, Riel S J V, et al. Towards automatic pulmonary nodule management in lung cancer screening with deep learning. Sci Rep, 2017, 7: 46479.
33.	Shen Wei, Zhou Mu, Yang Feng, et al. Multi-scale convolutional neural networks for lung nodule classification. Inf Process Med Imaging, 2015, 24: 588-599.
34.	徐久强, 洪丽萍, 朱宏博, 等. 一种用于肺结节恶性度分类的生成对抗网络. 东北大学学报: 自然科学版, 2018, 39(11): 1556-1561.
35.	Suarez P L, Sappa A D, Vintimilla B X. Infrared image colorization based on a triplet DCGAN architecture// 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu: IEEE, 2017: 212-217.
36.	Mao Xudong, Li Qing, Xie Haoran, et al. Least squares generative adversarial networks// 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2794-2802.
37.	Hussein S, Cao K, Song Q, et al. Risk stratification of lung nodules using 3d CNN-based multi-task learning// International Conference on Information Processing in Medical Imaging. Boone: IPMI, 2017: 249-260.
38.	Zamir A R, Sax A, Shen W, et al. Taskonomy: Disentangling task transfer learning// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3712-3722.
39.	陈诗慧, 刘维湘, 秦璟, 等. 基于深度学习和医学图像的癌症计算机辅助诊断研究进展. 生物医学工程学杂志, 2017, 34(2): 314-319.
40.	Yuan Xiaofeng, Huang Biao, Wang Yalin, et al. Deep learning-based feature representation and its application for soft sensor modeling with variable-wise weighted SAE. IEEE Trans Industr Inform, 2018, 14(7): 3235-3243.
41.	Lu Na, Li Tengfei, Ren Xiaodong, et al. A deep learning scheme for motor imagery classification based on restricted boltzmann machines. IEEE Trans Neur Syst Rehabil Eng, 2017, 25(6): 566-576.

1. Torre L A, Siegel R L, Jemal A. Lung cancer statistics. Adv Exp Med Biol, 2016, 893: 1-19.
2. Chen Wanqing, Zheng Rongshou, Baade P D, et al. Cancer statistics in China, 2015. CA Cancer J Clin, 2016, 66(2): 115-132.
3. Cao Maomao, Chen Wanqing. Epidemiology of lung cancer in China. Thoracic Cancer, 2019, 10(1): 3-7.
4. Kermany D S, Goldbaum M, Cai Wenjia, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell, 2018, 172(5): 1122-1131.
5. Lecun Y, Boser B, Denker J S, et al. Backpropagation applied to handwritten zip code recognition. Neural Comput, 1989, 1(4): 541-551.
6. Lecun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
7. Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Commun ACM, 2017, 60(6): 84-90.
8. Donahue J, Hendricks L A, Rohrbach M A, et al. Long-term recurrent convolutional networks for visual recognition and description. IEEE Trans Pattern Anal Mach Intell, 2017, 39(4): 677-691.
9. Havaei M, Davy A, Warde-Farley D, et al. Brain tumor segmentation with deep neural networks. Med Image Anal, 2017, 35: 18-31.
10. He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 770-778.
11. Szegedy C, Liu W, Jia Y Q, et al. Going deeper with convolutions// 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2015: 1-9.
12. Setio A A A, Ciompi F, Litjens G, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging, 2016, 35(5): 1160-1169.
13. Setio A A A, Traverso A, De Bel T, et al. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge. Med Image Anal, 2017, 42: 1-13.
14. 苗光, 李朝锋. 二维和 3 维卷积神经网络相结合的 CT 图像肺结节检测方法. 激光与光电子学进展, 2018, 55(5): 135-143.
15. Han Y, Ye J C. Framing U-net via deep convolutional framelets: application to sparse-view CT. IEEE Trans Med Imaging, 2018, 37(6): 1418-1429.
16. Ghafoorian M, Karssemeijer N, Heskes T, et al. Deep multi-scale location-aware 3D convolutional neural networks for automated detection of lacunes of presumed vascular origin. Neuroimage Clin, 2017, 14: 391-399.
17. Dou Qi, Chen Hao, Yu Lequan, et al. Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans Biomed Eng, 2017, 64(7): 1558-1567.
18. Pezeshk A, Hamidian S, Petrick N, et al. 3D convolutional neural networks for automatic detection of pulmonary nodules in chest CT. IEEE J Biomed Health Inform, 2018: 2168-2194.
19. Winkels M, Cohen T S. 3D G-CNNs for pulmonary nodule detection. arXiv preprint arXiv: 2018, 1804.04656.
20. Huang X J, Shan J J, Vaidya V, et al. Lung nodule detection in CT using 3D convolutional neural networks. New York: IEEE, 2017: 379-383.
21. Anirudh R, Thiagarajan J J, Bremer T, et al. Lung nodule detection using 3D convolutional neural networks trained on weakly labeled data// Medical Imaging 2016: Computer-Aided Diagnosis. San Diego: International Society for Optics and Photonics, 2016, 9785: 978532.
22. Hamidian S, Sahiner B, Petrick N, et al. 3D convolutional neural network for automatic detection of lung nodules in chest CT// Proceedings of SPIE. Orlando: The International Society for Optical Engineering, 2017, 10134: 1013409.
23. Roth H R, Oda H, Zhou Xiangrong, et al. An application of cascaded 3D fully convolutional networks for medical image segmentation. Comput Med Imaging Graph, 2018, 66: 90-99.
24. Gu Yu, Lu Xiaoqi, Yang Lidong, et al. Automatic lung nodule detection using a 3D deep convolutional neural network combined with a multi-scale prediction strategy in chest CTs. Comput Biol Med, 2018, 103: 220-231.
25. Ren Shaoqing, He Kaiming, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell, 2017, 39(6): 1137-1149.
26. Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation// 2014 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2014: 580-587.
27. He K, Gkioxari G, Dollar P, et al. Mask R-CNN// 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2961-2969.
28. Xie Hongtao, Yang Dongbao, Sun Nannan, et al. Automated pulmonary nodule detection in CT images using deep convolutional neural networks. Pattern Recognit, 2019, 85: 109-119.
29. Ding J, Li A, Hu Z, et al. Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks// International Conference on Medical Image Computing and Computer-Assisted Intervention. Quebec City: ICCAI, 2017: 559-567.
30. Hua K L, Hsu C H, Hidayati H C, et al. Computer-aided classification of lung nodules on computed tomography images via deep learning technique. Onco Targets Ther, 2015, 8: 2015-2022.
31. Liu Shuang, Xie Yiting, Jirapatnakul A, et al. Pulmonary nodule classification in lung cancer screening with three-dimensional convolutional neural networks. J Med Imaging, 2017, 4(4): 041308.
32. Ciompi F, Chung K, Riel S J V, et al. Towards automatic pulmonary nodule management in lung cancer screening with deep learning. Sci Rep, 2017, 7: 46479.
33. Shen Wei, Zhou Mu, Yang Feng, et al. Multi-scale convolutional neural networks for lung nodule classification. Inf Process Med Imaging, 2015, 24: 588-599.
34. 徐久强, 洪丽萍, 朱宏博, 等. 一种用于肺结节恶性度分类的生成对抗网络. 东北大学学报: 自然科学版, 2018, 39(11): 1556-1561.
35. Suarez P L, Sappa A D, Vintimilla B X. Infrared image colorization based on a triplet DCGAN architecture// 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu: IEEE, 2017: 212-217.
36. Mao Xudong, Li Qing, Xie Haoran, et al. Least squares generative adversarial networks// 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2794-2802.
37. Hussein S, Cao K, Song Q, et al. Risk stratification of lung nodules using 3d CNN-based multi-task learning// International Conference on Information Processing in Medical Imaging. Boone: IPMI, 2017: 249-260.
38. Zamir A R, Sax A, Shen W, et al. Taskonomy: Disentangling task transfer learning// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3712-3722.
39. 陈诗慧, 刘维湘, 秦璟, 等. 基于深度学习和医学图像的癌症计算机辅助诊断研究进展. 生物医学工程学杂志, 2017, 34(2): 314-319.
40. Yuan Xiaofeng, Huang Biao, Wang Yalin, et al. Deep learning-based feature representation and its application for soft sensor modeling with variable-wise weighted SAE. IEEE Trans Industr Inform, 2018, 14(7): 3235-3243.
41. Lu Na, Li Tengfei, Ren Xiaodong, et al. A deep learning scheme for motor imagery classification based on restricted boltzmann machines. IEEE Trans Neur Syst Rehabil Eng, 2017, 25(6): 566-576.

Previous Article
Advance of new dressings for promoting skin wound healing
Next Article
In vitro pathological model of Alzheimer's disease based on neuronal network chip and its real-time dynamic analysis

Journal of Biomedical Engineering

A review of deep learning methods for the detection and classification of pulmonary nodules

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content