A three dimensional convolutional neural network pulmonary nodule detection algorithm based on the multi-scale attention mechanism_Journal of Biomedical Engineering

Authors：

ZHAO Yudu , PENG Zhenwei , MA Jun , XIA Hao ,  WAN Honglin

Key Laboratory of Medical Physical Image Processing Technology, School of Physics and Electronic Science, Shandong Normal University, Jinan 250358, P. R. China;

Corresponding author：

WAN Honglin, Email: visage1979@sdu.edu.cn

Keywords：

Pulmonary nodule detection; Multi-scale feature extraction; Attention mechanism; Three dimensional convolutional neural network

DOI：

10.7507/1001-5515.202011058

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Early screening based on computed tomography (CT) pulmonary nodule detection is an important means to reduce lung cancer mortality, and in recent years three dimensional convolutional neural network (3D CNN) has achieved success and continuous development in the field of lung nodule detection. We proposed a pulmonary nodule detection algorithm by using 3D CNN based on a multi-scale attention mechanism. Aiming at the characteristics of different sizes and shapes of lung nodules, we designed a multi-scale feature extraction module to extract the corresponding features of different scales. Through the attention module, the correlation information between the features was mined from both spatial and channel perspectives to strengthen the features. The extracted features entered into a pyramid-similar fusion mechanism, so that the features would contain both deep semantic information and shallow location information, which is more conducive to target positioning and bounding box regression. On representative LUNA16 datasets, compared with other advanced methods, this method significantly improved the detection sensitivity, which can provide theoretical reference for clinical medicine.

Citation： ZHAO Yudu, PENG Zhenwei, MA Jun, XIA Hao, WAN Honglin. A three dimensional convolutional neural network pulmonary nodule detection algorithm based on the multi-scale attention mechanism. Journal of Biomedical Engineering, 2022, 39(2): 320-328. doi: 10.7507/1001-5515.202011058 Copy

1.	Ferlay J, Jacques I, Dikshit R, et al. Cancer incidence and mortality worldwide: sources methods and major patterns in GLOBOCAN 2012. Int J Cancer, 2015, 136(5): 359-386.
2.	Aberle D R, Adams A, MBerg C D, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. New Engl J Med, 2011, 365(5): 395-409.
3.	Setio A, Ciompi F, Litjens G, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging, 2016, 35(5): 1160-1169.
4.	Li C, Zhu G, Wu X, et al. False-positive reduction on lung nodules detection in chest radiographs by ensemble of convolutional neural networks. IEEE Access, 2018, 6: 16060-16067.
5.	Zhao J, Zhang C, Li D, et al. Combining multi-scale feature fusion with multi-attribute grading, a CNN model for benign and malignant classification of pulmonary nodules. J Digit Imaging, 2020, 33(4): 869-878.
6.	Zhang Y, Yi P, Zhou D, et al. CSANet: channel and spatial mixed attention CNN for pedestrian detection. IEEE Access, 2020, 8: 76243-76252.
7.	Ding J, Li A, Hu Z, et al. Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks// 2017 Proceedings of the 20th Medical Image Computing and Computer Assisted Intervention. Quebec City: MICCAI, 2017: 559-567.
8.	Zhu W, Liu C, Fan W. Deep lung: 3D deep convolutional nets for automated pulmonary nodule detection and classification// 2018 Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe: IEEE, 2018: 673-681.
9.	Lin T Y, Dollar P, Girshick R, et al. Feature pyramid networks for object detection. IEEE Access, 2018, 13(7): 1525-1528.
10.	Ozdemir O, Russell R L, Berlin A. A 3D probabilistic deep learning system for detection and diagnosis of lung cancer using low-dose CT scans. IEEE Trans Med Imaging, 2020, 39(5): 1419-1429.
11.	Tajbakhsh N, Shin J Y, Gurudu S R. Convolutional neural networks for medical image analysis: full training or fine tuning. IEEE Trans Med Imaging, 2016, 35(5): 1299-1312.
12.	Knight S B, Crosbie P A, Balata H. Progress and prospects of early detection in lung cancer. Open Biol, 2017, 67(6): 1256-1272.
13.	Zheng S, Guo J, Cui X. Automatic pulmonary nodule detection in CT scans using convolutional neural networks based on maximum intensity projection. IEEE Trans Neur Net Lear, 2019, 39(3): 797-805.
14.	Girshick R, Donahue J, Darrell T. Rich feature hierarchies for accurate object detection and semantic segmentation// 2014 Proceedings of the 27th IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 580-587.
15.	Cao H, Liu L, Song E. A two-stage convolutional neural networks for lung nodule detection. IEEE Trans Med Imaging, 2020, 24(7): 2006-2015.
16.	Xu Q, Xiao Y, Wang D. CSA-MSO3DCNN: multiscale octave 3D CNN with channel and spatial attention for hyperspectral image classification. Remote Sensing, 2020, 12(1): 188-196.
17.	Woo S, Park J, Lee J. CBAM: convolutional block attention module// 2018 Proceedings of the 15th European Conference on Computer Vision. Munich: IEEE, 2018: 3-19.
18.	Fu J, Liu J, Tian H. Dual attention network for scene segmentation// 2019 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 1-18.
19.	Tang H, Kim D, Xie X, et al. Automated pulmonary nodule detection in CT images using deep convolutional neural networks// 2018 Proceedings of the 15th International Symposium on Biomedical Imaging. Washington: IEEE, 2018: 123-134.
20.	Prasoon A, Petersen K, Igel C. Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network// 2013 Proceedings of the 16th Medical Image Computing and Computer-Assisted Intervention. Berlin: MICCAI, 2013: 246-253.
21.	Torre L A, Bray F, Siegel R L. Global cancer statistics, 2012. CA Cancer J Clin, 2015, 65(2): 87-108.
22.	Zuo W, Zhou F, Li Z. Multi-resolution CNN and knowledge transfer for candidate classification in lung nodule detection. IEEE Access, 2019: 32510-32521.
23.	Tan M, Wu F, Yang B. Pulmonary nodule detection using hybrid two-stage 3D CNNs. Med Phys, 2020, 47(8): 3376-3388.
24.	Hara K, Kataoka H, Satoh Y. Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and imagenet? // 2018 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Lake Tahoe: IEEE, 2018: 18-22.
25.	Dou Q, Chen H, Yu L. Multilevel contextual 3D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans Bio Med Eng, 2017, 64(7): 1558-1567.
26.	Liao F, Liang M, Li Z. Evaluate the malignancy of pulmonary nodules using the 3D deep leaky noisy-or network. IEEE Trans Neur Net Lear, 2019, 30(11): 3484-3495.
27.	Bray F, Ferlay J, Soerjomataram I. Global cancer statistics 2018: global estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2018, 68(6): 394-424.
28.	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks// 2012 Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe: IEEE, 2012: 1097-1105.
29.	Shi Y, Li H, Zhang H, et al. Accurate and efficient LIF-Nets for 3D detection and recognition. IEEE Access, 2020, 8: 98562-98571.
30.	Xie H, Yang D, Sun N, et al. Automated pulmonary nodule detection in CT images using deep convolutional neural networks. Pat Rec, 2018, 85(8): 109-119.
31.	Hu J, Shen L, Albanie S. Squeeze-and-excitation networks. IEEE Trans Med Imaging, 2020, 42(8): 2011-2023.
32.	Pezeshk A, Hamidian S, Petrick N. 3D convolutional neural networks for automatic detection of pulmonary nodules in chest CT. IEEE J Biomed Health, 2019, 23(5): 2080-2090.
33.	Eun H, Kim D, Jung C. Single-view 2D CNNs with fully automatic non-nodule categorization for false positive reduction in pulmonary nodule detection. Comput Meth Prog Bio, 2018, 165(14): 215-224.
34.	Fang F, Li L, Zhu H. Combining faster R-CNN and model-driven clustering for elongated object detection. IEEE Trans Med Imaging, 2019, 16(4): 2052-2065.

1. Ferlay J, Jacques I, Dikshit R, et al. Cancer incidence and mortality worldwide: sources methods and major patterns in GLOBOCAN 2012. Int J Cancer, 2015, 136(5): 359-386.
2. Aberle D R, Adams A, MBerg C D, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. New Engl J Med, 2011, 365(5): 395-409.
3. Setio A, Ciompi F, Litjens G, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging, 2016, 35(5): 1160-1169.
4. Li C, Zhu G, Wu X, et al. False-positive reduction on lung nodules detection in chest radiographs by ensemble of convolutional neural networks. IEEE Access, 2018, 6: 16060-16067.
5. Zhao J, Zhang C, Li D, et al. Combining multi-scale feature fusion with multi-attribute grading, a CNN model for benign and malignant classification of pulmonary nodules. J Digit Imaging, 2020, 33(4): 869-878.
6. Zhang Y, Yi P, Zhou D, et al. CSANet: channel and spatial mixed attention CNN for pedestrian detection. IEEE Access, 2020, 8: 76243-76252.
7. Ding J, Li A, Hu Z, et al. Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks// 2017 Proceedings of the 20th Medical Image Computing and Computer Assisted Intervention. Quebec City: MICCAI, 2017: 559-567.
8. Zhu W, Liu C, Fan W. Deep lung: 3D deep convolutional nets for automated pulmonary nodule detection and classification// 2018 Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe: IEEE, 2018: 673-681.
9. Lin T Y, Dollar P, Girshick R, et al. Feature pyramid networks for object detection. IEEE Access, 2018, 13(7): 1525-1528.
10. Ozdemir O, Russell R L, Berlin A. A 3D probabilistic deep learning system for detection and diagnosis of lung cancer using low-dose CT scans. IEEE Trans Med Imaging, 2020, 39(5): 1419-1429.
11. Tajbakhsh N, Shin J Y, Gurudu S R. Convolutional neural networks for medical image analysis: full training or fine tuning. IEEE Trans Med Imaging, 2016, 35(5): 1299-1312.
12. Knight S B, Crosbie P A, Balata H. Progress and prospects of early detection in lung cancer. Open Biol, 2017, 67(6): 1256-1272.
13. Zheng S, Guo J, Cui X. Automatic pulmonary nodule detection in CT scans using convolutional neural networks based on maximum intensity projection. IEEE Trans Neur Net Lear, 2019, 39(3): 797-805.
14. Girshick R, Donahue J, Darrell T. Rich feature hierarchies for accurate object detection and semantic segmentation// 2014 Proceedings of the 27th IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 580-587.
15. Cao H, Liu L, Song E. A two-stage convolutional neural networks for lung nodule detection. IEEE Trans Med Imaging, 2020, 24(7): 2006-2015.
16. Xu Q, Xiao Y, Wang D. CSA-MSO3DCNN: multiscale octave 3D CNN with channel and spatial attention for hyperspectral image classification. Remote Sensing, 2020, 12(1): 188-196.
17. Woo S, Park J, Lee J. CBAM: convolutional block attention module// 2018 Proceedings of the 15th European Conference on Computer Vision. Munich: IEEE, 2018: 3-19.
18. Fu J, Liu J, Tian H. Dual attention network for scene segmentation// 2019 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 1-18.
19. Tang H, Kim D, Xie X, et al. Automated pulmonary nodule detection in CT images using deep convolutional neural networks// 2018 Proceedings of the 15th International Symposium on Biomedical Imaging. Washington: IEEE, 2018: 123-134.
20. Prasoon A, Petersen K, Igel C. Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network// 2013 Proceedings of the 16th Medical Image Computing and Computer-Assisted Intervention. Berlin: MICCAI, 2013: 246-253.
21. Torre L A, Bray F, Siegel R L. Global cancer statistics, 2012. CA Cancer J Clin, 2015, 65(2): 87-108.
22. Zuo W, Zhou F, Li Z. Multi-resolution CNN and knowledge transfer for candidate classification in lung nodule detection. IEEE Access, 2019: 32510-32521.
23. Tan M, Wu F, Yang B. Pulmonary nodule detection using hybrid two-stage 3D CNNs. Med Phys, 2020, 47(8): 3376-3388.
24. Hara K, Kataoka H, Satoh Y. Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and imagenet? // 2018 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Lake Tahoe: IEEE, 2018: 18-22.
25. Dou Q, Chen H, Yu L. Multilevel contextual 3D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans Bio Med Eng, 2017, 64(7): 1558-1567.
26. Liao F, Liang M, Li Z. Evaluate the malignancy of pulmonary nodules using the 3D deep leaky noisy-or network. IEEE Trans Neur Net Lear, 2019, 30(11): 3484-3495.
27. Bray F, Ferlay J, Soerjomataram I. Global cancer statistics 2018: global estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2018, 68(6): 394-424.
28. Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks// 2012 Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe: IEEE, 2012: 1097-1105.
29. Shi Y, Li H, Zhang H, et al. Accurate and efficient LIF-Nets for 3D detection and recognition. IEEE Access, 2020, 8: 98562-98571.
30. Xie H, Yang D, Sun N, et al. Automated pulmonary nodule detection in CT images using deep convolutional neural networks. Pat Rec, 2018, 85(8): 109-119.
31. Hu J, Shen L, Albanie S. Squeeze-and-excitation networks. IEEE Trans Med Imaging, 2020, 42(8): 2011-2023.
32. Pezeshk A, Hamidian S, Petrick N. 3D convolutional neural networks for automatic detection of pulmonary nodules in chest CT. IEEE J Biomed Health, 2019, 23(5): 2080-2090.
33. Eun H, Kim D, Jung C. Single-view 2D CNNs with fully automatic non-nodule categorization for false positive reduction in pulmonary nodule detection. Comput Meth Prog Bio, 2018, 165(14): 215-224.
34. Fang F, Li L, Zhu H. Combining faster R-CNN and model-driven clustering for elongated object detection. IEEE Trans Med Imaging, 2019, 16(4): 2052-2065.

Journal of Biomedical Engineering

A three dimensional convolutional neural network pulmonary nodule detection algorithm based on the multi-scale attention mechanism

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content