Application of semantic segmentation based on convolutional neural network in medical images_Journal of Biomedical Engineering

Authors：

WU Yuchao ,  LIN Lan , WANG Jingxuan , WU Shuicai

College of Life Science and Bio-engineering, Beijing University of Technology, Beijing 100124, P.R.China;

Corresponding author：

LIN Lan, Email: lanlin@bjut.edu.cn

Keywords：

convolutional neural network; semantic segmentation; medical application

DOI：

10.7507/1001-5515.201906067

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

With the rapid development of network structure, convolutional neural networks (CNN) consolidated its position as a leading machine learning tool in the field of image analysis. Therefore, semantic segmentation based on CNN has also become a key high-level task in medical image understanding. This paper reviews the research progress on CNN-based semantic segmentation in the field of medical image. A variety of classical semantic segmentation methods are reviewed, whose contributions and significance are highlighted. On this basis, their applications in the segmentation of some major physiological and pathological anatomical structures are further summarized and discussed. Finally, the open challenges and potential development direction of semantic segmentation based on CNN in the area of medical image are discussed.

Citation： WU Yuchao, LIN Lan, WANG Jingxuan, WU Shuicai. Application of semantic segmentation based on convolutional neural network in medical images. Journal of Biomedical Engineering, 2020, 37(3): 533-540. doi: 10.7507/1001-5515.201906067 Copy

1.	Meiburger K M, Acharya U R, Molinari F. Automated localization and segmentation techniques for B-mode ultrasound images: a review. Comput Biol Med, 2018, 92: 210-235.
2.	Garcia-Garcia A, Orts-Escolano S, Oprea S, et al. A survey on deep learning techniques for image and video semantic segmentation. Appl Soft Comput, 2018, 70: 41-65.
3.	Gu Jiuxiang, Wang Zhenhua, Kuen J, et al. Recent advances in convolutional neural networks. Pattern Recognition, 2018, 77: 354-377.
4.	Zhang Qianru, Meng Zhang, Chen Tinghuan, et al. Recent advances in convolutional neural network acceleration. Neurocomputing, 2019, 323: 37-51.
5.	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015: 3431-3440.
6.	Lateef F, Yassine R. Survey on semantic segmentation using deep learning techniques. Neurocomputing, 2019, 338: 321-348.
7.	Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention, Cham: Springer, 2015: 234-241.
8.	Zhao Hengshuang, Shi Jianping, Qi Xiaojuan, et al. Pyramid scene parsing network//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu: IEEE, 2017: 2881-2890.
9.	Badrinarayanan V, Kendall A, Cipolla R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell, 2017, 39(12): 2481-2495.
10.	Chen L C, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation. arXiv: 1706.05587, (2017-12-05)[2019-06-18]. https://arxiv.org/abs/1706.05587.
11.	Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation//Proceedings of the European Conference on Computer Vision, Munich: Springer, 2018: 801-818.
12.	Bernal J, Kushibar K, Daniel S A, et al. Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review. Artif Intell Med, 2019, 95: 64-81.
13.	Chung S H, Gan K H, Achuthan A, et al. Liver tumor segmentation using triplanar convolutional neural network: a pilot study//10th International Conference on Robotics, Vision, Signal Processing and Power Applications, Singapore: Springer, 2019: 607-614.
14.	Milletari F, Navab N, Ahmadi S A. V-Net: fully convolutional neural networks for volumetric medical image segmentation//2016 Fourth International Conference on 3D Vision (3DV), California: IEEE, 2016: 565-571.
15.	Ghafoorian M, Mehrtash A, Kapur T, et al. Transfer learning for domain adaptation in MRI: application in brain lesion segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention, Cham: Springer, 2017: 516-524.
16.	田苗, 林岚, 张柏雯, 等. 深度学习在神经影像中的应用研究. 中国医疗设备, 2016, 31(12): 4-9.
17.	张柏雯, 林岚, 吴水才. 深度学习在轻度认知障碍转化与分类中的应用分析. 医疗卫生装备, 2017, 38(9): 105-111.
18.	王海鸥, 刘慧, 郭强, 等. 面向医学图像分割的超像素 U-Net 网络设计. 计算机辅助设计与图形学学报, 2019, 31(6): 1007-1017.
19.	Chen Yunjie, Cao Zhihui, Cao Chunzheng, et al. A modified U-Net for brain Mr image segmentation//International Conference on Cloud Computing and Security (ICCC 2018), Cham: Springer, 2018: 233-242.
20.	Nie Dong, Wang Li, Adeli E, et al. 3-D fully convolutional networks for multimodal isointense infant brain image segmentation. IEEE Trans Cybern, 2019, 49(3): 1123-1136.
21.	Wang Shaoyu, Yi Lirong, Qiang Chen, et al. Edge-aware fully convolutional network with CRF-RNN layer for hippocampus segmentation//2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing: IEEE, 2019: 803-806.
22.	Borne L, Mangin J F, Riviere D. Combining 3D U-Net and bottom-up geometric constraints for automatic cortical sulci recognition. Medical Imaging with Deep Learning, 2019. https://openreview.net/forum?id=ryl0VTZCtV.
23.	Cui Shaoguo, Lei Mao, Jiang Jingfeng, et al. Automatic semantic segmentation of brain gliomas from MRI images using a deep cascaded neural network. J Healthc Eng, 2018: 1-14.
24.	Anthimopoulos M, Christodoulidis S, Ebner L, et al. Semantic segmentation of pathological lung tissue with dilated fully convolutional networks. IEEE J Biomed Health Inform, 2019, 23(2): 714-722.
25.	Bouget D, Jørgensen A, Kiss G, et al. Semantic segmentation and detection of mediastinal lymph nodes and anatomical structures in CT data for lung cancer staging. Int J Comput Assist Radiol Surg, 2019, 14(6): 977-986.
26.	Astono I P, Welsh J S, Chalup S. Adjacent network for semantic segmentation of liver CT scans//2018 IEEE 18th International Conference on Bioinformatics and Bioengineering (BIBE), Taichung: IEEE, 2018: 35-40.
27.	李智能, 刘任任, 梁光明. 基于卷积神经网络的医学宫颈细胞图像的语义分割. 计算机应用与软件, 2019, 36(11): 152-156.
28.	Tran T, Kwon O H, Kwon K R, et al. Blood cell images segmentation using deep learning semantic segmentation//2018 IEEE International Conference on Electronics and Communication Engineering (ICECE), Xi'an: IEEE, 2018: 13-16.
29.	Liu Fang, Zhou Zhaoye, Jang H, et al. Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging. Magnetic Resonance in Medicine, 2018, 79(4): 2379-2391.
30.	Edupuganti V G, Chawla A, Amit K. Automatic optic disk and cup segmentation of fundus images using deep learning//2018 25th IEEE International Conference on Image Processing (ICIP), 2018: 2227-2231.
31.	Olah C, Arvind S, Johnson I, et al. The building blocks of interpretability. Distill, 2018, 3(3): e10.
32.	Olah C, Mordvintsev A, Schubert L. Feature visualization. Distill, 2017, 2(11): e7.
33.	Ding Xiuhua, Charnigo R J, Frederick A S, et al. Evaluating trajectories of episodic memory in normal cognition and mild cognitive impairment: results from ADNI. PLoS One, 2019, 14(2): e0212435.
34.	林岚, 吴玉超, 宋爽, 等. LIDC/IDRI 影像数据库在肺结节计算机辅助诊断中的研究进展. 医疗卫生装备, 2018, 39(10): 95-99.

1. Meiburger K M, Acharya U R, Molinari F. Automated localization and segmentation techniques for B-mode ultrasound images: a review. Comput Biol Med, 2018, 92: 210-235.
2. Garcia-Garcia A, Orts-Escolano S, Oprea S, et al. A survey on deep learning techniques for image and video semantic segmentation. Appl Soft Comput, 2018, 70: 41-65.
3. Gu Jiuxiang, Wang Zhenhua, Kuen J, et al. Recent advances in convolutional neural networks. Pattern Recognition, 2018, 77: 354-377.
4. Zhang Qianru, Meng Zhang, Chen Tinghuan, et al. Recent advances in convolutional neural network acceleration. Neurocomputing, 2019, 323: 37-51.
5. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015: 3431-3440.
6. Lateef F, Yassine R. Survey on semantic segmentation using deep learning techniques. Neurocomputing, 2019, 338: 321-348.
7. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention, Cham: Springer, 2015: 234-241.
8. Zhao Hengshuang, Shi Jianping, Qi Xiaojuan, et al. Pyramid scene parsing network//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu: IEEE, 2017: 2881-2890.
9. Badrinarayanan V, Kendall A, Cipolla R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell, 2017, 39(12): 2481-2495.
10. Chen L C, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation. arXiv: 1706.05587, (2017-12-05)[2019-06-18]. https://arxiv.org/abs/1706.05587.
11. Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation//Proceedings of the European Conference on Computer Vision, Munich: Springer, 2018: 801-818.
12. Bernal J, Kushibar K, Daniel S A, et al. Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review. Artif Intell Med, 2019, 95: 64-81.
13. Chung S H, Gan K H, Achuthan A, et al. Liver tumor segmentation using triplanar convolutional neural network: a pilot study//10th International Conference on Robotics, Vision, Signal Processing and Power Applications, Singapore: Springer, 2019: 607-614.
14. Milletari F, Navab N, Ahmadi S A. V-Net: fully convolutional neural networks for volumetric medical image segmentation//2016 Fourth International Conference on 3D Vision (3DV), California: IEEE, 2016: 565-571.
15. Ghafoorian M, Mehrtash A, Kapur T, et al. Transfer learning for domain adaptation in MRI: application in brain lesion segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention, Cham: Springer, 2017: 516-524.
16. 田苗, 林岚, 张柏雯, 等. 深度学习在神经影像中的应用研究. 中国医疗设备, 2016, 31(12): 4-9.
17. 张柏雯, 林岚, 吴水才. 深度学习在轻度认知障碍转化与分类中的应用分析. 医疗卫生装备, 2017, 38(9): 105-111.
18. 王海鸥, 刘慧, 郭强, 等. 面向医学图像分割的超像素 U-Net 网络设计. 计算机辅助设计与图形学学报, 2019, 31(6): 1007-1017.
19. Chen Yunjie, Cao Zhihui, Cao Chunzheng, et al. A modified U-Net for brain Mr image segmentation//International Conference on Cloud Computing and Security (ICCC 2018), Cham: Springer, 2018: 233-242.
20. Nie Dong, Wang Li, Adeli E, et al. 3-D fully convolutional networks for multimodal isointense infant brain image segmentation. IEEE Trans Cybern, 2019, 49(3): 1123-1136.
21. Wang Shaoyu, Yi Lirong, Qiang Chen, et al. Edge-aware fully convolutional network with CRF-RNN layer for hippocampus segmentation//2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing: IEEE, 2019: 803-806.
22. Borne L, Mangin J F, Riviere D. Combining 3D U-Net and bottom-up geometric constraints for automatic cortical sulci recognition. Medical Imaging with Deep Learning, 2019. https://openreview.net/forum?id=ryl0VTZCtV.
23. Cui Shaoguo, Lei Mao, Jiang Jingfeng, et al. Automatic semantic segmentation of brain gliomas from MRI images using a deep cascaded neural network. J Healthc Eng, 2018: 1-14.
24. Anthimopoulos M, Christodoulidis S, Ebner L, et al. Semantic segmentation of pathological lung tissue with dilated fully convolutional networks. IEEE J Biomed Health Inform, 2019, 23(2): 714-722.
25. Bouget D, Jørgensen A, Kiss G, et al. Semantic segmentation and detection of mediastinal lymph nodes and anatomical structures in CT data for lung cancer staging. Int J Comput Assist Radiol Surg, 2019, 14(6): 977-986.
26. Astono I P, Welsh J S, Chalup S. Adjacent network for semantic segmentation of liver CT scans//2018 IEEE 18th International Conference on Bioinformatics and Bioengineering (BIBE), Taichung: IEEE, 2018: 35-40.
27. 李智能, 刘任任, 梁光明. 基于卷积神经网络的医学宫颈细胞图像的语义分割. 计算机应用与软件, 2019, 36(11): 152-156.
28. Tran T, Kwon O H, Kwon K R, et al. Blood cell images segmentation using deep learning semantic segmentation//2018 IEEE International Conference on Electronics and Communication Engineering (ICECE), Xi'an: IEEE, 2018: 13-16.
29. Liu Fang, Zhou Zhaoye, Jang H, et al. Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging. Magnetic Resonance in Medicine, 2018, 79(4): 2379-2391.
30. Edupuganti V G, Chawla A, Amit K. Automatic optic disk and cup segmentation of fundus images using deep learning//2018 25th IEEE International Conference on Image Processing (ICIP), 2018: 2227-2231.
31. Olah C, Arvind S, Johnson I, et al. The building blocks of interpretability. Distill, 2018, 3(3): e10.
32. Olah C, Mordvintsev A, Schubert L. Feature visualization. Distill, 2017, 2(11): e7.
33. Ding Xiuhua, Charnigo R J, Frederick A S, et al. Evaluating trajectories of episodic memory in normal cognition and mild cognitive impairment: results from ADNI. PLoS One, 2019, 14(2): e0212435.
34. 林岚, 吴玉超, 宋爽, 等. LIDC/IDRI 影像数据库在肺结节计算机辅助诊断中的研究进展. 医疗卫生装备, 2018, 39(10): 95-99.

Previous Article
A review on the current state of ball-on-socket type artificial lumbar disc prosthesis
Next Article
Research progress on multiscale entropy algorithm and its application in neural signal analysis

Journal of Biomedical Engineering

Application of semantic segmentation based on convolutional neural network in medical images

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content