Multi-scale medical image segmentation based on pixel encoding and spatial attention mechanism_Journal of Biomedical Engineering

Authors：

WAN Yulong ¹ ,  ZHOU Dongming ¹ , WANG Changcheng ¹ , LIU Yisong ¹ , BAI Chongbin ²

1. School of Information Science and Engineering, Yunnan University, Kunming 650504, P. R. China;
2. The Second People's Hospital of Honghe Prefecture, Jianshui, Yunnan 654300, P. R. China;

Corresponding author：

ZHOU Dongming, Email: zhoudm@ynu.edu.cn

Keywords：

Medical image segmentation; U-Net; Transformer; Multi-scale semantic information; Attention module

DOI：

10.7507/1001-5515.202310001

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

In response to the issues of single-scale information loss and large model parameter size during the sampling process in U-Net and its variants for medical image segmentation, this paper proposes a multi-scale medical image segmentation method based on pixel encoding and spatial attention. Firstly, by redesigning the input strategy of the Transformer structure, a pixel encoding module is introduced to enable the model to extract global semantic information from multi-scale image features, obtaining richer feature information. Additionally, deformable convolutions are incorporated into the Transformer module to accelerate convergence speed and improve module performance. Secondly, a spatial attention module with residual connections is introduced to allow the model to focus on the foreground information of the fused feature maps. Finally, through ablation experiments, the network is lightweighted to enhance segmentation accuracy and accelerate model convergence. The proposed algorithm achieves satisfactory results on the Synapse dataset, an official public dataset for multi-organ segmentation provided by the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), with Dice similarity coefficient (DSC) and 95% Hausdorff distance (HD95) scores of 77.65 and 18.34, respectively. The experimental results demonstrate that the proposed algorithm can enhance multi-organ segmentation performance, potentially filling the gap in multi-scale medical image segmentation algorithms, and providing assistance for professional physicians in diagnosis.

Citation： WAN Yulong, ZHOU Dongming, WANG Changcheng, LIU Yisong, BAI Chongbin. Multi-scale medical image segmentation based on pixel encoding and spatial attention mechanism. Journal of Biomedical Engineering, 2024, 41(3): 511-519. doi: 10.7507/1001-5515.202310001 Copy

1.	徐光宪, 冯春, 马飞. 基于UNet的医学图像分割综述. 计算机科学与探索, 2023, 17(8): 1776-1792.
2.	Pun T. A new method for grey-level picture thresholding using the entropy of the histogram. Signal Processing, 1980, 2(3): 223-237.
3.	Yen J C, Chang F J, Chang S. A new criterion for automatic multilevel thresholding. IEEE Transactions on Image Processing, 1995, 4(3): 370-378.
4.	Batenburg K J, Sijbers J. Adaptive thresholding of tomograms by projection distance minimization. Pattern Recognition, 2009, 42(10): 2297-2305.
5.	Cheng Y. Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995, 17(8): 790-799.
6.	Salvador S, Chan P. Determining the number of clusters/segments in hierarchical clustering/segmentation algorithms//16th IEEE International Conference on Tools with Artificial Intelligence, IEEE, 2004: 576-584.
7.	徐国雄, 张骁, 胡进贤, 等. 基于阈值分割和轮廓提取的图像边缘检测算法. 计算机技术与发展, 2015, 25(12): 64-67,71.
8.	Khan J F, Bhuiyan S M A, Adhami R R. Image segmentation and shape analysis for road-sign detection. IEEE Transactions on Intelligent Transportation Systems, 2010, 12(1): 83-96.
9.	岳欣华,邓彩霞,张兆茹. BP神经网络与形态学融合的边缘检测算法. 哈尔滨理工大学学报, 2021, 26(5): 83-90.
10.	Tremeau A, Borel N. A region growing and merging algorithm to color segmentation. Pattern Recognition, 1997, 30(7): 1191-1203.
11.	Bhargavi K, Jyothi S. A survey on threshold based segmentation technique in image processing. International Journal of Innovative Research and Development, 2014, 3(12): 234-239.
12.	Masood A, Al-Jumaily A A. Fuzzy C mean thresholding based level set for automated segmentation of skin lesions. Journal of Signal and Information Processing, 2013, 4(3): 66.
13.	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3431-3440.
14.	Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation//Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany: Springer International Publishing, 2015: 234-241.
15.	Zhou Z, Rahman Siddiquee M M, Tajbakhsh N, et al. Unet++: a nested U-net architecture for medical image segmentation//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop and 8th International Workshop, Granada, Spain: Springer International Publishing, 2018: 3-11.
16.	Qin X, Zhang Z, Huang C, et al. U2-Net: going deeper with nested U-structure for salient object detection. Pattern Recognition, 2020, 106: 107404.
17.	Oktay O, Schlemper J, Folgoc L L, et al. Attention U-net: learning where to look for the pancreas. arXiv preprint, 2018, arXiv: 1804.03999.
18.	Jin Q, Meng Z, Sun C, et al. RA-UNet: a hybrid deep attention-aware network to extract liver and tumor in CT scans. Frontiers in Bioengineering and Biotechnology, 2020, 8: 605132.
19.	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000-6010.
20.	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: transformers for image recognition at scale. arXiv preprint, 2020, arXiv: 2010.11929.
21.	Chen J, Lu Y, Yu Q, et al. TransUNet: Transformers make strong encoders for medical image segmentation. arXiv preprint, 2021, arXiv: 2102.04306.
22.	Wang H, Cao P, Wang J, et al. Uctransnet: rethinking the skip connections in u-net from a channel-wise perspective with transformer//Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(3): 2441-2449.
23.	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60(6): 84-90.
24.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
25.	Yang H, Zhou D, Cao J, et al. RainFormer: a pyramid transformer for single image deraining. The Journal of Supercomputing, 2023, 79(6): 6115-6140.
26.	Zhu X, Su W, Lu L, et al. Deformable DETR: deformable transformers for end-to-end object detection. arXiv preprint, 2020, arXiv: 2010.04159.
27.	Wang W, Tu Z. Rethinking the value of transformer components. arXiv preprint, 2020, arXiv: 2011.03803.
28.	尹稳, 周冬明, 范腾, 等. 基于密集空洞空间金字塔池化和注意力机制的皮肤病灶图像分割方法. 生物医学工程学杂志, 2022, 39(6): 1108-1116.
29.	陈英, 张伟, 林洪平, 等. 医学图像分割算法的损失函数综述. 生物医学工程学杂志, 2023, 40(2): 392-400.
30.	Milletari F, Navab N, Ahmadi S A. V-net: fully convolutional neural networks for volumetric medical image segmentation//2016 fourth international conference on 3D vision (3DV). IEEE, 2016: 565-571.
31.	Fu S, Lu Y, Wang Y, et al. Domain adaptive relational reasoning for 3D multi-organ segmentation//Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru: Springer International Publishing, 2020: 656-666.

1. 徐光宪, 冯春, 马飞. 基于UNet的医学图像分割综述. 计算机科学与探索, 2023, 17(8): 1776-1792.
2. Pun T. A new method for grey-level picture thresholding using the entropy of the histogram. Signal Processing, 1980, 2(3): 223-237.
3. Yen J C, Chang F J, Chang S. A new criterion for automatic multilevel thresholding. IEEE Transactions on Image Processing, 1995, 4(3): 370-378.
4. Batenburg K J, Sijbers J. Adaptive thresholding of tomograms by projection distance minimization. Pattern Recognition, 2009, 42(10): 2297-2305.
5. Cheng Y. Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995, 17(8): 790-799.
6. Salvador S, Chan P. Determining the number of clusters/segments in hierarchical clustering/segmentation algorithms//16th IEEE International Conference on Tools with Artificial Intelligence, IEEE, 2004: 576-584.
7. 徐国雄, 张骁, 胡进贤, 等. 基于阈值分割和轮廓提取的图像边缘检测算法. 计算机技术与发展, 2015, 25(12): 64-67,71.
8. Khan J F, Bhuiyan S M A, Adhami R R. Image segmentation and shape analysis for road-sign detection. IEEE Transactions on Intelligent Transportation Systems, 2010, 12(1): 83-96.
9. 岳欣华,邓彩霞,张兆茹. BP神经网络与形态学融合的边缘检测算法. 哈尔滨理工大学学报, 2021, 26(5): 83-90.
10. Tremeau A, Borel N. A region growing and merging algorithm to color segmentation. Pattern Recognition, 1997, 30(7): 1191-1203.
11. Bhargavi K, Jyothi S. A survey on threshold based segmentation technique in image processing. International Journal of Innovative Research and Development, 2014, 3(12): 234-239.
12. Masood A, Al-Jumaily A A. Fuzzy C mean thresholding based level set for automated segmentation of skin lesions. Journal of Signal and Information Processing, 2013, 4(3): 66.
13. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3431-3440.
14. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation//Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany: Springer International Publishing, 2015: 234-241.
15. Zhou Z, Rahman Siddiquee M M, Tajbakhsh N, et al. Unet++: a nested U-net architecture for medical image segmentation//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop and 8th International Workshop, Granada, Spain: Springer International Publishing, 2018: 3-11.
16. Qin X, Zhang Z, Huang C, et al. U2-Net: going deeper with nested U-structure for salient object detection. Pattern Recognition, 2020, 106: 107404.
17. Oktay O, Schlemper J, Folgoc L L, et al. Attention U-net: learning where to look for the pancreas. arXiv preprint, 2018, arXiv: 1804.03999.
18. Jin Q, Meng Z, Sun C, et al. RA-UNet: a hybrid deep attention-aware network to extract liver and tumor in CT scans. Frontiers in Bioengineering and Biotechnology, 2020, 8: 605132.
19. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000-6010.
20. Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: transformers for image recognition at scale. arXiv preprint, 2020, arXiv: 2010.11929.
21. Chen J, Lu Y, Yu Q, et al. TransUNet: Transformers make strong encoders for medical image segmentation. arXiv preprint, 2021, arXiv: 2102.04306.
22. Wang H, Cao P, Wang J, et al. Uctransnet: rethinking the skip connections in u-net from a channel-wise perspective with transformer//Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(3): 2441-2449.
23. Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60(6): 84-90.
24. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
25. Yang H, Zhou D, Cao J, et al. RainFormer: a pyramid transformer for single image deraining. The Journal of Supercomputing, 2023, 79(6): 6115-6140.
26. Zhu X, Su W, Lu L, et al. Deformable DETR: deformable transformers for end-to-end object detection. arXiv preprint, 2020, arXiv: 2010.04159.
27. Wang W, Tu Z. Rethinking the value of transformer components. arXiv preprint, 2020, arXiv: 2011.03803.
28. 尹稳, 周冬明, 范腾, 等. 基于密集空洞空间金字塔池化和注意力机制的皮肤病灶图像分割方法. 生物医学工程学杂志, 2022, 39(6): 1108-1116.
29. 陈英, 张伟, 林洪平, 等. 医学图像分割算法的损失函数综述. 生物医学工程学杂志, 2023, 40(2): 392-400.
30. Milletari F, Navab N, Ahmadi S A. V-net: fully convolutional neural networks for volumetric medical image segmentation//2016 fourth international conference on 3D vision (3DV). IEEE, 2016: 565-571.
31. Fu S, Lu Y, Wang Y, et al. Domain adaptive relational reasoning for 3D multi-organ segmentation//Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru: Springer International Publishing, 2020: 656-666.

Journal of Biomedical Engineering

Multi-scale medical image segmentation based on pixel encoding and spatial attention mechanism

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content