Colorectal polyp segmentation method based on fusion of transformer and cross-level phase awareness_Journal of Biomedical Engineering

Authors：

LIANG Liming ¹ , HE Anjun ¹ , ZHU Chenkun ¹ ,  SHENG Xiaoqi ²

1. School of Electrical Engineering and Automation, Jiangxi University of Science and Technology, Ganzhou, Jiangxi 341000, P. R. China;
2. School of Computer Science and Engineering, South China University of Technology, Guangzhou 510000, P. R. China;

Corresponding author：

SHENG Xiaoqi, Email: 18570279448@163.com

Keywords：

Image segmentation; Colorectal polyps; Transformer; Phase-aware fusion module; Position oriented function module

DOI：

10.7507/1001-5515.202211067

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

In order to address the issues of spatial induction bias and lack of effective representation of global contextual information in colon polyp image segmentation, which lead to the loss of edge details and mis-segmentation of lesion areas, a colon polyp segmentation method that combines Transformer and cross-level phase-awareness is proposed. The method started from the perspective of global feature transformation, and used a hierarchical Transformer encoder to extract semantic information and spatial details of lesion areas layer by layer. Secondly, a phase-aware fusion module (PAFM) was designed to capture cross-level interaction information and effectively aggregate multi-scale contextual information. Thirdly, a position oriented functional module (POF) was designed to effectively integrate global and local feature information, fill in semantic gaps, and suppress background noise. Fourthly, a residual axis reverse attention module (RA-IA) was used to improve the network’s ability to recognize edge pixels. The proposed method was experimentally tested on public datasets CVC-ClinicDB, Kvasir, CVC-ColonDB, and EITS, with Dice similarity coefficients of 94.04%, 92.04%, 80.78%, and 76.80%, respectively, and mean intersection over union of 89.31%, 86.81%, 73.55%, and 69.10%, respectively. The simulation experimental results show that the proposed method can effectively segment colon polyp images, providing a new window for the diagnosis of colon polyps.

Citation： LIANG Liming, HE Anjun, ZHU Chenkun, SHENG Xiaoqi. Colorectal polyp segmentation method based on fusion of transformer and cross-level phase awareness. Journal of Biomedical Engineering, 2023, 40(2): 234-243. doi: 10.7507/1001-5515.202211067 Copy

1.	梁礼明, 周珑颂, 冯骏, 等. 基于高分辨率复合网络的皮肤病变分割. 光学精密工程, 2022, 30(16): 2021-2038.
2.	Song P, Li J, Fan H. Attention based multi-scale parallel network for polyp segmentation. Comput Biol Med, 2022, 146: 105476-105476.
3.	Nogueira-Rodríguez A, Dominguez-Carbajales R, Campos-Tato F, et al. Real-time polyp detection model using convolutional neural networks. Neural Comput Appl, 2022, 34(13): 10375-10396.
4.	敬雅冉, 千奕, 蒲天磊, 等. 塑闪阵列探测器读出ASIC阈值产生与调节电路的设计. 电子科技大学学报, 2022, 51(3): 402-407.
5.	Zhao C, Shuai R, Ma L, et al. Segmentation of skin lesions image based on U-Net++. Multimed Tools Appl, 2022, 81(6): 8691-8717.
6.	支佩佩, 邓健志, 钟震霄. 基于卷积和注意力机制的医学细胞核图像分割网络. 生物医学工程学杂志, 2022, 39(4): 730-739.
7.	Poudel S, Lee S W. Deep multi-scale attentional features for medical image segmentation. Appl Soft Comput, 2021, 109: 107445.
8.	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv, 2020: 2010.11929.
9.	Chen J, Lu Y, Yu Q, et al. TransUNet: Transformers make strong encoders for medical image segmentation. arXiv, 2021: 2102.04306.
10.	Dong B, Wang W, Fan D P, et al. Polyp-PVT: Polyp segmentation with pyramid vision transformers. arXiv, 2021: 2108.06932.
11.	Gao Y, Zhou M, Metaxas D N. UTNet: a hybrid transformer architecture for medical image segmentation// de Bruijne M, Cattin P C, Cotin S, et al. Medical Image Computing and Computer Assisted Intervention–MICCAI 2021. Cham: Springer International Publishing, 2021, 12903: 61-71.
12.	Xie E, Wang W, Yu Z, et al. SegFormer: Simple and efficient design for semantic segmentation with transformers. Adv Neural Inf Process Syst, 2021, 34: 12077-12090.
13.	Zhou B, Zhao H, Puig X, et al. Semantic understanding of scenes through the ADE20K dataset. Int J Comput Vis, 2019, 127(3): 302-321.
14.	徐昌佳, 易见兵, 曹锋, 等. 采用DoubleUNet网络的结直肠息肉分割算法. 光学精密工程, 2022, 30(8): 970-983.
15.	Huang X, Zhuo L, Zhang H, et al. Polyp segmentation network with hybrid channel-spatial attention and pyramid global context guided feature fusion. Comput Med Imaging Graph, 2022, 98: 102072.
16.	Fan D P, Ji G P, Zhou T, et al. Pranet: Parallel reverse attention network for polyp segmentation// Martel A L, Abolmaesumi P, Stoyanov D, et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Cham: Springer, 2020, 12266: 263-273.
17.	Lou A, Guan S, Ko H, et al. CaraNet: context axial reverse attention network for segmentation of small medical objects// Medical Imaging 2022: Image Processing. San Diego: SPIE, 2022, 12032: 81-92.
18.	Tang Y, Han K, Guo J, et al. An image patch is a wave: Phase-aware vision MLP// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 10935-10944.
19.	Bernal J, Sánchez F J, Fernández-Esparrach G, et al. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput Med Imaging Graph, 2015, 43: 99-111.
20.	Jha D, Smedsrud P H, Riegler M A, et al. Kvasir-SEG: A segmented polyp dataset// Ro Y M, Cheng W, Kim J, et al. International Conference on Multimedia Modeling. Cham: Springer, 2020: 451-462.
21.	Tajbakhsh N, Gurudu S R, Liang J. Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans Med Imaging, 2015, 35(2): 630-644.
22.	Silva J, Histace A, Romain O, et al. Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer. Int J Comput Assist Radiol Surg, 2014, 9(2): 283-293.
23.	Zhang Z, Liu Q, Wang Y. Road extraction by deep residual U-Net. IEEE Geosci Remote Sens Lett, 2018, 15(5): 749-753.
24.	Wang J, Huang Q, Tang F, et al. Stepwise feature fusion: Local guides global. arXiv, 2022: 2203.03635.
25.	Fang Y, Chen C, Yuan Y, et al. Selective feature aggregation network with area-boundary constraints for polyp segmentation// Shen Dinggang, Liu Tianming, Peters T M, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2019: 302-310.
26.	Zhang R, Li G, Li Z, et al. Adaptive context selection for polyp segmentation// Martel A L, Abolmaesumi P, Stoyanov D, et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Cham: Springer, 2020, 12266: 253-262.
27.	Patel K, Bur A M, Wang G. Enhanced U-Net: A feature enhancement network for polyp segmentation// 2021 18th Conference on Robots and Vision (CRV). Burnaby: IEEE, 2021: 181-188.
28.	Wei J, Hu Y, Zhang R, et al. Shallow attention network for polyp segmentation// de Bruijne M, Cattin P C, Cotin S, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2021, 12901: 699-708.
29.	Qiu Z, Wang Z, Zhang M, et al. BDG-Net: boundary distribution guided network for accurate polyp segmentation// Medical Imaging 2022: Image Processing. San Diego: SPIE, 2022, 12032: 792-799.
30.	Jin Y, Hu Y, Jiang Z, et al. Polyp segmentation with convolutional MLP. Vis Comput, 2022: 1-19. DOI: 10.1007/s00371-022-02630-y.

1. 梁礼明, 周珑颂, 冯骏, 等. 基于高分辨率复合网络的皮肤病变分割. 光学精密工程, 2022, 30(16): 2021-2038.
2. Song P, Li J, Fan H. Attention based multi-scale parallel network for polyp segmentation. Comput Biol Med, 2022, 146: 105476-105476.
3. Nogueira-Rodríguez A, Dominguez-Carbajales R, Campos-Tato F, et al. Real-time polyp detection model using convolutional neural networks. Neural Comput Appl, 2022, 34(13): 10375-10396.
4. 敬雅冉, 千奕, 蒲天磊, 等. 塑闪阵列探测器读出ASIC阈值产生与调节电路的设计. 电子科技大学学报, 2022, 51(3): 402-407.
5. Zhao C, Shuai R, Ma L, et al. Segmentation of skin lesions image based on U-Net++. Multimed Tools Appl, 2022, 81(6): 8691-8717.
6. 支佩佩, 邓健志, 钟震霄. 基于卷积和注意力机制的医学细胞核图像分割网络. 生物医学工程学杂志, 2022, 39(4): 730-739.
7. Poudel S, Lee S W. Deep multi-scale attentional features for medical image segmentation. Appl Soft Comput, 2021, 109: 107445.
8. Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv, 2020: 2010.11929.
9. Chen J, Lu Y, Yu Q, et al. TransUNet: Transformers make strong encoders for medical image segmentation. arXiv, 2021: 2102.04306.
10. Dong B, Wang W, Fan D P, et al. Polyp-PVT: Polyp segmentation with pyramid vision transformers. arXiv, 2021: 2108.06932.
11. Gao Y, Zhou M, Metaxas D N. UTNet: a hybrid transformer architecture for medical image segmentation// de Bruijne M, Cattin P C, Cotin S, et al. Medical Image Computing and Computer Assisted Intervention–MICCAI 2021. Cham: Springer International Publishing, 2021, 12903: 61-71.
12. Xie E, Wang W, Yu Z, et al. SegFormer: Simple and efficient design for semantic segmentation with transformers. Adv Neural Inf Process Syst, 2021, 34: 12077-12090.
13. Zhou B, Zhao H, Puig X, et al. Semantic understanding of scenes through the ADE20K dataset. Int J Comput Vis, 2019, 127(3): 302-321.
14. 徐昌佳, 易见兵, 曹锋, 等. 采用DoubleUNet网络的结直肠息肉分割算法. 光学精密工程, 2022, 30(8): 970-983.
15. Huang X, Zhuo L, Zhang H, et al. Polyp segmentation network with hybrid channel-spatial attention and pyramid global context guided feature fusion. Comput Med Imaging Graph, 2022, 98: 102072.
16. Fan D P, Ji G P, Zhou T, et al. Pranet: Parallel reverse attention network for polyp segmentation// Martel A L, Abolmaesumi P, Stoyanov D, et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Cham: Springer, 2020, 12266: 263-273.
17. Lou A, Guan S, Ko H, et al. CaraNet: context axial reverse attention network for segmentation of small medical objects// Medical Imaging 2022: Image Processing. San Diego: SPIE, 2022, 12032: 81-92.
18. Tang Y, Han K, Guo J, et al. An image patch is a wave: Phase-aware vision MLP// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 10935-10944.
19. Bernal J, Sánchez F J, Fernández-Esparrach G, et al. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput Med Imaging Graph, 2015, 43: 99-111.
20. Jha D, Smedsrud P H, Riegler M A, et al. Kvasir-SEG: A segmented polyp dataset// Ro Y M, Cheng W, Kim J, et al. International Conference on Multimedia Modeling. Cham: Springer, 2020: 451-462.
21. Tajbakhsh N, Gurudu S R, Liang J. Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans Med Imaging, 2015, 35(2): 630-644.
22. Silva J, Histace A, Romain O, et al. Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer. Int J Comput Assist Radiol Surg, 2014, 9(2): 283-293.
23. Zhang Z, Liu Q, Wang Y. Road extraction by deep residual U-Net. IEEE Geosci Remote Sens Lett, 2018, 15(5): 749-753.
24. Wang J, Huang Q, Tang F, et al. Stepwise feature fusion: Local guides global. arXiv, 2022: 2203.03635.
25. Fang Y, Chen C, Yuan Y, et al. Selective feature aggregation network with area-boundary constraints for polyp segmentation// Shen Dinggang, Liu Tianming, Peters T M, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2019: 302-310.
26. Zhang R, Li G, Li Z, et al. Adaptive context selection for polyp segmentation// Martel A L, Abolmaesumi P, Stoyanov D, et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Cham: Springer, 2020, 12266: 253-262.
27. Patel K, Bur A M, Wang G. Enhanced U-Net: A feature enhancement network for polyp segmentation// 2021 18th Conference on Robots and Vision (CRV). Burnaby: IEEE, 2021: 181-188.
28. Wei J, Hu Y, Zhang R, et al. Shallow attention network for polyp segmentation// de Bruijne M, Cattin P C, Cotin S, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2021, 12901: 699-708.
29. Qiu Z, Wang Z, Zhang M, et al. BDG-Net: boundary distribution guided network for accurate polyp segmentation// Medical Imaging 2022: Image Processing. San Diego: SPIE, 2022, 12032: 792-799.
30. Jin Y, Hu Y, Jiang Z, et al. Polyp segmentation with convolutional MLP. Vis Comput, 2022: 1-19. DOI: 10.1007/s00371-022-02630-y.

Previous Article
Segmentation of prostate region in magnetic resonance images based on improved V-Net
Next Article
Primary study on recognition of vascular stiffness based on wavelet scattering neural network

Journal of Biomedical Engineering

Colorectal polyp segmentation method based on fusion of transformer and cross-level phase awareness

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content