Pulmonary PET /CT image instance segmentation based on dense interactive feature fusion Mask RCNN_Journal of Biomedical Engineering

Authors：

ZHOU Tao ^1,3 ,  ZHAO Yanan ^1,3 , LU Huiling ² , WANG Yaxing ^1,3 , ZHI Lijia ¹

1. School of Computer Science and Engineering, North Minzu University, Yinchuan 750021, P. R. China;
2. School of Medical Information & Engineering, Ningxia Medical University, Yinchuan 750004, P. R. China;
3. Key Laboratory of Image and Graphics Intelligent Processing of State Ethnic Affairs Commission, North Minzu University, Yinchuan 750021, P. R. China;

Corresponding author：

ZHAO Yanan, Email: zyn1644o@163.com

Keywords：

Positron emission tomography/ computed tomography; Instance segmentation; Interactive fusion; Dense connection; Mask RCNN

DOI：

10.7507/1001-5515.202309026

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

There are some problems in positron emission tomography/ computed tomography (PET/CT) lung images, such as little information of feature pixels in lesion regions, complex and diverse shapes, and blurred boundaries between lesions and surrounding tissues, which lead to inadequate extraction of tumor lesion features by the model. To solve the above problems, this paper proposes a dense interactive feature fusion Mask RCNN (DIF-Mask RCNN) model. Firstly, a feature extraction network with cross-scale backbone and auxiliary structures was designed to extract the features of lesions at different scales. Then, a dense interactive feature enhancement network was designed to enhance the lesion detail information in the deep feature map by interactively fusing the shallowest lesion features with neighboring features and current features in the form of dense connections. Finally, a dense interactive feature fusion feature pyramid network (FPN) network was constructed, and the shallow information was added to the deep features one by one in the bottom-up path with dense connections to further enhance the model’s perception of weak features in the lesion region. The ablation and comparison experiments were conducted on the clinical PET/CT lung image dataset. The results showed that the APdet, APseg, APdet_s and APseg_s indexes of the proposed model were 67.16%, 68.12%, 34.97% and 37.68%, respectively. Compared with Mask RCNN (ResNet50), APdet and APseg indexes increased by 7.11% and 5.14%, respectively. DIF-Mask RCNN model can effectively detect and segment tumor lesions. It provides important reference value and evaluation basis for computer-aided diagnosis of lung cancer.

Citation： ZHOU Tao, ZHAO Yanan, LU Huiling, WANG Yaxing, ZHI Lijia. Pulmonary PET /CT image instance segmentation based on dense interactive feature fusion Mask RCNN. Journal of Biomedical Engineering, 2024, 41(3): 527-534. doi: 10.7507/1001-5515.202309026 Copy

1.	Zhang Y, Dai B, Dong M, et al. A lung cancer detection and recognition method combining convolutional neural network and morphological features// 2022 IEEE 5th International Conference on Computer and Communication Engineering Technology (CCET). Beijing: IEEE, 2022: 145-149.
2.	Xing W, Zhu Z, Hou D, et al. A deep learning-based automatic segmentation approach for medical images by combining convolution and multilayer perceptron. Comput Biol Med, 2022, 147: 105797.
3.	Ridhwan A, Chen E, Selvam H, et al. Analysis and establishment of adult diagnostic reference level for whole-body 18F-FDG PET/CT imaging: A single institutional study. Radiat Phys Chem, 2023, 210: 111022.
4.	Zhou T, Niu Y, Lu H, et al. Vision Transformer: to discover the “Four Secrets” of image patches. Inform Fusion, 2024, 105: 102248.
5.	Redmon J, Divvala S, Girshick R, et al. You Only Look Once: Unified, real-time object detection. arXiv, 2015, 2015: 1506.02640.
6.	Zhou T, Liu F, Ye X, et al. CCGL-YOLOV5: A cross-modal cross-scale global-local attention YOLOV5 lung tumor detection model. Comput Biol Med, 2023, 165: 107387.
7.	Zhou T, Wang H, Du Y, et al. M3YOLOv5: Feature enhanced YOLOv5 model for mandibular fracture detection. Comput Biol Med, 2024, 173: 108291.
8.	Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation// Medical Image Computing and Computer-Assisted Intervention (MICCAI). Munich: Springer, 2015, 9351: 234-241.
9.	Hou S, Zhou T, Liu Y, et al. Teeth U-Net: A segmentation model of dental panoramic X-ray images for context semantics and contrast enhancement. Comput Biol Med, 2023, 152: 106296.
10.	周涛, 赵雅楠, 陆惠玲, 等. 医学图像实例分割: 从有候选区域向无候选区域. 生物医学工程学杂志, 2022, 39(6): 1218-1232.
11.	He K, Gkioxari G, Dollár P, et al. Mask RCNN// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 2980-2988.
12.	Cai Z, Vasconcelos N. Cascade RCNN: High quality object detection and instance segmentation. IEEE Trans Pattern Anal Mach Intell, 2019, 43(5): 1483-1498.
13.	Huang Z, Huang L, Gong Y, et al. Mask Scoring RCNN// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 6402-6411.
14.	Bolya D, Zhou C, Xiao F, et al. YOLACT: Real-time instance segmentation// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 2019: 9156-9165.
15.	Qiu X, Lei H, Xie H, et al. Segmentation of multiple myeloma cells using feature selection pyramid network and semantic cascade Mask RCNN// 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI). Kolkata: IEEE, 2022: 1-4.
16.	Hoorali F, Khosravi H, Moradi B. An automatic method for microscopic diagnosis of diseases based on URCNN. Biomed Signal Process Control, 2023, 80: 104240.
17.	Zhao X, Xu T, Peng L, et al. Recognition and segmentation of teeth and mandibular nerve canals in panoramic dental X-rays by Mask RCNN. Displays, 2023, 78: 102447.
18.	Varadharajan I, Siva R. An efficient lung disease classification from X-ray images using hybrid Mask-RCNN and BiDLSTM. Biomed Signal Process Control, 2023, 81: 104340.
19.	Yang L, Zhang R, Li L, et al. SimAM: A simple, parameter-free attention module for convolutional neural networks// International Conference on Machine Learning (ICML). PMLR, 2021, 139: 11863-11874.
20.	Lin T, Dollar P, Girshick R, et al. Feature pyramid networks for object detection// 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 936-944.

1. Zhang Y, Dai B, Dong M, et al. A lung cancer detection and recognition method combining convolutional neural network and morphological features// 2022 IEEE 5th International Conference on Computer and Communication Engineering Technology (CCET). Beijing: IEEE, 2022: 145-149.
2. Xing W, Zhu Z, Hou D, et al. A deep learning-based automatic segmentation approach for medical images by combining convolution and multilayer perceptron. Comput Biol Med, 2022, 147: 105797.
3. Ridhwan A, Chen E, Selvam H, et al. Analysis and establishment of adult diagnostic reference level for whole-body 18F-FDG PET/CT imaging: A single institutional study. Radiat Phys Chem, 2023, 210: 111022.
4. Zhou T, Niu Y, Lu H, et al. Vision Transformer: to discover the “Four Secrets” of image patches. Inform Fusion, 2024, 105: 102248.
5. Redmon J, Divvala S, Girshick R, et al. You Only Look Once: Unified, real-time object detection. arXiv, 2015, 2015: 1506.02640.
6. Zhou T, Liu F, Ye X, et al. CCGL-YOLOV5: A cross-modal cross-scale global-local attention YOLOV5 lung tumor detection model. Comput Biol Med, 2023, 165: 107387.
7. Zhou T, Wang H, Du Y, et al. M3YOLOv5: Feature enhanced YOLOv5 model for mandibular fracture detection. Comput Biol Med, 2024, 173: 108291.
8. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation// Medical Image Computing and Computer-Assisted Intervention (MICCAI). Munich: Springer, 2015, 9351: 234-241.
9. Hou S, Zhou T, Liu Y, et al. Teeth U-Net: A segmentation model of dental panoramic X-ray images for context semantics and contrast enhancement. Comput Biol Med, 2023, 152: 106296.
10. 周涛, 赵雅楠, 陆惠玲, 等. 医学图像实例分割: 从有候选区域向无候选区域. 生物医学工程学杂志, 2022, 39(6): 1218-1232.
11. He K, Gkioxari G, Dollár P, et al. Mask RCNN// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 2980-2988.
12. Cai Z, Vasconcelos N. Cascade RCNN: High quality object detection and instance segmentation. IEEE Trans Pattern Anal Mach Intell, 2019, 43(5): 1483-1498.
13. Huang Z, Huang L, Gong Y, et al. Mask Scoring RCNN// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 6402-6411.
14. Bolya D, Zhou C, Xiao F, et al. YOLACT: Real-time instance segmentation// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 2019: 9156-9165.
15. Qiu X, Lei H, Xie H, et al. Segmentation of multiple myeloma cells using feature selection pyramid network and semantic cascade Mask RCNN// 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI). Kolkata: IEEE, 2022: 1-4.
16. Hoorali F, Khosravi H, Moradi B. An automatic method for microscopic diagnosis of diseases based on URCNN. Biomed Signal Process Control, 2023, 80: 104240.
17. Zhao X, Xu T, Peng L, et al. Recognition and segmentation of teeth and mandibular nerve canals in panoramic dental X-rays by Mask RCNN. Displays, 2023, 78: 102447.
18. Varadharajan I, Siva R. An efficient lung disease classification from X-ray images using hybrid Mask-RCNN and BiDLSTM. Biomed Signal Process Control, 2023, 81: 104340.
19. Yang L, Zhang R, Li L, et al. SimAM: A simple, parameter-free attention module for convolutional neural networks// International Conference on Machine Learning (ICML). PMLR, 2021, 139: 11863-11874.
20. Lin T, Dollar P, Girshick R, et al. Feature pyramid networks for object detection// 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 936-944.

Journal of Biomedical Engineering

Pulmonary PET /CT image instance segmentation based on dense interactive feature fusion Mask RCNN

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content