Confocal probe localization algorithm based on region growing and endoscope size prior_Journal of Biomedical Engineering

Authors：

LIU Yuying , WANG Yifan ,  ZUO Siyang

School of Mechanical Engineering, Tianjin University, Tianjin 300350, P. R. China;

Corresponding author：

ZUO Siyang, Email: siyang_zuo@tju.edu.cn

Keywords：

Confocal probe localization; Region growing; Endoscope size prior

DOI：

10.7507/1001-5515.202205036

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Confocal laser endomicroscopy technology can obtain cell-level images in real time and in situ, which can assist doctors in real-time intraoperative diagnosis, but its non-invasiveness makes it difficult to relocate the optical biopsy site. The confocal probe localization algorithm can automatically calculate the coordinates of the probe tip, that is, the coordinates of the optical biopsy site. In this paper, a confocal probe localization algorithm based on region growing and endoscope size prior was proposed. The algorithm detected the probe region by region growing on the probe edge image, then searched for tip points based on a given probe axis, and iteratively optimized it. Finally, based on the single-degree-of-freedom motion characteristics of the probe, the three-dimensional coordinates of the tip of the probe were calculated by using the prior information of the size of the endoscope, which solved the scale uncertainty problem of the monocular camera. The confocal probe localization algorithm was tested on the dataset collected in this paper. The results showed that our algorithm no longer relied on the color information of the probe, avoided the influence of uneven illumination on the gray value of the probe pixels, and had a more robust location accuracy and running speed. Within the length of the probe extending out of the endoscope from 0 to 5 cm, the pixel error could be as low as 11.76 pixels, and the average relative position error could be as low as 1.66 mm, which can achieve the real-time and accurate localization of the confocal probe.

Citation： LIU Yuying, WANG Yifan, ZUO Siyang. Confocal probe localization algorithm based on region growing and endoscope size prior. Journal of Biomedical Engineering, 2022, 39(5): 945-957. doi: 10.7507/1001-5515.202205036 Copy

1.	Liu J T C, Loewke N O, Mandella M J, et al. Point-of-care pathology with miniature microscopes. Anal Cell Pathol, 2011, 34(3): 81-98.
2.	王娟娟, 魏学红. 激光共聚焦显微技术在共定位应用中的常见问题. 影像科学与光化学, 2018, 36(6): 532-538.
3.	Gong L, Wang H, Zuo S. Intensity-based nonrigid endomicroscopic image mosaicking incorporating texture relevance for compensation of tissue deformation. Comput Biol Med, 2022, 142: 105169.
4.	Gong L, Zheng J, Ping Z, et al. Robust mosaicing of endomicroscopic videos via context-weighted correlation ratio. IEEE T Bio-Med Eng, 2020, 68(2): 579-591.
5.	Zehri A H, Wyatt Ramey J F G, Mooney M A, et al. Neurosurgical confocal endomicroscopy: a review of contrast agents, confocal systems, and future imaging modalities. Surg Neurol Int, 2014, 5(1): 60.
6.	Wang K K, Carr-Locke D L, Singh S K, et al. Use of probe-based confocal laser endomicroscopy (pCLE) in gastrointestinal applications. A consensus report based on clinical evidence. United Eur Gastroent, 2015, 3(3): 230-254.
7.	De Palma G D, Esposito D, Luglio G, et al. Confocal laser endomicroscopy in breast surgery: a pilot study. BMC Cancer, 2015, 15(1): 1-7.
8.	Zhang H P, Yang S, Chen W H, et al. The diagnostic value of confocal laser endomicroscopy for gastric cancer and precancerous lesions among Asian population: a system review and meta-analysis. Scand J Gastroentero, 2017, 52(4): 382-388.
9.	He X, Liu D, Sun L. Diagnostic performance of confocal laser endomicroscopy for optical diagnosis of gastric intestinal metaplasia: a meta-analysis. BMC Gastroenterol, 2016, 16(1): 1-8.
10.	Shah T, Lippman R, Kohli D, et al. Accuracy of probe-based confocal laser endomicroscopy (pCLE) compared to random biopsies during endoscopic surveillance of Barrett’s esophagus. Endosc Int Open, 2018, 6(04): E414-E420.
11.	Allain B, Hu M, Lovat L B, et al. A system for biopsy site re-targeting with uncertainty in gastroenterology and oropharyngeal examinations// Jiang T, Navab N, Pluim J P W, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin, Heidelberg: Springer Berlin Heidelberg, 2010: 514-521.
12.	Allain B, Hu M, Lovat L B, et al. Re-localisation of a biopsy site in endoscopic images and characterisation of its uncertainty. Med Image Anal, 2012, 16(2): 482-496.
13.	Hartley R, Zisserman A. Multiple view geometry in computer vision. New York: Cambridge University Press, 2003.
14.	Mouton A, Ye M, Lacombe F, et al. Hybrid retargeting for high-speed targeted optical biopsies// Navab N, Hornegger J, Wells W, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Cham: Springer International Publishing, 2015: 471-479.
15.	Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision// Hayes P J. Proceedings of 7th International Joint Conference on Artificial Intelligence (IJCAI). San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1981: 674-679.
16.	Lucas B D. Generalized image matching by the method of differences. Pittsburgh: Carnegie Mellon University, 1985: 1-24.
17.	Yedjour H. Optical flow based on Lucas-Kanade method for motion estimation// Hatti M. International Conference in Artificial Intelligence in Renewable Energetic Systems. Cham: Springer International Publishing, 2020: 937-945.
18.	Kalal Z, Mikolajczyk K, Matas J. Tracking-learning-detection. IEEE T Pattern Anal, 2011, 34(7): 1409-1422.
19.	Mountney P, Giannarou S, Elson D, et al. Optical biopsy mapping for minimally invasive cancer screening// Yang G Z, Hawkes D, Rueckert D, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin, Heidelberg: Springer Berlin Heidelberg, 2009: 483-490.
20.	Wengert C, Bossard L, Häberling A, et al. Endoscopic navigation for minimally invasive suturing// Ayache N, Ourselin S, Maeder A. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin, Heidelberg: Springer Berlin Heidelberg, 2007: 620-627.
21.	Krupa A, Gangloff J, Doignon C, et al. Autonomous 3-D positioning of surgical instruments in robotized laparoscopic surgery using visual servoing. IEEE T Robotic Autom, 2003, 19(5): 842-853.
22.	袁益琴, 何国金, 王桂周, 等. 背景差分与帧间差分相融合的遥感卫星视频运动车辆检测方法. 中国科学院大学学报, 2018, 35(1): 50.
23.	王小鹏, 文昊天, 王伟, 等. 形态学边缘检测和区域生长相结合的遥感图像水体分割. 测绘科学技术学报, 2019, 36(2): 149-154.
24.	Sadad T, Munir A, Saba T, et al. Fuzzy C-means and region growing based classification of tumor from mammograms using hybrid texture feature. J Comput Sci-Neth, 2018, 29: 34-45.
25.	凌志豪, 朱锟鹏. 基于 Hough 变换与 CV 模型的微细铣刀边界提取. 组合机床与自动化加工技术, 2022 (11): 48-52, 57.
26.	Zheng F, Luo S, Song K, et al. Improved lane line detection algorithm based on Hough transform. Pattern Recognit Image Anal, 2018, 28(2): 254-260.
27.	Qi M, Zhang B, Xu Y, et al. Linear camera calibration by single image based on distortion correction// Proceedings of the 2nd International Conference on Graphics and Signal Processing (ICGSP). New York: Association for Computing Machinery, 2018: 21-25.
28.	Iqbal B, Iqbal W, Khan N, et al. Canny edge detection and Hough transform for high resolution video streams using Hadoop and Spark. Cluster Comput, 2020, 23(1): 397-408.
29.	Yamada A, Iizuka T, Kikuchi D, et al. Sa1639 endoscopic features of early gastric cancers with the newly developed narrow band imaging endoscopy, the Evis Lucera Elite (Olympus Co. ). Gastrointest Endosc, 2014, 79(5): AB285.
30.	Zhao Z, Tse Z T H. An electromagnetic tracking needle clip: an enabling design for low-cost image-guided therapy. Minim Invasiv Ther, 2019, 28(3): 165-171.

1. Liu J T C, Loewke N O, Mandella M J, et al. Point-of-care pathology with miniature microscopes. Anal Cell Pathol, 2011, 34(3): 81-98.
2. 王娟娟, 魏学红. 激光共聚焦显微技术在共定位应用中的常见问题. 影像科学与光化学, 2018, 36(6): 532-538.
3. Gong L, Wang H, Zuo S. Intensity-based nonrigid endomicroscopic image mosaicking incorporating texture relevance for compensation of tissue deformation. Comput Biol Med, 2022, 142: 105169.
4. Gong L, Zheng J, Ping Z, et al. Robust mosaicing of endomicroscopic videos via context-weighted correlation ratio. IEEE T Bio-Med Eng, 2020, 68(2): 579-591.
5. Zehri A H, Wyatt Ramey J F G, Mooney M A, et al. Neurosurgical confocal endomicroscopy: a review of contrast agents, confocal systems, and future imaging modalities. Surg Neurol Int, 2014, 5(1): 60.
6. Wang K K, Carr-Locke D L, Singh S K, et al. Use of probe-based confocal laser endomicroscopy (pCLE) in gastrointestinal applications. A consensus report based on clinical evidence. United Eur Gastroent, 2015, 3(3): 230-254.
7. De Palma G D, Esposito D, Luglio G, et al. Confocal laser endomicroscopy in breast surgery: a pilot study. BMC Cancer, 2015, 15(1): 1-7.
8. Zhang H P, Yang S, Chen W H, et al. The diagnostic value of confocal laser endomicroscopy for gastric cancer and precancerous lesions among Asian population: a system review and meta-analysis. Scand J Gastroentero, 2017, 52(4): 382-388.
9. He X, Liu D, Sun L. Diagnostic performance of confocal laser endomicroscopy for optical diagnosis of gastric intestinal metaplasia: a meta-analysis. BMC Gastroenterol, 2016, 16(1): 1-8.
10. Shah T, Lippman R, Kohli D, et al. Accuracy of probe-based confocal laser endomicroscopy (pCLE) compared to random biopsies during endoscopic surveillance of Barrett’s esophagus. Endosc Int Open, 2018, 6(04): E414-E420.
11. Allain B, Hu M, Lovat L B, et al. A system for biopsy site re-targeting with uncertainty in gastroenterology and oropharyngeal examinations// Jiang T, Navab N, Pluim J P W, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin, Heidelberg: Springer Berlin Heidelberg, 2010: 514-521.
12. Allain B, Hu M, Lovat L B, et al. Re-localisation of a biopsy site in endoscopic images and characterisation of its uncertainty. Med Image Anal, 2012, 16(2): 482-496.
13. Hartley R, Zisserman A. Multiple view geometry in computer vision. New York: Cambridge University Press, 2003.
14. Mouton A, Ye M, Lacombe F, et al. Hybrid retargeting for high-speed targeted optical biopsies// Navab N, Hornegger J, Wells W, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Cham: Springer International Publishing, 2015: 471-479.
15. Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision// Hayes P J. Proceedings of 7th International Joint Conference on Artificial Intelligence (IJCAI). San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1981: 674-679.
16. Lucas B D. Generalized image matching by the method of differences. Pittsburgh: Carnegie Mellon University, 1985: 1-24.
17. Yedjour H. Optical flow based on Lucas-Kanade method for motion estimation// Hatti M. International Conference in Artificial Intelligence in Renewable Energetic Systems. Cham: Springer International Publishing, 2020: 937-945.
18. Kalal Z, Mikolajczyk K, Matas J. Tracking-learning-detection. IEEE T Pattern Anal, 2011, 34(7): 1409-1422.
19. Mountney P, Giannarou S, Elson D, et al. Optical biopsy mapping for minimally invasive cancer screening// Yang G Z, Hawkes D, Rueckert D, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin, Heidelberg: Springer Berlin Heidelberg, 2009: 483-490.
20. Wengert C, Bossard L, Häberling A, et al. Endoscopic navigation for minimally invasive suturing// Ayache N, Ourselin S, Maeder A. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin, Heidelberg: Springer Berlin Heidelberg, 2007: 620-627.
21. Krupa A, Gangloff J, Doignon C, et al. Autonomous 3-D positioning of surgical instruments in robotized laparoscopic surgery using visual servoing. IEEE T Robotic Autom, 2003, 19(5): 842-853.
22. 袁益琴, 何国金, 王桂周, 等. 背景差分与帧间差分相融合的遥感卫星视频运动车辆检测方法. 中国科学院大学学报, 2018, 35(1): 50.
23. 王小鹏, 文昊天, 王伟, 等. 形态学边缘检测和区域生长相结合的遥感图像水体分割. 测绘科学技术学报, 2019, 36(2): 149-154.
24. Sadad T, Munir A, Saba T, et al. Fuzzy C-means and region growing based classification of tumor from mammograms using hybrid texture feature. J Comput Sci-Neth, 2018, 29: 34-45.
25. 凌志豪, 朱锟鹏. 基于 Hough 变换与 CV 模型的微细铣刀边界提取. 组合机床与自动化加工技术, 2022 (11): 48-52, 57.
26. Zheng F, Luo S, Song K, et al. Improved lane line detection algorithm based on Hough transform. Pattern Recognit Image Anal, 2018, 28(2): 254-260.
27. Qi M, Zhang B, Xu Y, et al. Linear camera calibration by single image based on distortion correction// Proceedings of the 2nd International Conference on Graphics and Signal Processing (ICGSP). New York: Association for Computing Machinery, 2018: 21-25.
28. Iqbal B, Iqbal W, Khan N, et al. Canny edge detection and Hough transform for high resolution video streams using Hadoop and Spark. Cluster Comput, 2020, 23(1): 397-408.
29. Yamada A, Iizuka T, Kikuchi D, et al. Sa1639 endoscopic features of early gastric cancers with the newly developed narrow band imaging endoscopy, the Evis Lucera Elite (Olympus Co. ). Gastrointest Endosc, 2014, 79(5): AB285.
30. Zhao Z, Tse Z T H. An electromagnetic tracking needle clip: an enabling design for low-cost image-guided therapy. Minim Invasiv Ther, 2019, 28(3): 165-171.

Journal of Biomedical Engineering

Confocal probe localization algorithm based on region growing and endoscope size prior

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content