Multiresolution discrete optimization registration method of ultrasound and magnetic resonance images based on key points_Journal of Biomedical Engineering

Authors：

TAN Zhenlin ¹ ,  GUO Shengwen ²

1. School of Material Science and Engineering, South China University of Technology, Guangzhou 510640, P. R. China;
2. School of Automation Science and Engineering, South China University of Technology, Guangzhou 510640, P. R. China;

Corresponding author：

GUO Shengwen, Email: shwguo@scut.edu.cn

Keywords：

Ultrasound image; Magnetic resonance image; Registration; Key points; Discrete optimization; Multi-resolution decomposition

DOI：

10.7507/1001-5515.202211022

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

The registration of preoperative magnetic resonance (MR) images and intraoperative ultrasound (US) images is very important in the planning of brain tumor surgery and during surgery. Considering that the two-modality images have different intensity range and resolution, and the US images are degraded by lots of speckle noises, a self-similarity context (SSC) descriptor based on local neighborhood information was adopted to define the similarity measure. The ultrasound images were considered as the reference, the corners were extracted as the key points using three-dimensional differential operators, and the dense displacement sampling discrete optimization algorithm was adopted for registration. The whole registration process was divided into two stages including the affine registration and the elastic registration. In the affine registration stage, the image was decomposed using multi-resolution scheme, and in the elastic registration stage, the displacement vectors of key points were regularized using the minimum convolution and mean field reasoning strategies. The registration experiment was performed on the preoperative MR images and intraoperative US images of 22 patients. The overall error after affine registration was (1.57 ± 0.30) mm, and the average computation time of each pair of images was only 1.36 s; while the overall error after elastic registration was further reduced to (1.40 ± 0.28) mm, and the average registration time was 1.53 s. The experimental results show that the proposed method has prominent registration accuracy and high computational efficiency.

Citation： TAN Zhenlin, GUO Shengwen. Multiresolution discrete optimization registration method of ultrasound and magnetic resonance images based on key points. Journal of Biomedical Engineering, 2023, 40(2): 202-207, 216. doi: 10.7507/1001-5515.202211022 Copy

1.	Gerard I J, Kersten-Oertel M, Hall J A, et al. Brain shift in neuronavigation of brain tumors: an updated review of intra-operative ultrasound applications. Front Oncol, 2021, 10: 618837.
2.	Bastos D C D A, Juvekar P, Tie Y, et al. Challenges and opportunities of intraoperative 3D ultrasound with neuronavigation in relation to intraoperative MRI. Front Oncol, 2021, 11: 656519.
3.	Reinertsen I, Lindseth F, Unsgaard G, et al. Clinical validation of vessel-based registration for correction of brain-shift. Med Image Anal, 2007, 11(6): 673-684.
4.	Coupé P, Hellier P, Morandi X, et al. 3D rigid registration of intraoperative ultrasound and preoperative MR brain images based on hyperechogenic structures. Int J Biomed Imaging, 2012, 2012(3): 531319.
5.	Farnia P, Makkiabadi B, Ahmadian A, et al. Curvelet based residual complexity objective function for non-rigid registration of pre-operative MRI with intra-operative ultrasound images// 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Orlando: IEEE, 2016: 1167-1170.
6.	Wein W, Ladikos A, Fuerst B, et al. Global registration of ultrasound to MRI using the LC² metric for enabling neurosurgical guidance// Mori K, Sakuma I, Sato Y, et al. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2013. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013: 34-41.
7.	Heinrich M P, Jenkinson M, Papież B W, et al. Towards realtime multimodal fusion for image-guided interventions using self-similarities// Mori K, Sakuma I, Sato Y, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013: 187-194.
8.	Rivaz H, Chen S J S, Collins D L. Automatic deformable MR-ultrasound registration for image-guided neurosurgery. IEEE Trans Med Imaging, 2015, 34(2): 366-380.
9.	Machado I, Toews M, George E, et al. Deformable MRI-Ultrasound registration using correlation-based attribute matching for brain shift correction: Accuracy and generality in multi-site data. NeuroImage, 2019, 202: 116094.
10.	Ou Y, Sotiras A, Paragios N, et al. DRAMMS: Deformable registration via attribute matching and mutual-saliency weighting. Med Image Anal, 2011, 15(4): 622-639.
11.	Arun K S, Huang T S, Blostein S D. Least-squares fitting of two 3-D point sets. IEEE Trans Pattern Anal Mach Intell, 1987, PAMI-9(5): 698-700.
12.	Bookstein F L. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Trans Pattern Anal Mach Intell, 1989, 11(6): 567-585.
13.	Rohr K. On 3D differential operators for detecting point landmarks. Image Vis Comput, 1997, 15(3): 219-233.
14.	Heinrich M P. Closing the gap between deep and conventional image registration using probabilistic dense displacement networks// Shen Dinggang, Liu Tianming, Peters T M, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2019: 50-58.
15.	Xiao Y, Fortin M, Unsgård G, et al. REtroSpective Evaluation of Cerebral Tumors (RESECT): A clinical database of pre-operative MRI and intra-operative ultrasound in low-grade glioma surgeries. Med Phys, 2017, 44(7): 3875-3882.
16.	Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern, 1979, 9(1): 62-66.
17.	Xiao Y, Rivaz H, Chabanas M, et al. Evaluation of MRI to ultrasound registration methods for brain shift correction: The CuRIOUS2018 challenge. IEEE Trans Med Imaging, 2020, 39(3): 777-786.
18.	Wein W. Brain-shift correction with image-based registration and landmark accuracy evaluation// Stoyanov D, Taylor Z, Aylward S, et al. Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Cham: Springer International Publishing, 2018: 146-151.
19.	Shams R, Boucher M-A, Kadoury S. Intra-operative brain shift correction with weighted locally linear correlations of 3DUS and MRI// Stoyanov D, Taylor Z, Aylward S, et al. Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Cham: Springer International Publishing, 2018: 179-184.
20.	Machado I, Toews M, Luo J, et al. Deformable MRI-ultrasound registration via attribute matching and mutual-saliency weighting for image-guided neurosurgery// Stoyanov D, Taylor Z, Aylward S, et al. Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Cham: Springer International Publishing, 2018: 165-171.

1. Gerard I J, Kersten-Oertel M, Hall J A, et al. Brain shift in neuronavigation of brain tumors: an updated review of intra-operative ultrasound applications. Front Oncol, 2021, 10: 618837.
2. Bastos D C D A, Juvekar P, Tie Y, et al. Challenges and opportunities of intraoperative 3D ultrasound with neuronavigation in relation to intraoperative MRI. Front Oncol, 2021, 11: 656519.
3. Reinertsen I, Lindseth F, Unsgaard G, et al. Clinical validation of vessel-based registration for correction of brain-shift. Med Image Anal, 2007, 11(6): 673-684.
4. Coupé P, Hellier P, Morandi X, et al. 3D rigid registration of intraoperative ultrasound and preoperative MR brain images based on hyperechogenic structures. Int J Biomed Imaging, 2012, 2012(3): 531319.
5. Farnia P, Makkiabadi B, Ahmadian A, et al. Curvelet based residual complexity objective function for non-rigid registration of pre-operative MRI with intra-operative ultrasound images// 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Orlando: IEEE, 2016: 1167-1170.
6. Wein W, Ladikos A, Fuerst B, et al. Global registration of ultrasound to MRI using the LC² metric for enabling neurosurgical guidance// Mori K, Sakuma I, Sato Y, et al. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2013. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013: 34-41.
7. Heinrich M P, Jenkinson M, Papież B W, et al. Towards realtime multimodal fusion for image-guided interventions using self-similarities// Mori K, Sakuma I, Sato Y, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013: 187-194.
8. Rivaz H, Chen S J S, Collins D L. Automatic deformable MR-ultrasound registration for image-guided neurosurgery. IEEE Trans Med Imaging, 2015, 34(2): 366-380.
9. Machado I, Toews M, George E, et al. Deformable MRI-Ultrasound registration using correlation-based attribute matching for brain shift correction: Accuracy and generality in multi-site data. NeuroImage, 2019, 202: 116094.
10. Ou Y, Sotiras A, Paragios N, et al. DRAMMS: Deformable registration via attribute matching and mutual-saliency weighting. Med Image Anal, 2011, 15(4): 622-639.
11. Arun K S, Huang T S, Blostein S D. Least-squares fitting of two 3-D point sets. IEEE Trans Pattern Anal Mach Intell, 1987, PAMI-9(5): 698-700.
12. Bookstein F L. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Trans Pattern Anal Mach Intell, 1989, 11(6): 567-585.
13. Rohr K. On 3D differential operators for detecting point landmarks. Image Vis Comput, 1997, 15(3): 219-233.
14. Heinrich M P. Closing the gap between deep and conventional image registration using probabilistic dense displacement networks// Shen Dinggang, Liu Tianming, Peters T M, et al. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2019: 50-58.
15. Xiao Y, Fortin M, Unsgård G, et al. REtroSpective Evaluation of Cerebral Tumors (RESECT): A clinical database of pre-operative MRI and intra-operative ultrasound in low-grade glioma surgeries. Med Phys, 2017, 44(7): 3875-3882.
16. Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern, 1979, 9(1): 62-66.
17. Xiao Y, Rivaz H, Chabanas M, et al. Evaluation of MRI to ultrasound registration methods for brain shift correction: The CuRIOUS2018 challenge. IEEE Trans Med Imaging, 2020, 39(3): 777-786.
18. Wein W. Brain-shift correction with image-based registration and landmark accuracy evaluation// Stoyanov D, Taylor Z, Aylward S, et al. Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Cham: Springer International Publishing, 2018: 146-151.
19. Shams R, Boucher M-A, Kadoury S. Intra-operative brain shift correction with weighted locally linear correlations of 3DUS and MRI// Stoyanov D, Taylor Z, Aylward S, et al. Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Cham: Springer International Publishing, 2018: 179-184.
20. Machado I, Toews M, Luo J, et al. Deformable MRI-ultrasound registration via attribute matching and mutual-saliency weighting for image-guided neurosurgery// Stoyanov D, Taylor Z, Aylward S, et al. Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Cham: Springer International Publishing, 2018: 165-171.

Journal of Biomedical Engineering

Multiresolution discrete optimization registration method of ultrasound and magnetic resonance images based on key points

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content