Reconstruction of elasticity modulus distribution base on semi-supervised neural network_Journal of Biomedical Engineering

Authors：

ZHANG Xiao ¹ ,  PENG Bo ¹ , WANG Rui ² , WEI Xingyue ² ,  LUO Jianwen ²

1. School of Computer Science and Software Engineering, Southwest Petroleum University, Chengdu 610500, P. R. China;
2. Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing 100084, P. R. China;

Corresponding author：

PENG Bo, Email: bopeng@swpu.edu.cn; LUO Jianwen, Email: luo_jianwen@tsinghua.edu.cn

Keywords：

Semi-supervise; Inverse problem; Deep learning; Reconstruction of elasticity modulus distribution

DOI：

10.7507/1001-5515.202306008

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Accurate reconstruction of tissue elasticity modulus distribution has always been an important challenge in ultrasound elastography. Considering that existing deep learning-based supervised reconstruction methods only use simulated displacement data with random noise in training, which cannot fully provide the complexity and diversity brought by in-vivo ultrasound data, this study introduces the use of displacement data obtained by tracking in-vivo ultrasound radio frequency signals (i.e., real displacement data) during training, employing a semi-supervised approach to enhance the prediction accuracy of the model. Experimental results indicate that in phantom experiments, the semi-supervised model augmented with real displacement data provides more accurate predictions, with mean absolute errors and mean relative errors both around 3%, while the corresponding data for the fully supervised model are around 5%. When processing real displacement data, the area of prediction error of semi-supervised model was less than that of fully supervised model. The findings of this study confirm the effectiveness and practicality of the proposed approach, providing new insights for the application of deep learning methods in the reconstruction of elastic distribution from in-vivo ultrasound data.

Citation： ZHANG Xiao, PENG Bo, WANG Rui, WEI Xingyue, LUO Jianwen. Reconstruction of elasticity modulus distribution base on semi-supervised neural network. Journal of Biomedical Engineering, 2024, 41(2): 262-271. doi: 10.7507/1001-5515.202306008 Copy

1.	Gennisson J L, Deffieux T, Fink M, et al. Ultrasound elastography: principles and techniques. Diagnostic and Interventional Imaging, 2013, 94(5): 487-495.
2.	Parker K J, Doyley M M, Rubens D J. Imaging the elastic properties of tissue: the 20 year perspective. Physics in Medicine & Biology, 2010, 56(1): R1-R29.
3.	Doyley M M, Bamber J C, Shiina T, et al. Reconstruction of elastic modulus distribution from envelope detected B-mode data//1996 IEEE Ultrasonics Symposium. Proceedings, IEEE, 1996, 2: 1611-1614.
4.	Hansen H H, Richards M S, Doyley M M, et al. Noninvasive vascular displacement estimation for relative elastic modulus reconstruction in transversal imaging planes. Sensors, 2013, 13(3): 3341-3357.
5.	Wang Z, Liu Y, Sun L Z, et al. Elasto-mammography: elastic property reconstruction in breast tissues. MRS Online Proceedings Library (OPL), 2005, 874: 69.
6.	Zhi H, Ou B, Luo B M, et al. Comparison of ultrasound elastography, mammography, and sonography in the diagnosis of solid breast lesions. J Ultrasound Med, 2007, 26(6): 807-815.
7.	Sigrist R M S, Liau J, Kaffas A E, et al. Ultrasound elastography: review of techniques and clinical applications. Theranostics, 2017, 7(5): 1303-1329.
8.	Oberai A A, Gokhale N H, Feijóo G R. Solution of inverse problems in elasticity imaging using the adjoint method. Inverse Problems, 2003, 19(2): 297.
9.	Goenezen S, Barbone P, Oberai A A. Solution of the nonlinear elasticity imaging inverse problem: the incompressible case. Computer Methods in Applied Mechanics and Engineering, 2011, 200(13-16): 1406-1420.
10.	Oberai A A, Gokhale N H, Doyley M M, et al. Evaluation of the adjoint equation based algorithm for elasticity imaging. Physics in Medicine & Biology, 2004, 49(13): 2955-2974.
11.	Richards M S, Barbone P E, Oberai A A. Quantitative three-dimensional elasticity imaging from quasi-static deformation: a phantom study. Physics in Medicine & Biology, 2009, 54(3): 757-779.
12.	Barbone P E, Oberai A A. Elastic modulus imaging: some exact solutions of the compressible elastography inverse problem. Physics in Medicine & Biology, 2007, 52(6): 1577-1593.
13.	Barbone P E, Rivas C E, Harari I, et al. Adjoint-weighted variational formulation for the direct solution of inverse problems of general linear elasticity with full interior data. International Journal for Numerical Methods in Engineering, 2010, 81(13): 1713-1736.
14.	Babaniyi O A, Oberai A A, Barbone P E. Direct error in constitutive equation formulation for plane stress inverse elasticity problem. Computer Methods in Applied Mechanics and Engineering, 2017, 314: 3-18.
15.	Doyley M M. Model-based elastography: a survey of approaches to the inverse elasticity problem. Physics in Medicine & Biology, 2012, 57(3): R35.
16.	Chen C T, Gu G X. Learning hidden elasticity with deep neural networks. Proc Natl Acad Sci U S A, 2021, 118(31): e2102721118.
17.	Ni B, Gao H. A deep learning approach to the inverse problem of modulus identification in elasticity. MRS Bulletin, 2021, 46: 19-25.
18.	Zhang X, Wang R, Wei X, et al. Displacement-based reconstruction of elasticity distribution with deep neural network//2022 IEEE International Ultrasonics Symposium (IUS). IEEE, 2022: 1-5.
19.	Wen S, Peng B, Wei X, et al. Convolutional neural network-based speckle tracking for ultrasound strain elastography: an unsupervised learning approach. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 2023, 70(5): 354-367.
20.	Wei X, Wang Y, Ge L, et al. Unsupervised convolutional neural network for motion estimation in ultrasound elastography. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 2022, 69(7): 2236-2247.
21.	Mao X, Li Q, Xie H, et al. Least squares generative adversarial networks//Proceedings of the IEEE International Conference on Computer Vision, IEEE, 2017: 2794-2802.
22.	Kieu M, Berlincioni L, Galteri L, et al. Robust pedestrian detection in thermal imagery using synthesized images//2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021: 8804-8811.
23.	Lei L, Mardani M. Semi-supervised super-resolution GANs for MRI reconstruction//31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach. 2017.
24.	Hammernik K, Schlemper J, Qin C, et al. Systematic evaluation of iterative deep neural networks for fast parallel MRI reconstruction with sensitivity-weighted coil combination. Magn Reson Med, 2021, 86(4): 1859-1872.
25.	Dai J, Qi H, Xiong Y, et al. Deformable convolutional networks//Proceedings of the IEEE International Conference on Computer Vision. 2017: 764-773.
26.	Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 1125-1134.
27.	Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint, 2015, arXiv: 1511.06434.
28.	Hall T J, Zhu Y, Spalding C S. in vivo real-time freehand palpation imaging. Ultrasound Med Biol, 2003, 29(3): 427-435.
29.	Wen S, Peng B, Jiang H, et al. Augmenting 3D ultrasound strain elastography by combining Bayesian inference with local polynomial fitting in region-growing-based motion tracking//2021 IEEE International Conference on Image Processing (ICIP). IEEE, 2021: 2963-2967.
30.	He L, Peng B, Yang T, et al. An application of super-resolution generative adversary networks for quasi-static ultrasound strain elastography: a feasibility study. IEEE Access, 2020, 8: 65769-65779.
31.	Athanasiou A, Tardivon A, Tanter M, et al. Breast lesions: quantitative elastography with supersonic shear imaging--preliminary results. Radiology, 2010, 256(1): 297-303.

1. Gennisson J L, Deffieux T, Fink M, et al. Ultrasound elastography: principles and techniques. Diagnostic and Interventional Imaging, 2013, 94(5): 487-495.
2. Parker K J, Doyley M M, Rubens D J. Imaging the elastic properties of tissue: the 20 year perspective. Physics in Medicine & Biology, 2010, 56(1): R1-R29.
3. Doyley M M, Bamber J C, Shiina T, et al. Reconstruction of elastic modulus distribution from envelope detected B-mode data//1996 IEEE Ultrasonics Symposium. Proceedings, IEEE, 1996, 2: 1611-1614.
4. Hansen H H, Richards M S, Doyley M M, et al. Noninvasive vascular displacement estimation for relative elastic modulus reconstruction in transversal imaging planes. Sensors, 2013, 13(3): 3341-3357.
5. Wang Z, Liu Y, Sun L Z, et al. Elasto-mammography: elastic property reconstruction in breast tissues. MRS Online Proceedings Library (OPL), 2005, 874: 69.
6. Zhi H, Ou B, Luo B M, et al. Comparison of ultrasound elastography, mammography, and sonography in the diagnosis of solid breast lesions. J Ultrasound Med, 2007, 26(6): 807-815.
7. Sigrist R M S, Liau J, Kaffas A E, et al. Ultrasound elastography: review of techniques and clinical applications. Theranostics, 2017, 7(5): 1303-1329.
8. Oberai A A, Gokhale N H, Feijóo G R. Solution of inverse problems in elasticity imaging using the adjoint method. Inverse Problems, 2003, 19(2): 297.
9. Goenezen S, Barbone P, Oberai A A. Solution of the nonlinear elasticity imaging inverse problem: the incompressible case. Computer Methods in Applied Mechanics and Engineering, 2011, 200(13-16): 1406-1420.
10. Oberai A A, Gokhale N H, Doyley M M, et al. Evaluation of the adjoint equation based algorithm for elasticity imaging. Physics in Medicine & Biology, 2004, 49(13): 2955-2974.
11. Richards M S, Barbone P E, Oberai A A. Quantitative three-dimensional elasticity imaging from quasi-static deformation: a phantom study. Physics in Medicine & Biology, 2009, 54(3): 757-779.
12. Barbone P E, Oberai A A. Elastic modulus imaging: some exact solutions of the compressible elastography inverse problem. Physics in Medicine & Biology, 2007, 52(6): 1577-1593.
13. Barbone P E, Rivas C E, Harari I, et al. Adjoint-weighted variational formulation for the direct solution of inverse problems of general linear elasticity with full interior data. International Journal for Numerical Methods in Engineering, 2010, 81(13): 1713-1736.
14. Babaniyi O A, Oberai A A, Barbone P E. Direct error in constitutive equation formulation for plane stress inverse elasticity problem. Computer Methods in Applied Mechanics and Engineering, 2017, 314: 3-18.
15. Doyley M M. Model-based elastography: a survey of approaches to the inverse elasticity problem. Physics in Medicine & Biology, 2012, 57(3): R35.
16. Chen C T, Gu G X. Learning hidden elasticity with deep neural networks. Proc Natl Acad Sci U S A, 2021, 118(31): e2102721118.
17. Ni B, Gao H. A deep learning approach to the inverse problem of modulus identification in elasticity. MRS Bulletin, 2021, 46: 19-25.
18. Zhang X, Wang R, Wei X, et al. Displacement-based reconstruction of elasticity distribution with deep neural network//2022 IEEE International Ultrasonics Symposium (IUS). IEEE, 2022: 1-5.
19. Wen S, Peng B, Wei X, et al. Convolutional neural network-based speckle tracking for ultrasound strain elastography: an unsupervised learning approach. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 2023, 70(5): 354-367.
20. Wei X, Wang Y, Ge L, et al. Unsupervised convolutional neural network for motion estimation in ultrasound elastography. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 2022, 69(7): 2236-2247.
21. Mao X, Li Q, Xie H, et al. Least squares generative adversarial networks//Proceedings of the IEEE International Conference on Computer Vision, IEEE, 2017: 2794-2802.
22. Kieu M, Berlincioni L, Galteri L, et al. Robust pedestrian detection in thermal imagery using synthesized images//2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021: 8804-8811.
23. Lei L, Mardani M. Semi-supervised super-resolution GANs for MRI reconstruction//31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach. 2017.
24. Hammernik K, Schlemper J, Qin C, et al. Systematic evaluation of iterative deep neural networks for fast parallel MRI reconstruction with sensitivity-weighted coil combination. Magn Reson Med, 2021, 86(4): 1859-1872.
25. Dai J, Qi H, Xiong Y, et al. Deformable convolutional networks//Proceedings of the IEEE International Conference on Computer Vision. 2017: 764-773.
26. Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 1125-1134.
27. Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint, 2015, arXiv: 1511.06434.
28. Hall T J, Zhu Y, Spalding C S. in vivo real-time freehand palpation imaging. Ultrasound Med Biol, 2003, 29(3): 427-435.
29. Wen S, Peng B, Jiang H, et al. Augmenting 3D ultrasound strain elastography by combining Bayesian inference with local polynomial fitting in region-growing-based motion tracking//2021 IEEE International Conference on Image Processing (ICIP). IEEE, 2021: 2963-2967.
30. He L, Peng B, Yang T, et al. An application of super-resolution generative adversary networks for quasi-static ultrasound strain elastography: a feasibility study. IEEE Access, 2020, 8: 65769-65779.
31. Athanasiou A, Tardivon A, Tanter M, et al. Breast lesions: quantitative elastography with supersonic shear imaging--preliminary results. Radiology, 2010, 256(1): 297-303.

Journal of Biomedical Engineering

Reconstruction of elasticity modulus distribution base on semi-supervised neural network

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content