Application of generative adversarial network in magnetic resonance image reconstruction_Journal of Biomedical Engineering

Authors：

CAI Xin ¹ , HOU Xuewen ¹ , YANG Guang ² ,  NIE Shengdong ¹

1. School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, P. R. China;
2. Shanghai Key Laboratory of Magnetic Resonance, East China Normal University, Shanghai 200062, P. R. China;

Corresponding author：

Keywords：

Magnetic resonance imaging; Image reconstruction; Generative adversarial network; Deep learning

DOI：

10.7507/1001-5515.202204007

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Magnetic resonance imaging (MRI) is an important medical imaging method, whose major limitation is its long scan time due to the imaging mechanism, increasing patients’ cost and waiting time for the examination. Currently, parallel imaging (PI) and compress sensing (CS) together with other reconstruction technologies have been proposed to accelerate image acquisition. However, the image quality of PI and CS depends on the image reconstruction algorithms, which is far from satisfying in respect to both the image quality and the reconstruction speed. In recent years, image reconstruction based on generative adversarial network (GAN) has become a research hotspot in the field of magnetic resonance imaging because of its excellent performance. In this review, we summarized the recent development of application of GAN in MRI reconstruction in both single- and multi-modality acceleration, hoping to provide a useful reference for interested researchers. In addition, we analyzed the characteristics and limitations of existing technologies and forecasted some development trends in this field.

Citation： CAI Xin, HOU Xuewen, YANG Guang, NIE Shengdong. Application of generative adversarial network in magnetic resonance image reconstruction. Journal of Biomedical Engineering, 2023, 40(3): 582-588. doi: 10.7507/1001-5515.202204007 Copy

1.	Islam R, Islam M S, Uddin M S. Compressed sensing in parallel MRI: a review. International Journal of Image and Graphics, 2021, 22(4): 2250038.
2.	Delattre B M A, Boudabbous S, Hansen C, et al. Compressed sensing MRI of different organs: ready for clinical daily practice?. Eur Radiol, 2020, 30(1): 308-319.
3.	Goodfellow I, Pouget-abadie J, Mirza M, et al. Generative adversarial nets//The 27th International Conference on Neural Information Processing Systems, 2014, 2: 2672-2680.
4.	Yi X, Walia E, Babyn P. Generative adversarial network in medical imaging: a review. Medical Image Analysis, 2019, 58: 101552.
5.	Kazeminia S, Baur C, Kuijper A, et al. GANs for medical image analysis. Artificial Intelligence in Medicine, 2020, 109: 101938.
6.	Deora P, Vasudeva B, Bhattacharya S, et al. Structure preserving compressive sensing MRI reconstruction using generative adversarial networks//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle: ICCVW, 2020: 2211-2219.
7.	Zhang A, Su L, Zhang Y, et al. EEG data augmentation for emotion recognition with a multiple generator conditional Wasserstein GAN. Complex & Intelligent Systems, 2022, 8: 3059-3071.
8.	Liu Xiaofeng, Xing Fangxu, Jerry L, Dual-cycle constrained bijective VAE-GAN for tagged-to-cine magnetic resonance image synthesis//2021 IEEE 18th International Symposium on Biomedical Imaging, New York: IEEE, 2021: 1448-1452.
9.	Lv J, Zhu J, Yang G. Which GAN? A comparative study of generative adversarial network-based fast MRI reconstruction. Philos Trans A Math Phys Eng Sci, 2021, 379(2200): 20200203.
10.	Yang G, Yu S, Dong H, et al. DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans Med Imaging, 2018, 37(6): 1310-1321.
11.	Yuan Z, Jiang M, Wang Y, et al. SARA-GAN: self-attention and relative average discriminator based generative adversarial networks for fast compressed sensing MRI reconstruction. Frontiers in Neuro informatics, 2020, 14: 611666.
12.	Shaul R, David I, Shitrit O, et al. Subsampled brain MRI reconstruction by generative adversarial neural networks. Medical Image Analysis, 2020, 65(4): 101747.
13.	Li Y, Li J, Ma F, et al. High quality and fast compressed sensing MRI reconstruction via edge-enhanced dual discriminator generative adversarial network. Magnetic Resonance Imaging, 2021, 77: 124-136.
14.	Huang J, Ding W, Lv J, et al. Edge-enhanced dual discriminator generative adversarial network for fast MRI with parallel imaging using multi-view information. Applied Intelligence, 2022, 52(13): 14693-14710.
15.	Li G, Lyu J, Wang C, et al. WavTrans: synergizing wavelet and cross-attention transformer for multi-contrast MRI super-resolution//Medical Image Computing and Computer Assisted Intervention (MICCAI 2022), Xiamen: Springer, 2022: 463–473.
16.	Huang J, Wu Y, Wu H, et al. Fast MRI reconstruction: how powerful transformers are?. Annu Int Conf IEEE Eng Med Biol Soc, 2022, 2022: 2066-2070.
17.	Cole E K, Ong F, Vasanawala S S, et al. Fast unsupervised MRI reconstruction without fully-sampled ground truth data using generative adversarial networks//2021 IEEE/CVF International Conference on Computer Vision Workshops, Montreal: ICCVW, 2021: 3971-3980.
18.	Oh G, Sim B, Chung H, et al. Unpaired deep learning for accelerated MRI using optimal transport driven cycleGAN. IEEE Transactions on Computational Imaging, 2020, 6: 1285-1296.
19.	Li G, Lv J, Wang C. A modified generative adversarial network using spatial and channel-wise attention for CS-MRI reconstruction. IEEE Access, 2021, 9: 83185-83198.
20.	Belov A, Stadelmann J, Kastryulin S, et al. Towards ultrafast MRI via extreme k-space under sampling and super resolution//Medical Image Computing and Computer Assisted Intervention, Strasbourg: Springer, 2021: 254-264.
21.	Lv J, Wang P, Tong X, et al. Parallel imaging with a combination of sensitivity encoding and generative adversarial networks. Quant Imaging Med Surg, 2020, 10(12): 2260-2273.
22.	Lv J, Wang C, Yang G. PIC-GAN: a parallel imaging coupled generative adversarial network for accelerated multi-channel MRI reconstruction. Diagnostics, 2021, 11(1): 61.
23.	Lv J, Li G, Tong X, et al. Transfer learning enhanced generative adversarial networks for multi-channel MRI reconstruction. Computers in Biology and Medicine, 2021, 134: 104504.
24.	Edupuganti V, Mardani M, Vasanawala S, et al. Uncertainty quantification in deep MRI reconstruction. IEEE Transactions on Medical Imaging, 2021, 40(1): 239-250.
25.	Sun L, Fan Z, Fu X, et al. A deep information sharing network for multi-contrast compressed sensing MRI reconstruction. IEEE Trans Image Process, 2019, 28(12): 6141-6153.
26.	Liu X, Wang J, Lin S, ed al. Optimizing multi contrast MRI reconstruction with shareable feature aggregation and selection. NMR Biomed, 2021, 34(8): e4540.
27.	Liu X, Wang J, Sun H, et al. On the regularization of feature fusion and mapping for fast MR multi-contrast imaging via iterative networks. Magn Reson Imaging, 2021, 77: 159-168.
28.	Kim K H, Do W J, Park S H. Improving resolution of MR images with an adversarial network incorporating images with different contrast. Medical Physics, 2018, 45(7): 3120-3131.
29.	Xiang L, Chen Y, Chang W T, et al. Deep-learning-based multi-modal fusion for fast MR reconstruction. IEEE Transactions on Biomedical Engineering, 2019, 66(7): 2105-2114.
30.	Do W, Seo S, Han Y, et al. Reconstruction of multi-contrast MR images through deep learning. Medical Physics, 2020, 47(3): 983-997.
31.	Dar S U, Yurt M, Shahdloo M, et al. Prior-guided image reconstruction for accelerated multi-contrast MRI via generative adversarial networks. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(6): 1072-1087.
32.	楼鑫杰. 基于深度学习与多模态融合的快速MRI重建研究. 杭州: 浙江工业大学, 2021.

1. Islam R, Islam M S, Uddin M S. Compressed sensing in parallel MRI: a review. International Journal of Image and Graphics, 2021, 22(4): 2250038.
2. Delattre B M A, Boudabbous S, Hansen C, et al. Compressed sensing MRI of different organs: ready for clinical daily practice?. Eur Radiol, 2020, 30(1): 308-319.
3. Goodfellow I, Pouget-abadie J, Mirza M, et al. Generative adversarial nets//The 27th International Conference on Neural Information Processing Systems, 2014, 2: 2672-2680.
4. Yi X, Walia E, Babyn P. Generative adversarial network in medical imaging: a review. Medical Image Analysis, 2019, 58: 101552.
5. Kazeminia S, Baur C, Kuijper A, et al. GANs for medical image analysis. Artificial Intelligence in Medicine, 2020, 109: 101938.
6. Deora P, Vasudeva B, Bhattacharya S, et al. Structure preserving compressive sensing MRI reconstruction using generative adversarial networks//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle: ICCVW, 2020: 2211-2219.
7. Zhang A, Su L, Zhang Y, et al. EEG data augmentation for emotion recognition with a multiple generator conditional Wasserstein GAN. Complex & Intelligent Systems, 2022, 8: 3059-3071.
8. Liu Xiaofeng, Xing Fangxu, Jerry L, Dual-cycle constrained bijective VAE-GAN for tagged-to-cine magnetic resonance image synthesis//2021 IEEE 18th International Symposium on Biomedical Imaging, New York: IEEE, 2021: 1448-1452.
9. Lv J, Zhu J, Yang G. Which GAN? A comparative study of generative adversarial network-based fast MRI reconstruction. Philos Trans A Math Phys Eng Sci, 2021, 379(2200): 20200203.
10. Yang G, Yu S, Dong H, et al. DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans Med Imaging, 2018, 37(6): 1310-1321.
11. Yuan Z, Jiang M, Wang Y, et al. SARA-GAN: self-attention and relative average discriminator based generative adversarial networks for fast compressed sensing MRI reconstruction. Frontiers in Neuro informatics, 2020, 14: 611666.
12. Shaul R, David I, Shitrit O, et al. Subsampled brain MRI reconstruction by generative adversarial neural networks. Medical Image Analysis, 2020, 65(4): 101747.
13. Li Y, Li J, Ma F, et al. High quality and fast compressed sensing MRI reconstruction via edge-enhanced dual discriminator generative adversarial network. Magnetic Resonance Imaging, 2021, 77: 124-136.
14. Huang J, Ding W, Lv J, et al. Edge-enhanced dual discriminator generative adversarial network for fast MRI with parallel imaging using multi-view information. Applied Intelligence, 2022, 52(13): 14693-14710.
15. Li G, Lyu J, Wang C, et al. WavTrans: synergizing wavelet and cross-attention transformer for multi-contrast MRI super-resolution//Medical Image Computing and Computer Assisted Intervention (MICCAI 2022), Xiamen: Springer, 2022: 463–473.
16. Huang J, Wu Y, Wu H, et al. Fast MRI reconstruction: how powerful transformers are?. Annu Int Conf IEEE Eng Med Biol Soc, 2022, 2022: 2066-2070.
17. Cole E K, Ong F, Vasanawala S S, et al. Fast unsupervised MRI reconstruction without fully-sampled ground truth data using generative adversarial networks//2021 IEEE/CVF International Conference on Computer Vision Workshops, Montreal: ICCVW, 2021: 3971-3980.
18. Oh G, Sim B, Chung H, et al. Unpaired deep learning for accelerated MRI using optimal transport driven cycleGAN. IEEE Transactions on Computational Imaging, 2020, 6: 1285-1296.
19. Li G, Lv J, Wang C. A modified generative adversarial network using spatial and channel-wise attention for CS-MRI reconstruction. IEEE Access, 2021, 9: 83185-83198.
20. Belov A, Stadelmann J, Kastryulin S, et al. Towards ultrafast MRI via extreme k-space under sampling and super resolution//Medical Image Computing and Computer Assisted Intervention, Strasbourg: Springer, 2021: 254-264.
21. Lv J, Wang P, Tong X, et al. Parallel imaging with a combination of sensitivity encoding and generative adversarial networks. Quant Imaging Med Surg, 2020, 10(12): 2260-2273.
22. Lv J, Wang C, Yang G. PIC-GAN: a parallel imaging coupled generative adversarial network for accelerated multi-channel MRI reconstruction. Diagnostics, 2021, 11(1): 61.
23. Lv J, Li G, Tong X, et al. Transfer learning enhanced generative adversarial networks for multi-channel MRI reconstruction. Computers in Biology and Medicine, 2021, 134: 104504.
24. Edupuganti V, Mardani M, Vasanawala S, et al. Uncertainty quantification in deep MRI reconstruction. IEEE Transactions on Medical Imaging, 2021, 40(1): 239-250.
25. Sun L, Fan Z, Fu X, et al. A deep information sharing network for multi-contrast compressed sensing MRI reconstruction. IEEE Trans Image Process, 2019, 28(12): 6141-6153.
26. Liu X, Wang J, Lin S, ed al. Optimizing multi contrast MRI reconstruction with shareable feature aggregation and selection. NMR Biomed, 2021, 34(8): e4540.
27. Liu X, Wang J, Sun H, et al. On the regularization of feature fusion and mapping for fast MR multi-contrast imaging via iterative networks. Magn Reson Imaging, 2021, 77: 159-168.
28. Kim K H, Do W J, Park S H. Improving resolution of MR images with an adversarial network incorporating images with different contrast. Medical Physics, 2018, 45(7): 3120-3131.
29. Xiang L, Chen Y, Chang W T, et al. Deep-learning-based multi-modal fusion for fast MR reconstruction. IEEE Transactions on Biomedical Engineering, 2019, 66(7): 2105-2114.
30. Do W, Seo S, Han Y, et al. Reconstruction of multi-contrast MR images through deep learning. Medical Physics, 2020, 47(3): 983-997.
31. Dar S U, Yurt M, Shahdloo M, et al. Prior-guided image reconstruction for accelerated multi-contrast MRI via generative adversarial networks. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(6): 1072-1087.
32. 楼鑫杰. 基于深度学习与多模态融合的快速MRI重建研究. 杭州: 浙江工业大学, 2021.

Journal of Biomedical Engineering

Application of generative adversarial network in magnetic resonance image reconstruction

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content