Applications of generative adversarial networks in medical image processing_Journal of Biomedical Engineering

Authors：

PAN Dan ^1,2 , JIA Longfei ³ ,  ZENG An ^3,4 , SONG Xiaowei ⁵

1. Modern Education Technology Center, Guangdong Construction Polytechnic, Guangzhou 510440, P.R.China;
2. Guangzhou Benzhen Networks Technology Co. Ltd., Guangzhou 510000, P.R.China;
3. Faculty of Computer, Guangdong University of Technology, Guangzhou 510006, P.R.China;
4. Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou 510006, P.R.China;
5. ImageTech Lab, Simon Fraser University, Vancouver V6B 5K3, Canada;

Corresponding author：

Keywords：

generative adversarial networks; medical images; deep learning

DOI：

10.7507/1001-5515.201803025

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

In recent years, researchers have introduced various methods in many domains into medical image processing so that its effectiveness and efficiency can be improved to some extent. The applications of generative adversarial networks (GAN) in medical image processing are evolving very fast. In this paper, the state of the art in this area has been reviewed. Firstly, the basic concepts of the GAN were introduced. And then, from the perspectives of the medical image denoising, detection, segmentation, synthesis, reconstruction and classification, the applications of the GAN were summarized. Finally, prospects for further research in this area were presented.

Citation： PAN Dan, JIA Longfei, ZENG An, SONG Xiaowei. Applications of generative adversarial networks in medical image processing. Journal of Biomedical Engineering, 2018, 35(6): 970-976. doi: 10.7507/1001-5515.201803025 Copy

1.	周光华, 李岳峰, 孟群. 医学图像处理技术与应用分析. 中国卫生信息管理杂志, 2011, 8(6): 44-47.
2.	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets//Proceedings of the 2014 Conference on Advances in Neural Information Processing Systems 27. Montreal, Canada: Curran Associates, Inc., 2014: 2672-2680.
3.	Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv, 2015: 1511.06434.
4.	Xue Yuan, Xu Tao, Zhang Han, et al. SegAN: Adversarial network with multi-scale L₁ loss for medical image segmentation. Neuroinformatics, 2018, 16(3/4): 383-392.
5.	Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network. CVPR, 2017, 2(3): 4.
6.	Isola P, Zhu Junyan, Zhou Tinghui, et al. Image-to-image translation with conditional adversarial networks//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017: 5967-5976.
7.	Wang Chaoyue, Xu Chang, Wang Chaohui, et al. Perceptual adversarial networks for image-to-image transformation. IEEE Trans Image Process, 2018, 27(8): 4066-4079.
8.	Li Jie, Skinner K A, Eustice R M, et al. WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robotics and Automation Letters, 2018, 3(1): 387-394.
9.	Wolterink J M, Leiner T, Viergever M A, et al. Generative adversarial networks for noise reduction in low-dose CT. IEEE Trans Med Imaging, 2017, 36(12): 2536-2545.
10.	Yang Qingsong, Yan Pingkun, Zhang Yanbo, et al. Low-dose CT image denoising using a generative adversarial network with wasserstein distance and perceptual loss. IEEE Trans Med Imaging, 2018, 37(6): 1348-1357.
11.	Arjovsky M, Chintala S, Bottou L. Wasserstein GAN. arXiv preprint arXiv, 2017: 1701.07875.
12.	Yi Xin, Babyn P. Sharpness-aware low-dose CT denoising using conditional generative adversarial network. J Digit Imaging, 2018, 31(5): 655-669.
13.	Schlegl T, Seeböck P, Waldstein S M, et al. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery//International Conference on Information Processing in Medical Imaging. Cham: Springer, 2017: 146-157.
14.	Kohl S, Bonekamp D, Schlemmer H P, et al. Adversarial networks for the detection of aggressive prostate cancer. arXiv preprint arXiv, 2017: 1702.
15.	Baumgartner C F, Koch L M, Tezcan K C, et al. Visual feature attribution using Wasserstein GANs//Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2017: 8309-8319.
16.	唐思源, 邢俊凤, 杨敏. 基于 BP 神经网络的医学图像分割新方法. 计算机科学, 2017, 44(S1): 240-243.
17.	Zhang Y, Yang L, Chen J, et al. Deep adversarial networks for biomedical image segmentation utilizing unannotated images//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2017: 408-416.
18.	Kamnitsas K, Baumgartner C, Ledig C, et al. Unsupervised domain adaptation in brain lesion segmentation with adversarial networks//International Conference on Information Processing in Medical Imaging. Cham: Springer, 2017: 597-609.
19.	Dai W, Dong N, Wang Z, et al. SCAN: Structure correcting adversarial network for organ segmentation in chest X-rays//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Cham: Springer, 2018: 263-273.
20.	Nie D, Trullo R, Lian J, et al. Medical image synthesis with context-aware generative adversarial networks//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2017: 417-425.
21.	Tu Zhuowen, Bai Xiang. Auto-context and its application to high-level vision tasks and 3D brain image segmentation. IEEE Trans Pattern Anal Mach Intell, 2010, 32(10): 1744-1757.
22.	Wolterink J M, Dinkla A M, Savenije M H F, et al. Deep MR to CT synthesis using unpaired data//International Workshop on Simulation and Synthesis in Medical Imaging. Cham: Springer, 2017: 14-23.
23.	Ben-Cohen A, Klang E, Raskin S P, et al. Virtual PET images from CT data using deep convolutional networks: initial results//International Workshop on Simulation and Synthesis in Medical Imaging. Cham: Springer, 2017: 49-57.
24.	Bi L, Kim J, Kumar A, et al. Synthesis of positron emission tomography (PET) images via multi-channel generative adversarial networks (GANs)//Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment. Cham: Springer, 2017: 43-51.
25.	Chartsias A, Joyce T, Dharmakumar R, et al. Adversarial image synthesis for unpaired multi-modal cardiac data//International Workshop on Simulation and Synthesis in Medical Imaging. Cham: Springer, 2017: 3-13.
26.	Costa P, Galdran A, Meyer M I, et al. End-to-end adversarial retinal image synthesis. IEEE Trans Med Imaging, 2018, 37(3, SI): 781-791.
27.	Zhao He, Li Huiqi, Maurer-Stroh S, et al. Synthesizing retinal and neuronal images with generative adversarial nets. Med Image Anal, 2018, 49: 14-26.
28.	Yang Guang, Yu Simiao, Dong Hao, et al. DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans Med Imaging, 2018, 37(6): 1310-1321.
29.	Shitrit O, Raviv T R. Accelerated magnetic resonance imaging by adversarial neural network//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Cham: Springer, 2017: 30-38.
30.	Li Z, Wang Y, Yu J. Reconstruction of thin-slice medical images using generative adversarial network//International Workshop on Machine Learning in Medical Imaging. Cham: Springer, 2017: 325-333.
31.	Zhang L, Gooya A, Frangi A F. Semi-supervised assessment of incomplete LV coverage in cardiac MRI using generative adversarial nets//International Workshop on Simulation and Synthesis in Medical Imaging. Cham: Springer, 2017: 61-68.
32.	Hu B, Tang Y, Eric I, et al. Unsupervised learning for cell-level visual representation with generative adversarial networks. IEEE Journal of Biomedical and Health Informatics, 2018: 1.
33.	Gulrajani I, Ahmed F, Arjovsky M, et al. Improved training of Wasserstein GANs//Advances in Neural Information Processing Systems, 2017: 5767-5777.
34.	Chen X, Duan Y, Houthooft R, et al. InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets//Advances in Neural Information Processing Systems, 2016: 2172-2180.
35.	Frid-Adar M, Klang E, Amitai M, et al. Synthetic data augmentation using GAN for improved liver lesion classification//2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE, 2018: 289-293.
36.	Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks//IEEE International Conference on Computer Vision, 2017: 2223-2232.
37.	Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.
38.	Chen H, Qi X, Yu L, et al. DCAN: deep contour-aware networks for accurate gland segmentation//Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2016: 2487-2496.
39.	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv, 2014: 1409.1556.
40.	Yan P, Xu S, Rastinehad A R, et al. Adversarial image registration with application for MR and TRUS image fusion. arXiv preprint arXiv, 2018: 1804.11024.

1. 周光华, 李岳峰, 孟群. 医学图像处理技术与应用分析. 中国卫生信息管理杂志, 2011, 8(6): 44-47.
2. Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets//Proceedings of the 2014 Conference on Advances in Neural Information Processing Systems 27. Montreal, Canada: Curran Associates, Inc., 2014: 2672-2680.
3. Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv, 2015: 1511.06434.
4. Xue Yuan, Xu Tao, Zhang Han, et al. SegAN: Adversarial network with multi-scale L₁ loss for medical image segmentation. Neuroinformatics, 2018, 16(3/4): 383-392.
5. Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network. CVPR, 2017, 2(3): 4.
6. Isola P, Zhu Junyan, Zhou Tinghui, et al. Image-to-image translation with conditional adversarial networks//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017: 5967-5976.
7. Wang Chaoyue, Xu Chang, Wang Chaohui, et al. Perceptual adversarial networks for image-to-image transformation. IEEE Trans Image Process, 2018, 27(8): 4066-4079.
8. Li Jie, Skinner K A, Eustice R M, et al. WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robotics and Automation Letters, 2018, 3(1): 387-394.
9. Wolterink J M, Leiner T, Viergever M A, et al. Generative adversarial networks for noise reduction in low-dose CT. IEEE Trans Med Imaging, 2017, 36(12): 2536-2545.
10. Yang Qingsong, Yan Pingkun, Zhang Yanbo, et al. Low-dose CT image denoising using a generative adversarial network with wasserstein distance and perceptual loss. IEEE Trans Med Imaging, 2018, 37(6): 1348-1357.
11. Arjovsky M, Chintala S, Bottou L. Wasserstein GAN. arXiv preprint arXiv, 2017: 1701.07875.
12. Yi Xin, Babyn P. Sharpness-aware low-dose CT denoising using conditional generative adversarial network. J Digit Imaging, 2018, 31(5): 655-669.
13. Schlegl T, Seeböck P, Waldstein S M, et al. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery//International Conference on Information Processing in Medical Imaging. Cham: Springer, 2017: 146-157.
14. Kohl S, Bonekamp D, Schlemmer H P, et al. Adversarial networks for the detection of aggressive prostate cancer. arXiv preprint arXiv, 2017: 1702.
15. Baumgartner C F, Koch L M, Tezcan K C, et al. Visual feature attribution using Wasserstein GANs//Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2017: 8309-8319.
16. 唐思源, 邢俊凤, 杨敏. 基于 BP 神经网络的医学图像分割新方法. 计算机科学, 2017, 44(S1): 240-243.
17. Zhang Y, Yang L, Chen J, et al. Deep adversarial networks for biomedical image segmentation utilizing unannotated images//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2017: 408-416.
18. Kamnitsas K, Baumgartner C, Ledig C, et al. Unsupervised domain adaptation in brain lesion segmentation with adversarial networks//International Conference on Information Processing in Medical Imaging. Cham: Springer, 2017: 597-609.
19. Dai W, Dong N, Wang Z, et al. SCAN: Structure correcting adversarial network for organ segmentation in chest X-rays//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Cham: Springer, 2018: 263-273.
20. Nie D, Trullo R, Lian J, et al. Medical image synthesis with context-aware generative adversarial networks//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2017: 417-425.
21. Tu Zhuowen, Bai Xiang. Auto-context and its application to high-level vision tasks and 3D brain image segmentation. IEEE Trans Pattern Anal Mach Intell, 2010, 32(10): 1744-1757.
22. Wolterink J M, Dinkla A M, Savenije M H F, et al. Deep MR to CT synthesis using unpaired data//International Workshop on Simulation and Synthesis in Medical Imaging. Cham: Springer, 2017: 14-23.
23. Ben-Cohen A, Klang E, Raskin S P, et al. Virtual PET images from CT data using deep convolutional networks: initial results//International Workshop on Simulation and Synthesis in Medical Imaging. Cham: Springer, 2017: 49-57.
24. Bi L, Kim J, Kumar A, et al. Synthesis of positron emission tomography (PET) images via multi-channel generative adversarial networks (GANs)//Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment. Cham: Springer, 2017: 43-51.
25. Chartsias A, Joyce T, Dharmakumar R, et al. Adversarial image synthesis for unpaired multi-modal cardiac data//International Workshop on Simulation and Synthesis in Medical Imaging. Cham: Springer, 2017: 3-13.
26. Costa P, Galdran A, Meyer M I, et al. End-to-end adversarial retinal image synthesis. IEEE Trans Med Imaging, 2018, 37(3, SI): 781-791.
27. Zhao He, Li Huiqi, Maurer-Stroh S, et al. Synthesizing retinal and neuronal images with generative adversarial nets. Med Image Anal, 2018, 49: 14-26.
28. Yang Guang, Yu Simiao, Dong Hao, et al. DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans Med Imaging, 2018, 37(6): 1310-1321.
29. Shitrit O, Raviv T R. Accelerated magnetic resonance imaging by adversarial neural network//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Cham: Springer, 2017: 30-38.
30. Li Z, Wang Y, Yu J. Reconstruction of thin-slice medical images using generative adversarial network//International Workshop on Machine Learning in Medical Imaging. Cham: Springer, 2017: 325-333.
31. Zhang L, Gooya A, Frangi A F. Semi-supervised assessment of incomplete LV coverage in cardiac MRI using generative adversarial nets//International Workshop on Simulation and Synthesis in Medical Imaging. Cham: Springer, 2017: 61-68.
32. Hu B, Tang Y, Eric I, et al. Unsupervised learning for cell-level visual representation with generative adversarial networks. IEEE Journal of Biomedical and Health Informatics, 2018: 1.
33. Gulrajani I, Ahmed F, Arjovsky M, et al. Improved training of Wasserstein GANs//Advances in Neural Information Processing Systems, 2017: 5767-5777.
34. Chen X, Duan Y, Houthooft R, et al. InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets//Advances in Neural Information Processing Systems, 2016: 2172-2180.
35. Frid-Adar M, Klang E, Amitai M, et al. Synthetic data augmentation using GAN for improved liver lesion classification//2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE, 2018: 289-293.
36. Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks//IEEE International Conference on Computer Vision, 2017: 2223-2232.
37. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.
38. Chen H, Qi X, Yu L, et al. DCAN: deep contour-aware networks for accurate gland segmentation//Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2016: 2487-2496.
39. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv, 2014: 1409.1556.
40. Yan P, Xu S, Rastinehad A R, et al. Adversarial image registration with application for MR and TRUS image fusion. arXiv preprint arXiv, 2018: 1804.11024.

Journal of Biomedical Engineering

Applications of generative adversarial networks in medical image processing

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content