Research on grading algorithm of diabetic retinopathy based on cross-layer bilinear pooling_Journal of Biomedical Engineering

Authors：

 LIANG Liming , PENG Renjie , FENG Jun , YIN Jiang

School of Electrical Engineering and Automation, Jiangxi University of Science and Technology, Ganzhou, Jiangxi 341000, P. R. China;

Corresponding author：

LIANG Liming, Email: lianglm67@163.com

Keywords：

Cross-layer bilinear pooling; Retinopathy grade; Random puzzle generator; Progressive training; Center loss

DOI：

10.7507/1001-5515.202104038

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Considering the small differences between different types in the diabetic retinopathy (DR) grading task, a retinopathy grading algorithm based on cross-layer bilinear pooling is proposed. Firstly, the input image is cropped according to the Hough circle transform (HCT), and then the image contrast is improved by the preprocessing method; then the squeeze excitation group residual network (SEResNeXt) is used as the backbone of the model, and a cross-layer bilinear pooling module is introduced for classification. Finally, a random puzzle generator is introduced in the training process for progressive training, and the center loss (CL) and focal loss (FL) methods are used to further improve the effect of the final classification. The quadratic weighted Kappa (QWK) is 90.84% in the Indian Diabetic Retinopathy Image Dataset (IDRiD), and the area under the receiver operating characteristic curve (AUC) in the Messidor-2 dataset (Messidor-2) is 88.54%. Experiments show that the algorithm proposed in this paper has a certain application value in the field of diabetic retina grading.

Citation： LIANG Liming, PENG Renjie, FENG Jun, YIN Jiang. Research on grading algorithm of diabetic retinopathy based on cross-layer bilinear pooling. Journal of Biomedical Engineering, 2022, 39(5): 928-936. doi: 10.7507/1001-5515.202104038 Copy

1.	Cho N H, Shaw J E, Karuranga S, et al. IDF diabetes atlas: global estimates of diabetes prevalence for 2017 and projections for 2045. Diabetes Res Clin Pract, 2018, 138: 271-281.
2.	Haneda S, Yamashita H. International clinical diabetic retinopathy disease severity scale. Nihon rinsho. Japanese Journal of Clinical Medicine, 2010, 68: 228-235.
3.	Cunha L P, Figueiredo E A, Araújo H P, et al. Non-mydriatic fundus retinography in screening for diabetic retinopathy: agreement between family physicians, general ophthalmologists, and a retinal specialist. Frontiers in endocrinology, 2018, 9: 251.
4.	Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 2016, 316(22): 2402-2410.
5.	Pratt H, Coenen F, Broadbent D M, et al. Convolutional neural networks for diabetic retinopathy. Procedia computer science, 2016, 90: 200-205.
6.	Zhou K, Gu Z, Liu W, et al. Multi-cell multi-task convolutional neural networks for diabetic retinopathy grading//The 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2018: 2724-2727.
7.	Ren F, Cao P, Zhao D, et al. Diabetic macular edema grading in retinal images using vector quantization and semi-supervised learning. Technol Health Care, 2018, 26(S1): 389-397.
8.	Girard F, Kavalec C, Cheriet F. Joint segmentation and classification of retinal arteries/veins from fundus images. Artif Intell Med, 2019, 94: 96-109.
9.	Mahiba C, Jayachandran A. Severity analysis of diabetic retinopathy in retinal images using hybrid structure descriptor and modified CNNs. Measurement, 2019, 135: 762-767.
10.	Porwal P, Pachade S, Kamble R, et al. Indian diabetic retinopathy image dataset (IDRiD): a database for diabetic retinopathy screening research. Data, 2018, 3(3): 25.
11.	Decencière E, Zhang X, Cazuguel G, et al. Feedback on a publicly distributed image database: the Messidor database. Image Analysis & Stereology, 2014, 33(3): 231-234.
12.	Wilkinson C P, Ferris F L, Klein R E, et al. Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales. Ophthalmology, 2003, 110(9): 1677-1682.
13.	Sánchez C I, Niemeijer M, Dumitrescu A V, et al. Evaluation of a computer-aided diagnosis system for diabetic retinopathy screening on public data. Invest Ophthalmol Vis Sci, 2011, 52(7): 4866-4871.
14.	Wang Z, Yin Y, Shi J, et al. Zoom-in-net: deep mining lesions for diabetic retinopathy detection//International Conference on Medical Image Computing and Computer-Assisted Intervention, Cham: Springer, 2017: 267-275.
15.	Hu J, Shen L, Sun G. Squeeze-and-excitation networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 7132-7141.
16.	Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1492-1500.
17.	Yu C, Zhao X, Zheng Q, et al. Hierarchical bilinear pooling for fine-grained visual recognition//Proceedings of the European Conference on Computer Vision (ECCV), München: Springer, 2018: 574-589.
18.	Qian Q, Shang L, Sun B, et al. Softtriple loss: deep metric learning without triplet sampling//Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul: IEEE, 2019: 6450-6458.
19.	Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection long//Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2980-2988.
20.	Karras T, Aila T, Laine S, et al. Progressive growing of gans for improved quality, stability, and variation. arXiv: 1710.10196, 2017. https://doi.org/10.48550/arXiv.1710.10196.
21.	Wei C, Xie L, Ren X, et al. Iterative reorganization with weak spatial constraints: solving arbitrary jigsaw puzzles for unsupervised representation learning//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach: IEEE, 2019: 1910-1919.
22.	Chen Y, Bai Y, Zhang W, et al. Destruction and construction learning for fine-grained image recognition//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach: IEEE, 2019: 5157-5166.
23.	Smith L N. Cyclical learning rates for training neural networks//2017 IEEE winter conference on applications of computer vision (WACV), Nevada: IEEE, 2017: 464-472.
24.	Wang Y, Morariu V I, Davis L S. Learning a discriminative filter bank within a CNN for fine-grained recognition//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Utah: IEEE, 2018: 4148-4157.
25.	Du R , Chang D , Bhunia A K , et al. Fine-grained visual classification via progressive multi-granularity training of jigsaw patches//European Conference on Computer Vision (ECCV 2020), Glasgow: ECCV, 2020.
26.	Porwal P, Pachade S, Kokare M, et al. IDRiD: diabetic retinopathy-segmentation and grading challenge. Med Image Anal, 2020, 59: 101561.
27.	Tian L, Ma L, Wen Z, et al. Learning discriminative representations for fine-grained diabetic retinopathy grading//2021 International Joint Conference on Neural Networks (IJCNN), Padua: IEEE, 2021: 1-8.
28.	Voets M, Møllersen K, Bongo L A. Reproduction study using public data of: development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. PLoS One, 2019, 14(6): e0217541.
29.	Toledo-Cortés S, de la Pava M, Perdómo O, et al. Hybrid deep learning gaussian process for diabetic retinopathy diagnosis and uncertainty quantification// 7^th International Workshop on Ophthalmic Medical Image Analysis (OMIA 2020). Lima, Peru: OMIA and MICCAI, 2020: 206-215.

1. Cho N H, Shaw J E, Karuranga S, et al. IDF diabetes atlas: global estimates of diabetes prevalence for 2017 and projections for 2045. Diabetes Res Clin Pract, 2018, 138: 271-281.
2. Haneda S, Yamashita H. International clinical diabetic retinopathy disease severity scale. Nihon rinsho. Japanese Journal of Clinical Medicine, 2010, 68: 228-235.
3. Cunha L P, Figueiredo E A, Araújo H P, et al. Non-mydriatic fundus retinography in screening for diabetic retinopathy: agreement between family physicians, general ophthalmologists, and a retinal specialist. Frontiers in endocrinology, 2018, 9: 251.
4. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 2016, 316(22): 2402-2410.
5. Pratt H, Coenen F, Broadbent D M, et al. Convolutional neural networks for diabetic retinopathy. Procedia computer science, 2016, 90: 200-205.
6. Zhou K, Gu Z, Liu W, et al. Multi-cell multi-task convolutional neural networks for diabetic retinopathy grading//The 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2018: 2724-2727.
7. Ren F, Cao P, Zhao D, et al. Diabetic macular edema grading in retinal images using vector quantization and semi-supervised learning. Technol Health Care, 2018, 26(S1): 389-397.
8. Girard F, Kavalec C, Cheriet F. Joint segmentation and classification of retinal arteries/veins from fundus images. Artif Intell Med, 2019, 94: 96-109.
9. Mahiba C, Jayachandran A. Severity analysis of diabetic retinopathy in retinal images using hybrid structure descriptor and modified CNNs. Measurement, 2019, 135: 762-767.
10. Porwal P, Pachade S, Kamble R, et al. Indian diabetic retinopathy image dataset (IDRiD): a database for diabetic retinopathy screening research. Data, 2018, 3(3): 25.
11. Decencière E, Zhang X, Cazuguel G, et al. Feedback on a publicly distributed image database: the Messidor database. Image Analysis & Stereology, 2014, 33(3): 231-234.
12. Wilkinson C P, Ferris F L, Klein R E, et al. Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales. Ophthalmology, 2003, 110(9): 1677-1682.
13. Sánchez C I, Niemeijer M, Dumitrescu A V, et al. Evaluation of a computer-aided diagnosis system for diabetic retinopathy screening on public data. Invest Ophthalmol Vis Sci, 2011, 52(7): 4866-4871.
14. Wang Z, Yin Y, Shi J, et al. Zoom-in-net: deep mining lesions for diabetic retinopathy detection//International Conference on Medical Image Computing and Computer-Assisted Intervention, Cham: Springer, 2017: 267-275.
15. Hu J, Shen L, Sun G. Squeeze-and-excitation networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Utah: IEEE, 2018: 7132-7141.
16. Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1492-1500.
17. Yu C, Zhao X, Zheng Q, et al. Hierarchical bilinear pooling for fine-grained visual recognition//Proceedings of the European Conference on Computer Vision (ECCV), München: Springer, 2018: 574-589.
18. Qian Q, Shang L, Sun B, et al. Softtriple loss: deep metric learning without triplet sampling//Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul: IEEE, 2019: 6450-6458.
19. Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection long//Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2980-2988.
20. Karras T, Aila T, Laine S, et al. Progressive growing of gans for improved quality, stability, and variation. arXiv: 1710.10196, 2017. https://doi.org/10.48550/arXiv.1710.10196" target="_blank">org/10.48550/arXiv.1710.10196">https://doi.org/10.48550/arXiv.1710.10196.
21. Wei C, Xie L, Ren X, et al. Iterative reorganization with weak spatial constraints: solving arbitrary jigsaw puzzles for unsupervised representation learning//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach: IEEE, 2019: 1910-1919.
22. Chen Y, Bai Y, Zhang W, et al. Destruction and construction learning for fine-grained image recognition//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach: IEEE, 2019: 5157-5166.
23. Smith L N. Cyclical learning rates for training neural networks//2017 IEEE winter conference on applications of computer vision (WACV), Nevada: IEEE, 2017: 464-472.
24. Wang Y, Morariu V I, Davis L S. Learning a discriminative filter bank within a CNN for fine-grained recognition//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Utah: IEEE, 2018: 4148-4157.
25. Du R , Chang D , Bhunia A K , et al. Fine-grained visual classification via progressive multi-granularity training of jigsaw patches//European Conference on Computer Vision (ECCV 2020), Glasgow: ECCV, 2020.
26. Porwal P, Pachade S, Kokare M, et al. IDRiD: diabetic retinopathy-segmentation and grading challenge. Med Image Anal, 2020, 59: 101561.
27. Tian L, Ma L, Wen Z, et al. Learning discriminative representations for fine-grained diabetic retinopathy grading//2021 International Joint Conference on Neural Networks (IJCNN), Padua: IEEE, 2021: 1-8.
28. Voets M, Møllersen K, Bongo L A. Reproduction study using public data of: development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. PLoS One, 2019, 14(6): e0217541.
29. Toledo-Cortés S, de la Pava M, Perdómo O, et al. Hybrid deep learning gaussian process for diabetic retinopathy diagnosis and uncertainty quantification// 7^th International Workshop on Ophthalmic Medical Image Analysis (OMIA 2020). Lima, Peru: OMIA and MICCAI, 2020: 206-215.

Journal of Biomedical Engineering

Research on grading algorithm of diabetic retinopathy based on cross-layer bilinear pooling

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content