The application and value evaluation of assisted diagnosis system for five fundus lesion based on artificial intelligence combined with optical coherence tomography_Chinese Journal of Ocular Fundus Diseases

Authors：

Ma Jian ¹ , Chen Shumei ^1,2 , Wang Min ³ , Wu Fuli ⁴ , Wu Jian ^4,5 ,  Fang Xiaoyun ¹

1. Department of Ophthalmology, Eye Center of the Second Affiliated Hospitial, School of Medicine, Zhejiang University, Hangzhou 310009, China;
2. Department of Ophthalmology, The First People's Hospital of Jiashan County, Jiaxing 314100, China;
3. Department of Ophthalmology, Nanxun District People's Hospital, Huzhou 313009, China;
4. Ruiyi Artificial Intelligence Research Center, Zhejiang University, Hangzhou 310000, China;
5. School of Public Health, Zhejiang University Medical College, Hangzhou 310000, China;
Wu Jian and Fang Xiaoyun are contributed equally to the article;

Corresponding author：

Fang Xiaoyun, Email: xiaoyunfang@zju.edu.cn

Keywords：

Artificial intelligence; Tomography, optical coherence; Retinal perforations; Macular edema; Choroidal neovascularization; Macular degeneration; Macular epiretinal membrane

DOI：

10.3760/cma.j.cn511434-20220104-00002

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To establish an artificial intelligence robot-assisted diagnosis system for fundus diseases based on deep learning optical coherence tomography (OCT) and evaluate its application value. Methods Diagnostic test studies. From 2016 to 2019, 25 000 OCT images of 25 000 patients treated at the Eye Center of the Second Affiliated Hospital of Zhejiang University School of Medicine were used as training sets and validation sets for the fundus intelligent assisted diagnosis system. Among them, macular epiretinal membrane (MERM), macular edema, macular hole, choroidal neovascularization (CNV), and age-related macular degeneration (AMD) were 5 000 sheets each. The training set and the verification set are 18 124 and 6 876 sheets, respectively. Through the transfer learning Attention ResNet structure algorithm, the OCT image was characterized by lesion identification, the disease feature was extracted by a specific procedure, and the given image was distinguished from other types of disease according to the statistical characteristics of the target lesion. The model algorithms of MERM, macular edema, macular hole, CNV and AMD were initially formed, and the fundus intelligent auxiliary diagnosis system of five models was established. The performance of each model-assisted diagnosis in the fundus intelligent auxiliary diagnostic system was evaluated by applying the subject working characteristic curve, area under the curve (AUC), sensitivity, and specificity. Results With the intelligent auxiliary diagnosis system, the diagnostic sensitivity of the MERM was 93.5%, the specificity was 99.23%, and AUC was 0.983 7; the diagnostic sensitivity of macular edema was 99.02%, the specificity was 98.17%, and AUC was 0.994 6; the diagnostic sensitivity of macular hole was 98.91%, the specificity was 99.91%, AUC was 0.996 2; the diagnostic sensitivity of CNV was 97.54%, the specificity was 94.71%, AUC was 0.987 5; the diagnostic sensitivity of AMD was 95.12%, the specificity was 97.09%, AUC was 0.985 3. Conclusions The artificial intelligence robot-assisted diagnosis system for fundus diseases based on deep learning for OCT images has accurate and efficient diagnostic performance for assisting the diagnosis of MERM, macular edema, macular hole, CNV, and AMD.

Citation： Ma Jian, Chen Shumei, Wang Min, Wu Fuli, Wu Jian, Fang Xiaoyun. The application and value evaluation of assisted diagnosis system for five fundus lesion based on artificial intelligence combined with optical coherence tomography. Chinese Journal of Ocular Fundus Diseases, 2022, 38(2): 126-131. doi: 10.3760/cma.j.cn511434-20220104-00002 Copy

1.	陈有信, 张碧磊, 张弘哲. 眼科人工智能技术的现状与问题[J]. 中华眼底病杂志, 2019, 35(2): 119-123. DOI: 10.3760/cma.j.issn.1005-1015.2019.02.003.Chen YX, Zhang BL, Zhang HZ. Insights and prospectives of ophthalmologic artificial intelligence technology[J]. Chin J Ocul Fundus Dis, 2019, 35(2): 119-123. DOI: 10.3760/cma.j.issn.1005-1015.2019.02.003.
2.	李建军, 徐亮, 路从磊, 等. 远程眼科阅片及时性及阅片报告详细性的初步研究[J]. 眼科, 2016, 25(1): 13-17. DOI: 10.13281/j.cnki.issn.1004-4469.2016.01.004.Li JJ, Xu L, Lu CL, et al. The timeliness of the image reading and the description details of the reading reports in teleophthalmology[J]. Ophthalmology in China, 2016, 25(1): 13-17. DOI: 10.13281/j.cnki.issn.1004-4469.2016.01.004.
3.	Kapoor R, Whigham BT, Al-Aswad LA. Artificial intelligence and optical coherence tomography imaging[J]. Asia Pac J Ophthalmol (Phila), 2019, 8(2): 187-194. DOI: 10.22608/APO.201904.
4.	Rahimy E. Deep learning applications in ophthalmology[J]. Curr Opin Ophthalmol, 2018, 29(3): 254-260. DOI: 10.1097/ICU.0000000000000470.
5.	Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis[J]. Med Image Anal, 2017, 42: 60-88. DOI: 10.1016/j.media.2017.07.005.
6.	Shen D, Wu G, Suk HI. Deep learning in medical image analysis[J]. Annu Rev Biomed Eng, 2017, 19: 221-248. DOI: 10.1146/annurev-bioeng-071516-044442.
7.	Du XL, Li WB, Hu BJ. Application of artificial intelligence in ophthalmology[J]. Int J Ophthalmol, 2018, 11(9): 1555-1561. DOI: 10.18240/ijo.2018.09.21.
8.	Ting DSW, Peng L, Varadarajan AV, et al. Deep learning in ophthalmology: the technical and clinical considerations[J/OL]. Prog Retin Eye Res, 2019, 72: 100759[2019-04-29]. https://pubmed.ncbi.nlm.nih.gov/31048019/. DOI: 10.1016/j.preteyeres.2019.04.003.
9.	Li Z, Li W, Wei Y, et al. Deep learning based automatic diagnosis of first-episode psychosis, bipolar disorder and healthy controls[J/OL]. Comput Med Imaging Graph, 2021, 89: 101882[2021-02-25]. https://pubmed.ncbi.nlm.nih.gov/33684730/. DOI: 10.1016/j.compmedimag.2021.101882.
10.	Hu J, Shen L, Albanie S, et al. Squeeze-and-excitation networks[J]. IEEE Trans Pattern Anal Mach Intell, 2020, 42(8): 2011-2023. DOI: 10.1109/tpami.2019.2913372.
11.	Huang G, Liu Z, Laurens van der M, et al. Densely connected convolutional networks[C]//IEEE 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, 2017: 2261-2269. DOI:10.1109/cvpr.2017.243.
12.	Wang F, Jiang MQ, Qian C, et al. Residual attention network for image classification[C]//IEEE 2017 IEEE Conference on ComputerVision and Pattern Recognition (CVPR). Honolulu, 2017: 6450-6458. DOI:10.1109/CVPR.2017.683.
13.	Wang X, Tang F, Chen H, et al. UD-MIL: uncertainty-driven deep multiple instance learning for OCT image classification[J]. IEEE J Biomed Health Inform, 2020, 24(12): 3431-3442. DOI: 10.1109/JBHI.2020.2983730.
14.	Lee CS, Baughman DM, Lee AY. Deep learning is effective for the classification of OCT images of normal versus age-related macular degeneration[J]. Ophthalmol Retina, 2017, 1(4): 322-327. DOI: 10.1016/j.oret.2016.12.009.
15.	Kermany DS, Goldbaum M, Cai W, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning[J]. Cell, 2018, 172(5): 1122-1131. DOI: 10.1016/j.cell.2018.02.010.
16.	Mehta N, Lee CS, Mendonça LSM, et al. Model-to-data approach for deep learning in optical coherence tomography intraretinal fluid segmentation[J]. JAMA Ophthalmol, 2020, 138(10): 1017-1024. DOI: 10.1001/jamaophthalmol.2020.2769.
17.	Lu W, Tong Y, Yu Y, et al. Deep learning-based automated classification of multi-categorical abnormalities from optical coherence tomography images[J]. Transl Vis Sci Technol, 2018, 7(6): 41. DOI: 10.1167/tvst.7.6.41.
18.	Hwang DK, Hsu CC, Chang KJ, et al. Artificial intelligence-based decision-making for age-related macular degeneration[J]. Theranostics, 2019, 9(1): 232-245. DOI: 10.7150/thno.28447.
19.	Milea D, Najjar RP, Zhubo J, et al. Artificial intelligence to detect papilledema from ocular fundus photographs[J]. N Engl J Med, 2020, 382(18): 1687-1695. DOI: 10.1056/NEJMoa1917130.
20.	Goldberg RA, Waheed NK, Duker JS. Optical coherence tomography in the preoperative and postoperative management of macular hole and epiretinal membrane[J/OL]. Br J Ophthalmol, 2014, 98 Suppl 2: Sii20-ii23[2014-03-13]. https://pubmed.ncbi.nlm.nih.gov/24627250/. DOI: 10.1136/bjophthalmol-2013-304447.
21.	Hee MR, Puliafito CA, Wong C, et al. Optical coherence tomography of macular holes[J]. Ophthalmology, 1995, 102(5): 748-756. DOI: 10.1016/s0161-6420(95)30959-1.
22.	Hee MR, Puliafito CA, Wong C, et al. Quantitative assessment of macular edema with optical coherence tomography[J]. Arch Ophthalmol, 1995, 113(8): 1019-1029. DOI: 10.1001/archopht.1995.01100080071031.
23.	Ferrara D, Waheed NK, Duker JS. Investigating the choriocapillaris and choroidal vasculature with new optical coherence tomography technologies[J]. Prog Retin Eye Res, 2016, 52: 130-155. DOI: 10.1016/j.preteyeres.2015.10.002.
24.	Zhukova SI. Znachenie opticheskoi kogerentnoi tomografii s funktsiei angiografii v diagnostike miopicheskoi khorioidal'noi neovaskulyarizatsii [Optical coherence tomography angiography in the diagnosis of myopic choroidal neovascularization][J]. Vestn Oftalmol, 2021, 137(5): 68-77. DOI: 10.17116/oftalma202113705168.
25.	Kanagasingam Y, Bhuiyan A, Abràmoff MD, et al. Progress on retinal image analysis for age related macular degeneration[J]. Prog Retin Eye Res, 2014, 38: 20-42. DOI: 10.1016/j.preteyeres.2013.10.002.

1. 陈有信, 张碧磊, 张弘哲. 眼科人工智能技术的现状与问题[J]. 中华眼底病杂志, 2019, 35(2): 119-123. DOI: 10.3760/cma.j.issn.1005-1015.2019.02.003.Chen YX, Zhang BL, Zhang HZ. Insights and prospectives of ophthalmologic artificial intelligence technology[J]. Chin J Ocul Fundus Dis, 2019, 35(2): 119-123. DOI: 10.3760/cma.j.issn.1005-1015.2019.02.003.
2. 李建军, 徐亮, 路从磊, 等. 远程眼科阅片及时性及阅片报告详细性的初步研究[J]. 眼科, 2016, 25(1): 13-17. DOI: 10.13281/j.cnki.issn.1004-4469.2016.01.004.Li JJ, Xu L, Lu CL, et al. The timeliness of the image reading and the description details of the reading reports in teleophthalmology[J]. Ophthalmology in China, 2016, 25(1): 13-17. DOI: 10.13281/j.cnki.issn.1004-4469.2016.01.004.
3. Kapoor R, Whigham BT, Al-Aswad LA. Artificial intelligence and optical coherence tomography imaging[J]. Asia Pac J Ophthalmol (Phila), 2019, 8(2): 187-194. DOI: 10.22608/APO.201904.
4. Rahimy E. Deep learning applications in ophthalmology[J]. Curr Opin Ophthalmol, 2018, 29(3): 254-260. DOI: 10.1097/ICU.0000000000000470.
5. Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis[J]. Med Image Anal, 2017, 42: 60-88. DOI: 10.1016/j.media.2017.07.005.
6. Shen D, Wu G, Suk HI. Deep learning in medical image analysis[J]. Annu Rev Biomed Eng, 2017, 19: 221-248. DOI: 10.1146/annurev-bioeng-071516-044442.
7. Du XL, Li WB, Hu BJ. Application of artificial intelligence in ophthalmology[J]. Int J Ophthalmol, 2018, 11(9): 1555-1561. DOI: 10.18240/ijo.2018.09.21.
8. Ting DSW, Peng L, Varadarajan AV, et al. Deep learning in ophthalmology: the technical and clinical considerations[J/OL]. Prog Retin Eye Res, 2019, 72: 100759[2019-04-29]. https://pubmed.ncbi.nlm.nih.gov/31048019/. DOI: 10.1016/j.preteyeres.2019.04.003.
9. Li Z, Li W, Wei Y, et al. Deep learning based automatic diagnosis of first-episode psychosis, bipolar disorder and healthy controls[J/OL]. Comput Med Imaging Graph, 2021, 89: 101882[2021-02-25]. https://pubmed.ncbi.nlm.nih.gov/33684730/. DOI: 10.1016/j.compmedimag.2021.101882.
10. Hu J, Shen L, Albanie S, et al. Squeeze-and-excitation networks[J]. IEEE Trans Pattern Anal Mach Intell, 2020, 42(8): 2011-2023. DOI: 10.1109/tpami.2019.2913372.
11. Huang G, Liu Z, Laurens van der M, et al. Densely connected convolutional networks[C]//IEEE 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, 2017: 2261-2269. DOI:10.1109/cvpr.2017.243.
12. Wang F, Jiang MQ, Qian C, et al. Residual attention network for image classification[C]//IEEE 2017 IEEE Conference on ComputerVision and Pattern Recognition (CVPR). Honolulu, 2017: 6450-6458. DOI:10.1109/CVPR.2017.683.
13. Wang X, Tang F, Chen H, et al. UD-MIL: uncertainty-driven deep multiple instance learning for OCT image classification[J]. IEEE J Biomed Health Inform, 2020, 24(12): 3431-3442. DOI: 10.1109/JBHI.2020.2983730.
14. Lee CS, Baughman DM, Lee AY. Deep learning is effective for the classification of OCT images of normal versus age-related macular degeneration[J]. Ophthalmol Retina, 2017, 1(4): 322-327. DOI: 10.1016/j.oret.2016.12.009.
15. Kermany DS, Goldbaum M, Cai W, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning[J]. Cell, 2018, 172(5): 1122-1131. DOI: 10.1016/j.cell.2018.02.010.
16. Mehta N, Lee CS, Mendonça LSM, et al. Model-to-data approach for deep learning in optical coherence tomography intraretinal fluid segmentation[J]. JAMA Ophthalmol, 2020, 138(10): 1017-1024. DOI: 10.1001/jamaophthalmol.2020.2769.
17. Lu W, Tong Y, Yu Y, et al. Deep learning-based automated classification of multi-categorical abnormalities from optical coherence tomography images[J]. Transl Vis Sci Technol, 2018, 7(6): 41. DOI: 10.1167/tvst.7.6.41.
18. Hwang DK, Hsu CC, Chang KJ, et al. Artificial intelligence-based decision-making for age-related macular degeneration[J]. Theranostics, 2019, 9(1): 232-245. DOI: 10.7150/thno.28447.
19. Milea D, Najjar RP, Zhubo J, et al. Artificial intelligence to detect papilledema from ocular fundus photographs[J]. N Engl J Med, 2020, 382(18): 1687-1695. DOI: 10.1056/NEJMoa1917130.
20. Goldberg RA, Waheed NK, Duker JS. Optical coherence tomography in the preoperative and postoperative management of macular hole and epiretinal membrane[J/OL]. Br J Ophthalmol, 2014, 98 Suppl 2: Sii20-ii23[2014-03-13]. https://pubmed.ncbi.nlm.nih.gov/24627250/. DOI: 10.1136/bjophthalmol-2013-304447.
21. Hee MR, Puliafito CA, Wong C, et al. Optical coherence tomography of macular holes[J]. Ophthalmology, 1995, 102(5): 748-756. DOI: 10.1016/s0161-6420(95)30959-1.
22. Hee MR, Puliafito CA, Wong C, et al. Quantitative assessment of macular edema with optical coherence tomography[J]. Arch Ophthalmol, 1995, 113(8): 1019-1029. DOI: 10.1001/archopht.1995.01100080071031.
23. Ferrara D, Waheed NK, Duker JS. Investigating the choriocapillaris and choroidal vasculature with new optical coherence tomography technologies[J]. Prog Retin Eye Res, 2016, 52: 130-155. DOI: 10.1016/j.preteyeres.2015.10.002.
24. Zhukova SI. Znachenie opticheskoi kogerentnoi tomografii s funktsiei angiografii v diagnostike miopicheskoi khorioidal'noi neovaskulyarizatsii [Optical coherence tomography angiography in the diagnosis of myopic choroidal neovascularization][J]. Vestn Oftalmol, 2021, 137(5): 68-77. DOI: 10.17116/oftalma202113705168.
25. Kanagasingam Y, Bhuiyan A, Abràmoff MD, et al. Progress on retinal image analysis for age related macular degeneration[J]. Prog Retin Eye Res, 2014, 38: 20-42. DOI: 10.1016/j.preteyeres.2013.10.002.

Chinese Journal of Ocular Fundus Diseases

The application and value evaluation of assisted diagnosis system for five fundus lesion based on artificial intelligence combined with optical coherence tomography

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content