Research on classification method of multimodal magnetic resonance images of Alzheimer’s disease based on generalized convolutional neural networks_Journal of Biomedical Engineering

Authors：

QIN Zhiwei ^1,2 , LIU Zhao ³ ,  LU Yunmin ⁴ ,  ZHU Ping ^1,2

1. School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China;
2. National Engineering Research Center of Automotive Power and Intelligent Control, Shanghai Jiao Tong University, Shanghai 200240, P. R. China;
3. School of Design, Shanghai Jiao Tong University, Shanghai 200240, P. R. China;
4. Shanghai Jiao Tong University Affiliated Sixth People’s Hospital, Shanghai 200240, P. R. China;

Corresponding author：

LU Yunmin, Email: luyunmin@126.com; ZHU Ping, Email: pzhu@sjtu.edu.cn

Keywords：

Alzheimer’s disease; Multimodal magnetic resonance images; Generalized convolutional neural networks; Hybrid attention mechanism; Graph convolutional networks

DOI：

10.7507/1001-5515.202212046

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Alzheimer’s disease (AD) is a progressive and irreversible neurodegenerative disease. Neuroimaging based on magnetic resonance imaging (MRI) is one of the most intuitive and reliable methods to perform AD screening and diagnosis. Clinical head MRI detection generates multimodal image data, and to solve the problem of multimodal MRI processing and information fusion, this paper proposes a structural and functional MRI feature extraction and fusion method based on generalized convolutional neural networks (gCNN). The method includes a three-dimensional residual U-shaped network based on hybrid attention mechanism (3D HA-ResUNet) for feature representation and classification for structural MRI, and a U-shaped graph convolutional neural network (U-GCN) for node feature representation and classification of brain functional networks for functional MRI. Based on the fusion of the two types of image features, the optimal feature subset is selected based on discrete binary particle swarm optimization, and the prediction results are output by a machine learning classifier. The validation results of multimodal dataset from the AD Neuroimaging Initiative (ADNI) open-source database show that the proposed models have superior performance in their respective data domains. The gCNN framework combines the advantages of these two models and further improves the performance of the methods using single-modal MRI, improving the classification accuracy and sensitivity by 5.56% and 11.11%, respectively. In conclusion, the gCNN-based multimodal MRI classification method proposed in this paper can provide a technical basis for the auxiliary diagnosis of Alzheimer’s disease.

Citation： QIN Zhiwei, LIU Zhao, LU Yunmin, ZHU Ping. Research on classification method of multimodal magnetic resonance images of Alzheimer’s disease based on generalized convolutional neural networks. Journal of Biomedical Engineering, 2023, 40(2): 217-225. doi: 10.7507/1001-5515.202212046 Copy

1.	Alzheimer’s Association, 2021 Alzheimer’s disease facts and figures. Alzheimers Dement, 2021, 17(3): 327-406.
2.	Alzheimer’s Association, 2022 Alzheimer’s disease facts and figures. Alzheimers Dement, 2022, 18(4): 700-789.
3.	Beheshti I, Demirel H. Probability distribution function-based classification of structural MRI for the detection of Alzheimer’s disease. Comput Biol Med, 2015, 64: 208-216.
4.	Biswal B, Yetkin F Z, Haughton V M, et al. Functional connectivity in the motor cortex of resting human brain using echo‐planar MRI. Magn Reson Med, 1995, 34(4): 537-541.
5.	Alva M, Sonawane K. Hybrid feature vector generation for Alzheimer’s disease diagnosis using MRI images// 2019 IEEE 5th International Conference for Convergence in Technology (I2CT), Pune: ISBM School of Technology, 2019: 1-6.
6.	Saravanakumar S, Thangaraj P. A computer aided diagnosis system for identifying Alzheimer’s from MRI scan using improved Adaboost. J Med Syst, 2019, 43(3): 76-83.
7.	Uysal G, Ozturk M. Hippocampal atrophy based Alzheimer’s disease diagnosis via machine learning methods. J Neurosci Methods, 2020, 337: 108669.
8.	Jain R, Jain N, Aggarwal A, et al. Convolutional neural network based Alzheimer’s disease classification from magnetic resonance brain images. Cogn Syst Res, 2019, 57: 147-159.
9.	Ebrahimi A, Luo S, Chiong R. Deep sequence modelling for Alzheimer’s disease detection using MRI. Comput Biol Med, 2021, 134: 104537.
10.	Liu M, Li F, Yan H, et al. A multi-model deep convolutional neural network for automatic hippocampus segmentation and classification in Alzheimer’s disease. Neuroimage, 2020, 208: 116459.
11.	Yu F, Zhao B Q, Ge Q Q, et al. A lightweight spatial attention module with adaptive receptive fields in 3D convolutional neural network for Alzheimer’s disease classification// International Conference on Pattern Recognition (ICPR), Chile: Universidad de Talca, 2021: 575-586.
12.	Zhao Y, Dong Q, Zhang S, et al. Automatic recognition of fMRI-derived functional networks using 3D convolutional neural networks. IEEE Trans Biomed Eng, 2018, 65(9): 1975-1984.
13.	Ramzan F, Khan M U G, Rehamt A, et al. A deep learning approach for automated diagnosis and multi-class classification of Alzheimer’s disease stages using resting-state fMRI and residual neural networks. J Med Syst, 2019, 44(2): 37.
14.	Yu Q, Du Y, Chen J, et al. Application of graph theory to assess static and dynamic brain connectivity: approaches for building brain graphs. Proc IEEE, 2018, 106(5): 886-906.
15.	Bessadok A, Mahjoub M A, Rekik I. Graph neural networks in network neuroscience. IEEE Trans Pattern Anal Mach Intell, 2022. DOI: 10.1109/TPAMI.2022.3209686.
16.	Xu X, Li W, Tao M, et al. Effective and accurate diagnosis of subjective cognitive decline based on functional connection and graph theory view. Front Neurosci, 2020, 14: 577887.
17.	Song X, Elazab A, Zhang Y. Classification of mild cognitive impairment based on a combined high-order network and graph convolutional network. IEEE Access, 2020, 8: 42816-42827.
18.	Jiang H, Cao P, Xu M, et al. Hi-GCN: a hierarchical graph convolution network for graph embedding learning of brain network and brain disorders prediction. Comput Biol Med, 2020, 127: 104096.
19.	Wang L J, Yuan W F, Zeng L, et al. Dementia analysis from functional connectivity network with graph neural networks. Inf Process Manage, 2022, 59(3): 102901.
20.	Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation// Medical Image Computing and Computer-Assisted Intervention (MICCAI), Munich: Springer, 2015: 234-241.
21.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas: IEEE, 2016: 770-778.
22.	Hu J, Shen L, Albanie S, et al. Squeeze-and-Excitation networks. IEEE Trans Pattern Anal Mach Intell, 2020, 42(8): 2011-2023.
23.	VM Eguíluz, Chialvo D R, Cecchi G A, et al. Scale-Free brain functional networks. Phys Rev Lett, 2005, 94(1): 018102.
24.	Rubinov M, Sporns O. Complex network measures of brain connectivity: uses and interpretations. Neuroimage, 2010, 52(3): 1059-1069.
25.	Kipf T N, Welling M. Semi-Supervised classification with graph convolutional networks. arXiv preprint, 2016, arXiv: 1609.02907.
26.	Gao H Y, Ji S W. Graph U-Nets. IEEE Trans Pattern Anal Mach Intell, 2022, 44(9): 4948-4960.
27.	Kennedy J, Eberhart R. Particle swarm optimization// IEEE International Conference on Neural Networks (ICNN), Perth: IEEE, 1995, 48: 1942-1948.
28.	Kennedy J, Eberhart R C. A discrete binary version of the particle swarm algorithm// IEEE International Conference on Systems, Man, and Cybernetics, Orlando: IEEE, 1997.
29.	Kasun L, Zhou H, Huang G B, et al. Representational learning with ELMs for big data. IEEE Intell Syst, 2013, 28(6): 31-34.
30.	Hamilton W L, Ying R, Leskovec J. Inductive representation learning on large graphs// Proceedings of the 31st International Conference on Neural Information Processing Systems (NeurIPS), Long Beach: NeurIPS Foundation, 2017: 1025-1035.
31.	Du J, Zhang S, Wu G, et al. Topology adaptive graph convolutional networks// Proceedings of the 5th International Conference on Learning Representations (ICLR), Toulon: Academic Press, 2017.
32.	Morris C, Ritzert M, Fey M, et al. Weisfeiler and leman go neural: higher-order graph neural networks// Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI), Hawaii: AAAI, 2019: 4602-4609.

1. Alzheimer’s Association, 2021 Alzheimer’s disease facts and figures. Alzheimers Dement, 2021, 17(3): 327-406.
2. Alzheimer’s Association, 2022 Alzheimer’s disease facts and figures. Alzheimers Dement, 2022, 18(4): 700-789.
3. Beheshti I, Demirel H. Probability distribution function-based classification of structural MRI for the detection of Alzheimer’s disease. Comput Biol Med, 2015, 64: 208-216.
4. Biswal B, Yetkin F Z, Haughton V M, et al. Functional connectivity in the motor cortex of resting human brain using echo‐planar MRI. Magn Reson Med, 1995, 34(4): 537-541.
5. Alva M, Sonawane K. Hybrid feature vector generation for Alzheimer’s disease diagnosis using MRI images// 2019 IEEE 5th International Conference for Convergence in Technology (I2CT), Pune: ISBM School of Technology, 2019: 1-6.
6. Saravanakumar S, Thangaraj P. A computer aided diagnosis system for identifying Alzheimer’s from MRI scan using improved Adaboost. J Med Syst, 2019, 43(3): 76-83.
7. Uysal G, Ozturk M. Hippocampal atrophy based Alzheimer’s disease diagnosis via machine learning methods. J Neurosci Methods, 2020, 337: 108669.
8. Jain R, Jain N, Aggarwal A, et al. Convolutional neural network based Alzheimer’s disease classification from magnetic resonance brain images. Cogn Syst Res, 2019, 57: 147-159.
9. Ebrahimi A, Luo S, Chiong R. Deep sequence modelling for Alzheimer’s disease detection using MRI. Comput Biol Med, 2021, 134: 104537.
10. Liu M, Li F, Yan H, et al. A multi-model deep convolutional neural network for automatic hippocampus segmentation and classification in Alzheimer’s disease. Neuroimage, 2020, 208: 116459.
11. Yu F, Zhao B Q, Ge Q Q, et al. A lightweight spatial attention module with adaptive receptive fields in 3D convolutional neural network for Alzheimer’s disease classification// International Conference on Pattern Recognition (ICPR), Chile: Universidad de Talca, 2021: 575-586.
12. Zhao Y, Dong Q, Zhang S, et al. Automatic recognition of fMRI-derived functional networks using 3D convolutional neural networks. IEEE Trans Biomed Eng, 2018, 65(9): 1975-1984.
13. Ramzan F, Khan M U G, Rehamt A, et al. A deep learning approach for automated diagnosis and multi-class classification of Alzheimer’s disease stages using resting-state fMRI and residual neural networks. J Med Syst, 2019, 44(2): 37.
14. Yu Q, Du Y, Chen J, et al. Application of graph theory to assess static and dynamic brain connectivity: approaches for building brain graphs. Proc IEEE, 2018, 106(5): 886-906.
15. Bessadok A, Mahjoub M A, Rekik I. Graph neural networks in network neuroscience. IEEE Trans Pattern Anal Mach Intell, 2022. DOI: 10.1109/TPAMI.2022.3209686.
16. Xu X, Li W, Tao M, et al. Effective and accurate diagnosis of subjective cognitive decline based on functional connection and graph theory view. Front Neurosci, 2020, 14: 577887.
17. Song X, Elazab A, Zhang Y. Classification of mild cognitive impairment based on a combined high-order network and graph convolutional network. IEEE Access, 2020, 8: 42816-42827.
18. Jiang H, Cao P, Xu M, et al. Hi-GCN: a hierarchical graph convolution network for graph embedding learning of brain network and brain disorders prediction. Comput Biol Med, 2020, 127: 104096.
19. Wang L J, Yuan W F, Zeng L, et al. Dementia analysis from functional connectivity network with graph neural networks. Inf Process Manage, 2022, 59(3): 102901.
20. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation// Medical Image Computing and Computer-Assisted Intervention (MICCAI), Munich: Springer, 2015: 234-241.
21. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas: IEEE, 2016: 770-778.
22. Hu J, Shen L, Albanie S, et al. Squeeze-and-Excitation networks. IEEE Trans Pattern Anal Mach Intell, 2020, 42(8): 2011-2023.
23. VM Eguíluz, Chialvo D R, Cecchi G A, et al. Scale-Free brain functional networks. Phys Rev Lett, 2005, 94(1): 018102.
24. Rubinov M, Sporns O. Complex network measures of brain connectivity: uses and interpretations. Neuroimage, 2010, 52(3): 1059-1069.
25. Kipf T N, Welling M. Semi-Supervised classification with graph convolutional networks. arXiv preprint, 2016, arXiv: 1609.02907.
26. Gao H Y, Ji S W. Graph U-Nets. IEEE Trans Pattern Anal Mach Intell, 2022, 44(9): 4948-4960.
27. Kennedy J, Eberhart R. Particle swarm optimization// IEEE International Conference on Neural Networks (ICNN), Perth: IEEE, 1995, 48: 1942-1948.
28. Kennedy J, Eberhart R C. A discrete binary version of the particle swarm algorithm// IEEE International Conference on Systems, Man, and Cybernetics, Orlando: IEEE, 1997.
29. Kasun L, Zhou H, Huang G B, et al. Representational learning with ELMs for big data. IEEE Intell Syst, 2013, 28(6): 31-34.
30. Hamilton W L, Ying R, Leskovec J. Inductive representation learning on large graphs// Proceedings of the 31st International Conference on Neural Information Processing Systems (NeurIPS), Long Beach: NeurIPS Foundation, 2017: 1025-1035.
31. Du J, Zhang S, Wu G, et al. Topology adaptive graph convolutional networks// Proceedings of the 5th International Conference on Learning Representations (ICLR), Toulon: Academic Press, 2017.
32. Morris C, Ritzert M, Fey M, et al. Weisfeiler and leman go neural: higher-order graph neural networks// Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI), Hawaii: AAAI, 2019: 4602-4609.

Journal of Biomedical Engineering

Research on classification method of multimodal magnetic resonance images of Alzheimer’s disease based on generalized convolutional neural networks

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content