A review of researches on decoding algorithms of steady-state visual evoked potentials_Journal of Biomedical Engineering

Authors：

YANG Man ¹ , JUNG Tzyy-Ping ^1,2,3 , HAN Jin ² ,  XU Minpeng ^1,2 , MING Dong ^1,2

1. Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, P. R. China;
2. School of Precision Instrument and Opto-electronics Engineering, Tianjin University, Tianjin 300072, P. R. China;
3. University of California and Swartz Center for Computational Neuroscience, California 92093, USA;

Corresponding author：

Keywords：

Brain-computer interface; Steady-state visual evoked potentials; Decoding algorithm; Pattern recognition; Feature extraction

DOI：

10.7507/1001-5515.202111066

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Brain-computer interface (BCI) systems based on steady-state visual evoked potential (SSVEP) have become one of the major paradigms in BCI research due to their high signal-to-noise ratio and short training time required by users. Fast and accurate decoding of SSVEP features is a crucial step in SSVEP-BCI research. However, the current researches lack a systematic overview of SSVEP decoding algorithms and analyses of the connections and differences between them, so it is difficult for researchers to choose the optimum algorithm under different situations. To address this problem, this paper focuses on the progress of SSVEP decoding algorithms in recent years and divides them into two categories—trained and non-trained—based on whether training data are needed. This paper also explains the fundamental theories and application scopes of decoding algorithms such as canonical correlation analysis (CCA), task-related component analysis (TRCA) and the extended algorithms, concludes the commonly used strategies for processing decoding algorithms, and discusses the challenges and opportunities in this field in the end.

Citation： YANG Man, JUNG Tzyy-Ping, HAN Jin, XU Minpeng, MING Dong. A review of researches on decoding algorithms of steady-state visual evoked potentials. Journal of Biomedical Engineering, 2022, 39(2): 416-425. doi: 10.7507/1001-5515.202111066 Copy

1.	Wolpaw J R, Wolpaw E W. Brain-Computer Interface: Principles and practice. Oxford: Oxford University Press, 2012: 3-12.
2.	Xu Minpeng, He Feng, Jung T P, et al. Current challenges for the practical application of electroencephalography-based Brain-Computer Interfaces. Engineering, 2021, 7(12): 1710-1712.
3.	Ge Sheng, Jiang Yichuan, Zhang Mingming, et al. SSVEP-based Brain-Computer Interface with a limited number of frequencies based on dual-frequency biased coding. IEEE Trans Neural Syst Rehabil Eng, 2021, 29: 760-769.
4.	Zhao Xi, Wang Zhenyu, Zhang Min, et al. A comfortable steady state visual evoked potential stimulation paradigm using peripheral vision. J Neural Eng, 2021, 18(5): 056021.
5.	Fouad I A, Labib F E M, Mai S M, et al. Improving the performance of P300 BCI system using different methods. Netw Model Anal Health Inform Bioinform, 2020, 9(1): 1-13.
6.	Jin Jing, Chen Zongmei, Xu Ren, et al. Developing a novel tactile P300 Brain-Computer Interface with a cheeks-stim paradigm. IEEE Trans Biomed Eng, 2020, 67(9): 2585-2593.
7.	Xu Minpeng, Han Jin, Wang Yijun, et al. Implementing over 100 command codes for a high-speed hybrid Brain-Computer Interface using concurrent P300 and SSVEP features. IEEE Trans Biomed Eng, 2020, 67(11): 3073-3082.
8.	Kapgate D, Kalbande D, Shrawankar U. An optimized facial stimuli paradigm for hybrid SSVEP + P300 Brain Computer Interface. Cogn Syst Res, 2020, 59(C): 114-122.
9.	Beveridge R, Wilson S, Callaghan M, et al. Neurogaming with motion-onset visual evoked potentials (mVEPs): adults versus teenagers. IEEE Trans Neural Syst Rehabil Eng, 2019, 27(4): 572-581.
10.	Lee M, Kwon O, Kim Y, et al. EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy. GigaScience, 2019, 8(5): 1-16.
11.	Li A, Alimanov K, Fazli S, et al. Towards paradigm-independent Brain-Computer Interfaces// 2020 8th International Winter Conference on Brain-Computer Interface (BCI). Gangwon: IEEE, 2020: 1-6.
12.	Van B, Van S, Alimardini M. Inner speech classification using EEG signals: A deep learning approach// 2021 IEEE 2nd International Conference on Human-Machine Systems (ICHMS). Magdeburg: IEEE, 2021: 1-4.
13.	Cheng Ming, Gao Xiaorong, Gao Shangkai, et al. Design and implementation of a Brain-Computer Interface with high transfer rates. IEEE Trans Biomed Eng, 2002, 49: 1181-1186.
14.	Volosyak I. SSVEP based bremen-BCI Interface-boosting information transfer rates. J Neural Eng, 2011, 8(3): 036020.
15.	Zhang Yangsong, Xu Peng, Cheng Kaiwen, et al. Multivariate synchronization index for frequency recognition of SSVEP-based Brain-Computer Interface. J Neurosci Methods, 2014, 221(Complete): 32-40.
16.	Bin Guangyu, Gao Xiaorong, Yan Zheng, et al. An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. J Neural Eng, 2009, 6(4): 046002.
17.	Wong C M, Wang Boyu, Wang Ze, et al. Spatial filtering in SSVEP-based BCIs: Unified framework and new improvements. IEEE Trans Biomed Eng, 2020, 67(11): 3057-3072.
18.	Yuan Peng, Chen Xiaogang, Wang Yijun, et al. Enhancing performances of SSVEP-based Brain-Computer Interfaces via exploiting inter-subject information. J Neural Eng, 2015, 12(4): 046006.
19.	Lao K F, Wong C M, Wang Ze, et al. Learning prototype spatial filters for subject-independent SSVEP-based Brain-Computer Interface// 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Miyazaki: IEEE, 2018: 485-490.
20.	Doganaksoy N. A simplified formulation of likelihood ratio confidence intervals using a novel property. Technimetrics, 2021, 63(1): 127-135.
21.	Wong C M, Wang Ze, Wang Boyu, et al. Inter- and intra-subject transfer reduces calibration effort for high-speed SSVEP-based BCIs. IEEE Trans Neural Syst Rehabil Eng, 2020, 28(10): 2123-2135.
22.	Wong C M, Wan Feng, Wang Boyu, et al. Learning across multi-stimulus enhances target recognition methods in SSVEP-based BCIs. J Neural Eng, 2020, 17(1): 016026.
23.	Zhang Li, Guo Liang, Gao Hongli, et al. Instance-based ensemble deep transfer learning network: A new intelligent degradation recognition method and its application on ball screw. Mech Syst Signal Process, 2020, 140: 106681.
24.	Bin Guangyu, Gao Xiaorong, Wang Yijun, et al. A high-speed BCI based on code modulation VEP. J Neural Eng, 2011, 8(2): 025015.
25.	Zhang Yu, Zhou Guoxu, Zhao Qibin, et al. Multiway canonical correlation analysis for frequency components recognition in SSVEP-based BCIs// 18th International Conference on Neural Information Processing. Shanghai: Springer, 2011: 287-295.
26.	Chen Xiaogang, Wang Yijun, Nakanishi M, et al. High-speed spelling with a noninvasive Brain-Computer Interface. PNAS, 2015, 112(44): E6058-6067.
27.	Nakanishi M, Wang Yijun, Chen Xiaogang, et al. Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis. IEEE Trans Biomed Eng, 2018, 65(1): 104-112.
28.	Tanaka H, Miyakoshi M. Cross-correlation task-related component analysis (xTRCA) for enhancing evoked and induced responses of event-related potentials. NeuroImage, 2019, 197: 177-190.
29.	Qin Ke, Wang Raofen, Zhang Yu. Filter bank-driven multivariate synchronization index for training-free SSVEP BCI. IEEE Trans Neural Syst Rehabil Eng, 2021, 29: 934-943.
30.	Wong C M, Wang Ze, Rosa A C, et al. Transferring subject-specific knowledge across stimulus frequencies in SSVEP-based BCIs. IEEE Trans Autom Sci Eng, 2021, 18(2): 552-563.
31.	Yang Chen, Li Xiang, Shi Nanlin, et al. A dynamic window SSVEP-based brain-computer interface system using a spatio-temporal equalizer// Guger C, Allison B Z, Miller K. Brain–computer interface research. SpringerBriefs in electrical and computer engineering. Cham: Springer, 2020: 87-105.
32.	Wang Yijun, Chen Xiaogang, Gao xiaorong, et al. A benchmark dataset for SSVEP-based Brain-Computer Interfaces. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(10): 1746-1752.
33.	Portnova V. Lack of a sense of threat and higher emotional lability in patients with chronic microvascular ischemia as measured by non-linear EEG parameters. Front Neurol, 2020, 11: 122.
34.	Alhalaseh R, Alhalaseh S. Machine-learning-based emotion recognition system using EEG signals. Computers, 2020, 9(4): 95.
35.	Zhang Ruilong, Zong Qun, Dou Liqian, et al. Hybrid deep neural network using transfer learning for EEG motor imagery decoding. Biomed Signal Process Control, 2021, 63: 102144.
36.	Monga V, Li Y, Eldar Y C. Algorithm unrolling: interpretable, efficient deep learning for signal and image processing. IEEE Signal Process Mag, 2021, 38(2): 18-44.
37.	Haque I, Neubert J. Deep learning approaches to biomedical image segmentation. Inform Med Unlocked, 2020, 18: 100297.
38.	Gonzalez G, Evans L. Biomedical image processing with containers and deep learning: an automated analysis pipeline: data architecture, artificial intelligence, automated processing, containerization, and clusters orchestration ease the transition from data acquisition to insights in medium-to-large datasets. BioEssays, 2019, 41(6): 1900004.
39.	Azimi M, Eslamlou D, Pekcan G. Data-driven structural health monitoring and damage detection through deep learning: state-of-the-art review. Sensors, 2020, 20(10): 2778.
40.	Roberts R, Giancontieri G, Inzerillo L, et al. Towards low-cost pavement condition health monitoring and analysis using deep learning. Appl Sci, 2020, 10(1): 319.
41.	Ye X W, Jin T, Yun C B. A review on deep learning-based structural health monitoring of civil infrastructures. Smart Struct Syst, 2019, 24(5): 567-586.
42.	Piccialli F, Di Somma V, Giampaolo F, et al. A survey on deep learning in medicine: why, how and when? Information Fusion, 2021, 66: 111-137.
43.	Khalifa M, Taha N, Ali E, et al. Artificial intelligence technique for gene expression by tumor RNA-Seq data: a novel optimized deep learning approach. IEEE Access, 2020, 8: 22874-22883.
44.	Seal B, Das V, Goswami S, et al. Estimating gene expression from DNA methylation and copy number variation: a deep learning regression model for multi-omics integration. Genomics, 2020, 112(4): 2833-2841.
45.	Oh L, Hagiwara Y, Raghavendra U, et al. A deep learning approach for Parkinson’s disease diagnosis from EEG signals. Neural Comput Appl, 2020, 32(15): 10927-10933.
46.	Lih S, Jahmunah V, San R, et al. Comprehensive electrocardiographic diagnosis based on deep learning. Artif Intell Med, 2020, 103: 101789.
47.	Huang Dongjin, Wang Xianglong, Liu Jinhua, et al. Virtual reality safety training using deep EEG-net and physiology data. Vis Comput, 2021: 1-13.
48.	Zhu Yuanlu, Li Ying, Lu Jinling, et al. EEGNet with ensemble learning to improve the cross-session classification of SSVEP based BCI from ear-EEG. IEEE Access, 2021, 9: 15295-15303.
49.	Xu Lichao, Xu Minpeng, Ke Yufeng, et al. Cross-dataset variability problem in EEG decoding with deep learning. Front Hum Neurosci, 2020, 14: 103.
50.	Chiang K J, Wei C S, Nakanishi M, et al. Boosting template-based SSVEP decoding by cross-domain transfer learning. J Neural Eng, 2021, 18(1): 016002.

1. Wolpaw J R, Wolpaw E W. Brain-Computer Interface: Principles and practice. Oxford: Oxford University Press, 2012: 3-12.
2. Xu Minpeng, He Feng, Jung T P, et al. Current challenges for the practical application of electroencephalography-based Brain-Computer Interfaces. Engineering, 2021, 7(12): 1710-1712.
3. Ge Sheng, Jiang Yichuan, Zhang Mingming, et al. SSVEP-based Brain-Computer Interface with a limited number of frequencies based on dual-frequency biased coding. IEEE Trans Neural Syst Rehabil Eng, 2021, 29: 760-769.
4. Zhao Xi, Wang Zhenyu, Zhang Min, et al. A comfortable steady state visual evoked potential stimulation paradigm using peripheral vision. J Neural Eng, 2021, 18(5): 056021.
5. Fouad I A, Labib F E M, Mai S M, et al. Improving the performance of P300 BCI system using different methods. Netw Model Anal Health Inform Bioinform, 2020, 9(1): 1-13.
6. Jin Jing, Chen Zongmei, Xu Ren, et al. Developing a novel tactile P300 Brain-Computer Interface with a cheeks-stim paradigm. IEEE Trans Biomed Eng, 2020, 67(9): 2585-2593.
7. Xu Minpeng, Han Jin, Wang Yijun, et al. Implementing over 100 command codes for a high-speed hybrid Brain-Computer Interface using concurrent P300 and SSVEP features. IEEE Trans Biomed Eng, 2020, 67(11): 3073-3082.
8. Kapgate D, Kalbande D, Shrawankar U. An optimized facial stimuli paradigm for hybrid SSVEP + P300 Brain Computer Interface. Cogn Syst Res, 2020, 59(C): 114-122.
9. Beveridge R, Wilson S, Callaghan M, et al. Neurogaming with motion-onset visual evoked potentials (mVEPs): adults versus teenagers. IEEE Trans Neural Syst Rehabil Eng, 2019, 27(4): 572-581.
10. Lee M, Kwon O, Kim Y, et al. EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy. GigaScience, 2019, 8(5): 1-16.
11. Li A, Alimanov K, Fazli S, et al. Towards paradigm-independent Brain-Computer Interfaces// 2020 8th International Winter Conference on Brain-Computer Interface (BCI). Gangwon: IEEE, 2020: 1-6.
12. Van B, Van S, Alimardini M. Inner speech classification using EEG signals: A deep learning approach// 2021 IEEE 2nd International Conference on Human-Machine Systems (ICHMS). Magdeburg: IEEE, 2021: 1-4.
13. Cheng Ming, Gao Xiaorong, Gao Shangkai, et al. Design and implementation of a Brain-Computer Interface with high transfer rates. IEEE Trans Biomed Eng, 2002, 49: 1181-1186.
14. Volosyak I. SSVEP based bremen-BCI Interface-boosting information transfer rates. J Neural Eng, 2011, 8(3): 036020.
15. Zhang Yangsong, Xu Peng, Cheng Kaiwen, et al. Multivariate synchronization index for frequency recognition of SSVEP-based Brain-Computer Interface. J Neurosci Methods, 2014, 221(Complete): 32-40.
16. Bin Guangyu, Gao Xiaorong, Yan Zheng, et al. An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. J Neural Eng, 2009, 6(4): 046002.
17. Wong C M, Wang Boyu, Wang Ze, et al. Spatial filtering in SSVEP-based BCIs: Unified framework and new improvements. IEEE Trans Biomed Eng, 2020, 67(11): 3057-3072.
18. Yuan Peng, Chen Xiaogang, Wang Yijun, et al. Enhancing performances of SSVEP-based Brain-Computer Interfaces via exploiting inter-subject information. J Neural Eng, 2015, 12(4): 046006.
19. Lao K F, Wong C M, Wang Ze, et al. Learning prototype spatial filters for subject-independent SSVEP-based Brain-Computer Interface// 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Miyazaki: IEEE, 2018: 485-490.
20. Doganaksoy N. A simplified formulation of likelihood ratio confidence intervals using a novel property. Technimetrics, 2021, 63(1): 127-135.
21. Wong C M, Wang Ze, Wang Boyu, et al. Inter- and intra-subject transfer reduces calibration effort for high-speed SSVEP-based BCIs. IEEE Trans Neural Syst Rehabil Eng, 2020, 28(10): 2123-2135.
22. Wong C M, Wan Feng, Wang Boyu, et al. Learning across multi-stimulus enhances target recognition methods in SSVEP-based BCIs. J Neural Eng, 2020, 17(1): 016026.
23. Zhang Li, Guo Liang, Gao Hongli, et al. Instance-based ensemble deep transfer learning network: A new intelligent degradation recognition method and its application on ball screw. Mech Syst Signal Process, 2020, 140: 106681.
24. Bin Guangyu, Gao Xiaorong, Wang Yijun, et al. A high-speed BCI based on code modulation VEP. J Neural Eng, 2011, 8(2): 025015.
25. Zhang Yu, Zhou Guoxu, Zhao Qibin, et al. Multiway canonical correlation analysis for frequency components recognition in SSVEP-based BCIs// 18th International Conference on Neural Information Processing. Shanghai: Springer, 2011: 287-295.
26. Chen Xiaogang, Wang Yijun, Nakanishi M, et al. High-speed spelling with a noninvasive Brain-Computer Interface. PNAS, 2015, 112(44): E6058-6067.
27. Nakanishi M, Wang Yijun, Chen Xiaogang, et al. Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis. IEEE Trans Biomed Eng, 2018, 65(1): 104-112.
28. Tanaka H, Miyakoshi M. Cross-correlation task-related component analysis (xTRCA) for enhancing evoked and induced responses of event-related potentials. NeuroImage, 2019, 197: 177-190.
29. Qin Ke, Wang Raofen, Zhang Yu. Filter bank-driven multivariate synchronization index for training-free SSVEP BCI. IEEE Trans Neural Syst Rehabil Eng, 2021, 29: 934-943.
30. Wong C M, Wang Ze, Rosa A C, et al. Transferring subject-specific knowledge across stimulus frequencies in SSVEP-based BCIs. IEEE Trans Autom Sci Eng, 2021, 18(2): 552-563.
31. Yang Chen, Li Xiang, Shi Nanlin, et al. A dynamic window SSVEP-based brain-computer interface system using a spatio-temporal equalizer// Guger C, Allison B Z, Miller K. Brain–computer interface research. SpringerBriefs in electrical and computer engineering. Cham: Springer, 2020: 87-105.
32. Wang Yijun, Chen Xiaogang, Gao xiaorong, et al. A benchmark dataset for SSVEP-based Brain-Computer Interfaces. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(10): 1746-1752.
33. Portnova V. Lack of a sense of threat and higher emotional lability in patients with chronic microvascular ischemia as measured by non-linear EEG parameters. Front Neurol, 2020, 11: 122.
34. Alhalaseh R, Alhalaseh S. Machine-learning-based emotion recognition system using EEG signals. Computers, 2020, 9(4): 95.
35. Zhang Ruilong, Zong Qun, Dou Liqian, et al. Hybrid deep neural network using transfer learning for EEG motor imagery decoding. Biomed Signal Process Control, 2021, 63: 102144.
36. Monga V, Li Y, Eldar Y C. Algorithm unrolling: interpretable, efficient deep learning for signal and image processing. IEEE Signal Process Mag, 2021, 38(2): 18-44.
37. Haque I, Neubert J. Deep learning approaches to biomedical image segmentation. Inform Med Unlocked, 2020, 18: 100297.
38. Gonzalez G, Evans L. Biomedical image processing with containers and deep learning: an automated analysis pipeline: data architecture, artificial intelligence, automated processing, containerization, and clusters orchestration ease the transition from data acquisition to insights in medium-to-large datasets. BioEssays, 2019, 41(6): 1900004.
39. Azimi M, Eslamlou D, Pekcan G. Data-driven structural health monitoring and damage detection through deep learning: state-of-the-art review. Sensors, 2020, 20(10): 2778.
40. Roberts R, Giancontieri G, Inzerillo L, et al. Towards low-cost pavement condition health monitoring and analysis using deep learning. Appl Sci, 2020, 10(1): 319.
41. Ye X W, Jin T, Yun C B. A review on deep learning-based structural health monitoring of civil infrastructures. Smart Struct Syst, 2019, 24(5): 567-586.
42. Piccialli F, Di Somma V, Giampaolo F, et al. A survey on deep learning in medicine: why, how and when? Information Fusion, 2021, 66: 111-137.
43. Khalifa M, Taha N, Ali E, et al. Artificial intelligence technique for gene expression by tumor RNA-Seq data: a novel optimized deep learning approach. IEEE Access, 2020, 8: 22874-22883.
44. Seal B, Das V, Goswami S, et al. Estimating gene expression from DNA methylation and copy number variation: a deep learning regression model for multi-omics integration. Genomics, 2020, 112(4): 2833-2841.
45. Oh L, Hagiwara Y, Raghavendra U, et al. A deep learning approach for Parkinson’s disease diagnosis from EEG signals. Neural Comput Appl, 2020, 32(15): 10927-10933.
46. Lih S, Jahmunah V, San R, et al. Comprehensive electrocardiographic diagnosis based on deep learning. Artif Intell Med, 2020, 103: 101789.
47. Huang Dongjin, Wang Xianglong, Liu Jinhua, et al. Virtual reality safety training using deep EEG-net and physiology data. Vis Comput, 2021: 1-13.
48. Zhu Yuanlu, Li Ying, Lu Jinling, et al. EEGNet with ensemble learning to improve the cross-session classification of SSVEP based BCI from ear-EEG. IEEE Access, 2021, 9: 15295-15303.
49. Xu Lichao, Xu Minpeng, Ke Yufeng, et al. Cross-dataset variability problem in EEG decoding with deep learning. Front Hum Neurosci, 2020, 14: 103.
50. Chiang K J, Wei C S, Nakanishi M, et al. Boosting template-based SSVEP decoding by cross-domain transfer learning. J Neural Eng, 2021, 18(1): 016002.

Journal of Biomedical Engineering

A review of researches on decoding algorithms of steady-state visual evoked potentials

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content