A spatial-temporal hybrid feature extraction method for rapid serial visual presentation of electroencephalogram signals_Journal of Biomedical Engineering

Authors：

CUI Yujie ,  XIE Songyun ,  XIE Xinzhou , DUAN Xu , GAO Chuanlin

NPU-TUB Joint Laboratory for Neural informatics, School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710129, P. R. China;

Corresponding author：

XIE Songyun, Email: syxie@nwpu.edu.cn; XIE Xinzhou, Email: xinzhxie@nwpu.edu.cn

Keywords：

Brain-computer interface; Rapid serial visual presentation; Spatial-temporal hybrid; Event related potential; Target detection

DOI：

10.7507/1001-5515.202104049

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Rapid serial visual presentation-brain computer interface (RSVP-BCI) is the most popular technology in the early discover task based on human brain. This algorithm can obtain the rapid perception of the environment by human brain. Decoding brain state based on single-trial of multichannel electroencephalogram (EEG) recording remains a challenge due to the low signal-to-noise ratio (SNR) and nonstationary. To solve the problem of low classification accuracy of single-trial in RSVP-BCI, this paper presents a new feature extraction algorithm which uses principal component analysis (PCA) and common spatial pattern (CSP) algorithm separately in spatial domain and time domain, creating a spatial-temporal hybrid CSP-PCA (STHCP) algorithm. By maximizing the discrimination distance between target and non-target, the feature dimensionality was reduced effectively. The area under the curve (AUC) of STHCP algorithm is higher than that of the three benchmark algorithms (SWFP, CSP and PCA) by 17.9%, 22.2% and 29.2%, respectively. STHCP algorithm provides a new method for target detection.

Citation： CUI Yujie, XIE Songyun, XIE Xinzhou, DUAN Xu, GAO Chuanlin. A spatial-temporal hybrid feature extraction method for rapid serial visual presentation of electroencephalogram signals. Journal of Biomedical Engineering, 2022, 39(1): 39-46. doi: 10.7507/1001-5515.202104049 Copy

1.	Gerson A D, Parra L C, Sajda P. Cortically coupled computer vision for rapid image search. IEEE Trans Neural Syst Rehabil Eng, 2006, 14(2): 174-179.
2.	Poolman P, Frank R M, Luu P, et al. A single-trial analytic framework for EEG analysis and its application to target detection and classification. Neuroimage, 2008, 42(2): 787-798.
3.	Lees S, Dayan N, Cecotti H, et al. A review of rapid serial visual presentation-based brain-computer interfaces. J Neural Eng, 2018, 15: 021001.
4.	Chen J, Mao Z, Zheng R, et al. Feature selection of deep learning models for EEG-based RSVP target detection. IEICE Transactions on Information and Systems, 2019, 102-D(4): 836-844.
5.	Potter M C, Fox L F. Detecting and remembering simultaneous pictures in a rapid serial visual presentation. J Exp Psychol Hum Percept Perform, 2009, 35(1): 28-38.
6.	Ichikawa M, Miyoshi M. Perceived duration depends upon target detection in rapid serial visual presentation sequence. i-Perception, 2020, 11(6): 1-18.
7.	Marx S, Hansen-Goos O, Thrun M, et al. Rapid serial processing of natural scenes: color modulates detection but neither recognition nor the attentional blink. J Vis, 2014, 14(14): 4.
8.	赵仑. ERPs实验教程. 南京: 东南大学出版社, 2010: 70-73.
9.	Mantini D, Corbetta M, Perrucci M G, et al. Large-scale brain networks account for sustained and transient activity during target detection. Neuroimage, 2009, 44(1): 265-274.
10.	Kopp B, Steinke A, Visalli A. Cognitive flexibility and N2/P3 event-related brain potentials. Sci Rep UK, 2020, 10(1): 9859.
11.	Cecotti H, Eckstein M P, Giesbrecht B. Single-trial classification of event-related potentials in rapid serial visual presentation tasks using supervised spatial filtering. IEEE Trans Neural Netw Learn Syst, 2014, 25(11): 2030-2042.
12.	Rizi F S, Abootalebi V, Sadeghi M T. Spatial and spatio-temporal filtering based on common spatial patterns and Max-SNR for detection of P300 component. Biocybern Biomed Eng, 2017, 37(3): 365-372.
13.	Zhang Y, Zhou G, Zhao Q, et al. Spatial-temporal discriminant analysis for ERP-based brain-computer interface. IEEE Trans Neural Syst Rehabil Eng, 2013, 21(2): 233-243.
14.	Cecotti H. Single-trial detection with magnetoencephalography during a dual-rapid serial visual presentation task. IEEE Trans Biomed Eng, 2016, 63(1): 220-227.
15.	Parra L C, Christoforou C, Gerson A D, et al. Spatiotemporal linear decoding of brain state. IEEE Signal Proc Mag, 2008, 25(1): 107-115.
16.	Marathe A R, Ries A J, Mcdowell K. Sliding HDCA: single-trial EEG classification to overcome and quantify temporal variability. IEEE Trans Neural Syst Rehabil Eng, 2014, 22(2): 201-211.
17.	Fuhrmann A G, Manor R, Spanier A B, et al. Spatiotemporal representations of rapid visual target detection: a single-trial EEG classification algorithm. IEEE Trans Biomed Eng, 2014, 61(8): 2290-2303.
18.	Blankertz B, Tomioka R, Lemm S, et al. Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Processing Magazine, 2007, 25(1): 41-56.
19.	Ghani U, Signal N, Niazi I K, et al. ERP based measures of cognitive workload: a review. Neurosci Biobehav Rev, 2020, 118: 18-26.
20.	Acqualagna L, Blankertz B. Gaze-independent BCI-spelling using rapid serial visual presentation (RSVP). Clin Neurophysiol, 2013, 124(5): 901-908.
21.	Shih S. The attention cascade model and attentional blink. Cogn Psychol, 2008, 56(3): 210-236.
22.	Petrucci M, Pecchinenda A. Sparing and impairing: emotion modulation of the attentional blink and the spread of sparing in a 3-target RSVP task. Atten Percept Psycho, 2018, 80(2): 439-452.
23.	赖虹宇, 冯静雯, 王毅, 等. 抑郁症和精神分裂症患者静息态脑电信号的分类研究. 生物医学工程学杂志, 2019, 36(6): 916-923.
24.	Ramoser H, Müller-Gerking J, Pfurtscheller G. Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans Rehabil Eng, 2000, 8(4): 441-446.
25.	Müller-Gerking J, Pfurtscheller G, Flyvbjerg H, et al. Designing optimal spatial filters for single-trial EEG classification in a movement task. Clinical Neurophysiology, 1999, 110(5): 787-798.
26.	Lin Z, Zhang C, Zeng Y, et al. A novel P300 BCI speller based on the triple RSVP paradigm. Sci Rep UK, 2018, 8: 3350.
27.	Deng Xinyang, Liu Qi, Deng Yong, et al. An improved method to construct basic probability assignment based on the confusion matrix for classification problem. Inform Sciences, 2016, 340-341: 250-261.
28.	李亚兵, 谢松云, 于圳宁, 等. 基于动态偏定向相干的运动想象因效性网络分析. 生物医学工程学杂志, 2020, 37(1): 38-44.
29.	Wang X, Zeng Y, Lin Z, et al. Combining multiple ERP components for detecting targets in remote-sensing images. Intelligent Human-Machine Systems and Cybernetics (IHMSC), 2017, 2017: 167-170.
30.	Liu Shuang, Wang Wei, Sheng Yue, et al. Improving the cross-subject performance of the ERP-based brain-computer interface using rapid serial visual presentation and correlation analysis rank. Front Hum Neurosci, 2020, 14: 296.
31.	Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods, 2004, 134(1): 9-21.

1. Gerson A D, Parra L C, Sajda P. Cortically coupled computer vision for rapid image search. IEEE Trans Neural Syst Rehabil Eng, 2006, 14(2): 174-179.
2. Poolman P, Frank R M, Luu P, et al. A single-trial analytic framework for EEG analysis and its application to target detection and classification. Neuroimage, 2008, 42(2): 787-798.
3. Lees S, Dayan N, Cecotti H, et al. A review of rapid serial visual presentation-based brain-computer interfaces. J Neural Eng, 2018, 15: 021001.
4. Chen J, Mao Z, Zheng R, et al. Feature selection of deep learning models for EEG-based RSVP target detection. IEICE Transactions on Information and Systems, 2019, 102-D(4): 836-844.
5. Potter M C, Fox L F. Detecting and remembering simultaneous pictures in a rapid serial visual presentation. J Exp Psychol Hum Percept Perform, 2009, 35(1): 28-38.
6. Ichikawa M, Miyoshi M. Perceived duration depends upon target detection in rapid serial visual presentation sequence. i-Perception, 2020, 11(6): 1-18.
7. Marx S, Hansen-Goos O, Thrun M, et al. Rapid serial processing of natural scenes: color modulates detection but neither recognition nor the attentional blink. J Vis, 2014, 14(14): 4.
8. 赵仑. ERPs实验教程. 南京: 东南大学出版社, 2010: 70-73.
9. Mantini D, Corbetta M, Perrucci M G, et al. Large-scale brain networks account for sustained and transient activity during target detection. Neuroimage, 2009, 44(1): 265-274.
10. Kopp B, Steinke A, Visalli A. Cognitive flexibility and N2/P3 event-related brain potentials. Sci Rep UK, 2020, 10(1): 9859.
11. Cecotti H, Eckstein M P, Giesbrecht B. Single-trial classification of event-related potentials in rapid serial visual presentation tasks using supervised spatial filtering. IEEE Trans Neural Netw Learn Syst, 2014, 25(11): 2030-2042.
12. Rizi F S, Abootalebi V, Sadeghi M T. Spatial and spatio-temporal filtering based on common spatial patterns and Max-SNR for detection of P300 component. Biocybern Biomed Eng, 2017, 37(3): 365-372.
13. Zhang Y, Zhou G, Zhao Q, et al. Spatial-temporal discriminant analysis for ERP-based brain-computer interface. IEEE Trans Neural Syst Rehabil Eng, 2013, 21(2): 233-243.
14. Cecotti H. Single-trial detection with magnetoencephalography during a dual-rapid serial visual presentation task. IEEE Trans Biomed Eng, 2016, 63(1): 220-227.
15. Parra L C, Christoforou C, Gerson A D, et al. Spatiotemporal linear decoding of brain state. IEEE Signal Proc Mag, 2008, 25(1): 107-115.
16. Marathe A R, Ries A J, Mcdowell K. Sliding HDCA: single-trial EEG classification to overcome and quantify temporal variability. IEEE Trans Neural Syst Rehabil Eng, 2014, 22(2): 201-211.
17. Fuhrmann A G, Manor R, Spanier A B, et al. Spatiotemporal representations of rapid visual target detection: a single-trial EEG classification algorithm. IEEE Trans Biomed Eng, 2014, 61(8): 2290-2303.
18. Blankertz B, Tomioka R, Lemm S, et al. Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Processing Magazine, 2007, 25(1): 41-56.
19. Ghani U, Signal N, Niazi I K, et al. ERP based measures of cognitive workload: a review. Neurosci Biobehav Rev, 2020, 118: 18-26.
20. Acqualagna L, Blankertz B. Gaze-independent BCI-spelling using rapid serial visual presentation (RSVP). Clin Neurophysiol, 2013, 124(5): 901-908.
21. Shih S. The attention cascade model and attentional blink. Cogn Psychol, 2008, 56(3): 210-236.
22. Petrucci M, Pecchinenda A. Sparing and impairing: emotion modulation of the attentional blink and the spread of sparing in a 3-target RSVP task. Atten Percept Psycho, 2018, 80(2): 439-452.
23. 赖虹宇, 冯静雯, 王毅, 等. 抑郁症和精神分裂症患者静息态脑电信号的分类研究. 生物医学工程学杂志, 2019, 36(6): 916-923.
24. Ramoser H, Müller-Gerking J, Pfurtscheller G. Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans Rehabil Eng, 2000, 8(4): 441-446.
25. Müller-Gerking J, Pfurtscheller G, Flyvbjerg H, et al. Designing optimal spatial filters for single-trial EEG classification in a movement task. Clinical Neurophysiology, 1999, 110(5): 787-798.
26. Lin Z, Zhang C, Zeng Y, et al. A novel P300 BCI speller based on the triple RSVP paradigm. Sci Rep UK, 2018, 8: 3350.
27. Deng Xinyang, Liu Qi, Deng Yong, et al. An improved method to construct basic probability assignment based on the confusion matrix for classification problem. Inform Sciences, 2016, 340-341: 250-261.
28. 李亚兵, 谢松云, 于圳宁, 等. 基于动态偏定向相干的运动想象因效性网络分析. 生物医学工程学杂志, 2020, 37(1): 38-44.
29. Wang X, Zeng Y, Lin Z, et al. Combining multiple ERP components for detecting targets in remote-sensing images. Intelligent Human-Machine Systems and Cybernetics (IHMSC), 2017, 2017: 167-170.
30. Liu Shuang, Wang Wei, Sheng Yue, et al. Improving the cross-subject performance of the ERP-based brain-computer interface using rapid serial visual presentation and correlation analysis rank. Front Hum Neurosci, 2020, 14: 296.
31. Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods, 2004, 134(1): 9-21.

Journal of Biomedical Engineering

A spatial-temporal hybrid feature extraction method for rapid serial visual presentation of electroencephalogram signals

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content