Fatigue feature extraction and classification algorithm of forehead single-channel electroencephalography signals_Journal of Biomedical Engineering

Authors：

YANG Huizhou ¹ ,  LIU Yunfei ¹ , XIA Lijuan ²

1. College of Information Science and Technology & College of Artificial Intelligence, Nanjing Forestry University, Nanjing 210037, P. R. China;
2. CR/RIX1-AP, Bosch (China) Investment Ltd., Shanghai 200335, P. R. China;

Corresponding author：

LIU Yunfei, Email: lyf@njfu.edu.cn

Keywords：

Single-channel electroencephalography signal; Driving fatigue detection; Feature extraction; Supervised contrastive learning

DOI：

10.7507/1001-5515.202312026

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Aiming at the problem that the feature extraction ability of forehead single-channel electroencephalography (EEG) signals is insufficient, which leads to decreased fatigue detection accuracy, a fatigue feature extraction and classification algorithm based on supervised contrastive learning is proposed. Firstly, the raw signals are filtered by empirical modal decomposition to improve the signal-to-noise ratio. Secondly, considering the limitation of the one-dimensional signal in information expression, overlapping sampling is used to transform the signal into a two-dimensional structure, and simultaneously express the short-term and long-term changes of the signal. The feature extraction network is constructed by depthwise separable convolution to accelerate model operation. Finally, the model is globally optimized by combining the supervised contrastive loss and the mean square error loss. Experiments show that the average accuracy of the algorithm for classifying three fatigue states can reach 75.80%, which is greatly improved compared with other advanced algorithms, and the accuracy and feasibility of fatigue detection by single-channel EEG signals are significantly improved. The results provide strong support for the application of single-channel EEG signals, and also provide a new idea for fatigue detection research.

Citation： YANG Huizhou, LIU Yunfei, XIA Lijuan. Fatigue feature extraction and classification algorithm of forehead single-channel electroencephalography signals. Journal of Biomedical Engineering, 2024, 41(4): 732-741. doi: 10.7507/1001-5515.202312026 Copy

1.	Sikander G, Anwar S. Driver fatigue detection systems: A review. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(6): 2339-2352.
2.	Deng S, Yu H, Lu C. Research on operation characteristics and safety risk forecast of bus driven by multisource forewarning data. Journal of Advanced Transportation, 2020, 2020: 6623739.
3.	Riera L, Ozcan K, Merickel J, et al. Detecting and tracking unsafe lane departure events for predicting driver safety in challenging naturalistic driving data//2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas: IEEE, 2020: 238-245.
4.	Jeon Y, Kim B, Baek Y. Ensemble CNN to detect drowsy driving with in-vehicle sensor data. Sensors, 2021, 21(7): 2372.
5.	Jia Huijie, Xiao Zhongjun, Ji Peng. Fatigue driving detection based on deep learning and multi-index fusion. IEEE Access, 2021, 9: 147054-147062.
6.	Yang H, Liu L, Min W D, et al. Driver yawning detection based on subtle facial action recognition. IEEE Transactions on Multimedia, 2021, 23: 572-583.
7.	Huang R, Wang Y, Li Z J, et al. RF-DCM: multi-granularity deep convolutional model based on feature recalibration and fusion for driver fatigue detection. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(1): 630-640.
8.	张鸿. 基于驾驶员生理信号的疲劳驾驶检测及其预警方法研究. 长春: 吉林大学, 2022.
9.	Pei Z, Wang Hongtao, Bezerianos A, et al. EEG-based multiclass workload identification using feature fusion and selection. IEEE Transactions on Instrumentation and Measurement, 2020, 70: 4001108.
10.	Shi L C, Jiao Y Y, Lu B L. Differential entropy feature for EEG-based vigilance estimation//2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka: IEEE, 2013: 6627-6630.
11.	Wu W, Sun W, Wu Q, et al. Multimodal vigilance estimation using deep learning. IEEE Transactions on Cybernetics, 2020, 52(5): 3097-3110.
12.	Lawhern V J, Solon A J, Waytowich N R, et al. EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. Journal of Neural Engineering, 2018, 15(5): 056013.
13.	Ko W, Oh K, Jeon E, et al. VIGNET: a deep convolutional neural network for eeg-based driver vigilance estimation//2020 8th International Winter Conference on Brain-Computer Interface (BCI). Gangwon: IEEE, 2020: 1-3.
14.	Shi Jinxuan, Wang Kun. Fatigue driving detection method based on time-space-frequency features of multimodal signals. Biomedical Signal Processing and Control, 2023, 84: 104744.
15.	Zhang G, Etemad A. Capsule attention for multimodal EEG-EOG representation learning with application to driver vigilance estimation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29: 1138-1149.
16.	Mohib S W. Driver fatigue detection with single EEG channel using transfer learning//2019 IEEE International Conference on Imaging Systems and Techniques (IST), Abu Dhabi: IEEE, 2019: 1-6.
17.	崔兴然, 秦泽光, 高之琳, 等. 可穿戴脑电信号在抑郁症识别和个性化音乐干预中的应用与挑战. 生物医学工程学杂志, 2023, 40(6): 1093-1101.
18.	Ratti E, Waninger S, Berka C, et al. Comparison of medical and consumer wireless EEG systems for use in clinical trials. Frontiers in Human Neuroscience, 2017, 11: 398.
19.	张锐, 刘家俊, 陈明明, 等. 基于小波变换—集合经验模态分解的单通道脑电信号眼电伪迹自动去除研究. 生物医学工程学杂志, 2021, 38(3): 473-482.
20.	徐曹军. 单通道脑电信号的分析与应用研究. 南京: 南京邮电大学, 2022.
21.	Shahbakhti M, Beiramvand M, Nasiri E, et al. Fusion of EEG and eye blink analysis for detection of driver fatigue. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023, 31: 2037-2046.
22.	Arif S, Munawar S, Ali H. Driving drowsiness detection using spectral signatures of EEG-based neurophysiology. Frontiers in Physiology, 2023, 14: 1153268.
23.	Hadra M, Abdelrahman I. Automatic EEG-based alertness classification using sparse representation and dictionary learning. Journal of Biomedical Engineering and Medical Imaging, 2020, 7(5): 19-28.
24.	Huang N E, Shen Z, Long S R, et al. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proceedings Mathematical Physical and Engineering Sciences, 1998, 454(1971): 903-995.
25.	Guennec A L, Malinowski S, Tavenard R. Data augmentation for time series classification using convolutional neural networks//ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data, Riva Del Garda, Italy, 2016: 01357973.
26.	Wu Haixu, Hu Tengge, Liu Yong, et al. Timesnet: temporal 2D-variation modeling for general time series analysis. arXiv preprint, 2022, arXiv: 2210.02186.
27.	Howard A, Sandler M, Chu G, et al. Searching for mobilenetv3//2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul: IEEE, 2019: 1314-1324.
28.	Khosla P, Teterwak P, Wang C, et al. Supervised contrastive learning. Neural Information Processing Systems, 2020, 33: 18661-18673.
29.	Zheng W L, Lu B L. A multimodal approach to estimating vigilance using EEG and forehead EOG. Journal of Neural Engineering, 2017, 14(2): 026017.
30.	Ko L W, Komarov O, Lai W K, et al. Eyeblink recognition improves fatigue prediction from single-channel forehead EEG in a realistic sustained attention task. Journal of Neural Engineering, 2020, 17(3): 036015.
31.	Maaten L, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9(86): 2579-2605.

1. Sikander G, Anwar S. Driver fatigue detection systems: A review. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(6): 2339-2352.
2. Deng S, Yu H, Lu C. Research on operation characteristics and safety risk forecast of bus driven by multisource forewarning data. Journal of Advanced Transportation, 2020, 2020: 6623739.
3. Riera L, Ozcan K, Merickel J, et al. Detecting and tracking unsafe lane departure events for predicting driver safety in challenging naturalistic driving data//2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas: IEEE, 2020: 238-245.
4. Jeon Y, Kim B, Baek Y. Ensemble CNN to detect drowsy driving with in-vehicle sensor data. Sensors, 2021, 21(7): 2372.
5. Jia Huijie, Xiao Zhongjun, Ji Peng. Fatigue driving detection based on deep learning and multi-index fusion. IEEE Access, 2021, 9: 147054-147062.
6. Yang H, Liu L, Min W D, et al. Driver yawning detection based on subtle facial action recognition. IEEE Transactions on Multimedia, 2021, 23: 572-583.
7. Huang R, Wang Y, Li Z J, et al. RF-DCM: multi-granularity deep convolutional model based on feature recalibration and fusion for driver fatigue detection. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(1): 630-640.
8. 张鸿. 基于驾驶员生理信号的疲劳驾驶检测及其预警方法研究. 长春: 吉林大学, 2022.
9. Pei Z, Wang Hongtao, Bezerianos A, et al. EEG-based multiclass workload identification using feature fusion and selection. IEEE Transactions on Instrumentation and Measurement, 2020, 70: 4001108.
10. Shi L C, Jiao Y Y, Lu B L. Differential entropy feature for EEG-based vigilance estimation//2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka: IEEE, 2013: 6627-6630.
11. Wu W, Sun W, Wu Q, et al. Multimodal vigilance estimation using deep learning. IEEE Transactions on Cybernetics, 2020, 52(5): 3097-3110.
12. Lawhern V J, Solon A J, Waytowich N R, et al. EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. Journal of Neural Engineering, 2018, 15(5): 056013.
13. Ko W, Oh K, Jeon E, et al. VIGNET: a deep convolutional neural network for eeg-based driver vigilance estimation//2020 8th International Winter Conference on Brain-Computer Interface (BCI). Gangwon: IEEE, 2020: 1-3.
14. Shi Jinxuan, Wang Kun. Fatigue driving detection method based on time-space-frequency features of multimodal signals. Biomedical Signal Processing and Control, 2023, 84: 104744.
15. Zhang G, Etemad A. Capsule attention for multimodal EEG-EOG representation learning with application to driver vigilance estimation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29: 1138-1149.
16. Mohib S W. Driver fatigue detection with single EEG channel using transfer learning//2019 IEEE International Conference on Imaging Systems and Techniques (IST), Abu Dhabi: IEEE, 2019: 1-6.
17. 崔兴然, 秦泽光, 高之琳, 等. 可穿戴脑电信号在抑郁症识别和个性化音乐干预中的应用与挑战. 生物医学工程学杂志, 2023, 40(6): 1093-1101.
18. Ratti E, Waninger S, Berka C, et al. Comparison of medical and consumer wireless EEG systems for use in clinical trials. Frontiers in Human Neuroscience, 2017, 11: 398.
19. 张锐, 刘家俊, 陈明明, 等. 基于小波变换—集合经验模态分解的单通道脑电信号眼电伪迹自动去除研究. 生物医学工程学杂志, 2021, 38(3): 473-482.
20. 徐曹军. 单通道脑电信号的分析与应用研究. 南京: 南京邮电大学, 2022.
21. Shahbakhti M, Beiramvand M, Nasiri E, et al. Fusion of EEG and eye blink analysis for detection of driver fatigue. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023, 31: 2037-2046.
22. Arif S, Munawar S, Ali H. Driving drowsiness detection using spectral signatures of EEG-based neurophysiology. Frontiers in Physiology, 2023, 14: 1153268.
23. Hadra M, Abdelrahman I. Automatic EEG-based alertness classification using sparse representation and dictionary learning. Journal of Biomedical Engineering and Medical Imaging, 2020, 7(5): 19-28.
24. Huang N E, Shen Z, Long S R, et al. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proceedings Mathematical Physical and Engineering Sciences, 1998, 454(1971): 903-995.
25. Guennec A L, Malinowski S, Tavenard R. Data augmentation for time series classification using convolutional neural networks//ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data, Riva Del Garda, Italy, 2016: 01357973.
26. Wu Haixu, Hu Tengge, Liu Yong, et al. Timesnet: temporal 2D-variation modeling for general time series analysis. arXiv preprint, 2022, arXiv: 2210.02186.
27. Howard A, Sandler M, Chu G, et al. Searching for mobilenetv3//2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul: IEEE, 2019: 1314-1324.
28. Khosla P, Teterwak P, Wang C, et al. Supervised contrastive learning. Neural Information Processing Systems, 2020, 33: 18661-18673.
29. Zheng W L, Lu B L. A multimodal approach to estimating vigilance using EEG and forehead EOG. Journal of Neural Engineering, 2017, 14(2): 026017.
30. Ko L W, Komarov O, Lai W K, et al. Eyeblink recognition improves fatigue prediction from single-channel forehead EEG in a realistic sustained attention task. Journal of Neural Engineering, 2020, 17(3): 036015.
31. Maaten L, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9(86): 2579-2605.

Journal of Biomedical Engineering

Fatigue feature extraction and classification algorithm of forehead single-channel electroencephalography signals

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content