Cross-subject electroencephalogram emotion recognition based on maximum classifier discrepancy_Journal of Biomedical Engineering

Authors：

CAI Ziliang ^1,2 , GUO Miaomiao ^1,2 , YANG Xinsheng ^1,2 , CHEN Xintong ^1,2 ,  XU Guizhi ^1,2

1. State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin 300130, P.R.China;
2. Key Laboratory of Bioelectromagnetics and Neuroengineering of Hebei Province, Hebei University of Technology, Tianjin 300130, P.R.China;

Corresponding author：

XU Guizhi, Email: gzxu@hebut.edu.cn

Keywords：

affective brain-computer interfaces; cross-subject affective models; transfer learning; electroencephalogram

DOI：

10.7507/1001-5515.202012027

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Affective brain-computer interfaces (aBCIs) has important application value in the field of human-computer interaction. Electroencephalogram (EEG) has been widely concerned in the field of emotion recognition due to its advantages in time resolution, reliability and accuracy. However, the non-stationary characteristics and individual differences of EEG limit the generalization of emotion recognition model in different time and different subjects. In this paper, in order to realize the recognition of emotional states across different subjects and sessions, we proposed a new domain adaptation method, the maximum classifier difference for domain adversarial neural networks (MCD_DA). By establishing a neural network emotion recognition model, the shallow feature extractor was used to resist the domain classifier and the emotion classifier, respectively, so that the feature extractor could produce domain invariant expression, and train the decision boundary of classifier learning task specificity while realizing approximate joint distribution adaptation. The experimental results showed that the average classification accuracy of this method was 88.33% compared with 58.23% of the traditional general classifier. It improves the generalization ability of emotion brain-computer interface in practical application, and provides a new method for aBCIs to be used in practice.

Citation： CAI Ziliang, GUO Miaomiao, YANG Xinsheng, CHEN Xintong, XU Guizhi. Cross-subject electroencephalogram emotion recognition based on maximum classifier discrepancy. Journal of Biomedical Engineering, 2021, 38(3): 455-462. doi: 10.7507/1001-5515.202012027 Copy

1.	Mühl C, Brendan A, Anton N, et al. A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges. Brain-Comput Interfa, 2014, 1(2): 66-84.
2.	蒋小梅, 张俊然, 陈富琴, 等. 基于 J48 决策树分类器的情绪识别与结果分析. 计算机工程与设计, 2017, 38(3): 761-767.
3.	苏建新. 基于脑电信号的情绪识别研究. 南京: 南京邮电大学, 2015.
4.	杨豪, 张俊然, 蒋小梅, 等. 基于深度信念网络脑电信号表征情绪状态的识别研究. 生物医学工程学杂志, 2018, 35(2): 182-190.
5.	Jayaram V, Alamgir M, Altun Y, et al. Transfer learning in brain-computer interfaces. IEEE Comput Intell Mag, 2016, 11(1): 20-31.
6.	Pan S J, Tsang I W, Kwok J T, et al. Domain adaptation via transfer component analysis. IEEE Trans Neural Netw, 2011, 22(2): 199-210.
7.	Ponti M, Kittler J, Riva M, et al. A decision cognizant Kullback-Leibler divergence. Pattern Recog, 2017, 61(SI): 470-478.
8.	Fuglede B, Topsøe F. Jensen-Shannon divergence and Hilbert space embedding// International Symposium on Information Theory. Chicago: IEEE, 2005: 30.
9.	Sejdinovic D, Sriperumbudur B, Gretton A, et al. Equivalence of distance-based and RKHS-based statistics in hypothesis testing. Ann Stat, 2013, 41(5): 2263-2291.
10.	Zheng Weilong, Lu Baoliang. Personalizing EEG-based affective models with transfer learning// International Conference on Artificial Intelligence. New York: AAAI Press, 2016: 2732-2738.
11.	Jin Yiming, Luo Yudong, Zheng Weilong, et al. EEG-based emotion recognition using domain adaptation network// 2017 International Conference on Orange Technologies (ICOT). Singapore: IEEE, 2017: 222-225.
12.	Ghifary M, Kleijn W B, Zhang M. Domain adaptive neural networks for object recognition// Pacific Rim International Conference on Artificial Intelligence (PRICA). Gold Coast: Springer, 2014: 898-904.
13.	Li Jinpeng, Qiu Shuang, Du Changde, et al. Domain adaptation for EEG emotion recognition based on latent representation similarity. IEEE T Cogn Dev Syst, 2020, 12(2): 344-353.
14.	Long Mingsheng, Wang Jianmin, Ding Guiguang, et al. Transfer feature learning with joint distribution adaptation// Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV). Washington DC: IEEE, 2013: 2200-2207.
15.	Liu Zhaohua, Lu Baoliang, Wei Hualiang, et al. Fault diagnosis for electromechanical drivetrains using a joint distribution optimal deep domain adaptation approach. IEEE Sens J, 2019, 19(24): 12261-12270.
16.	Le Q V, Ranzato M, Monga R, et al. Building high-level features using large scale unsupervised learning// 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Edinburgh: IEEE, 2013: 8595-8598.
17.	Yosinski J, Clune J, Bengio Y, et al. How transferable are features in deep neural networks?// International Conference on Neural Information Processing Systems (NIPS’14). Montreal: MIT Press, 2014: 1-9.
18.	Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks// Advances in Neural Information Processing Systems (NIPS’14). Montreal: MIT Press, 2014, 3: 2672-2680.
19.	Zheng Weilong, Lu Baoliang. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE T Auton Ment De, 2017, 7(3): 162-175.
20.	Taud H, Mas J F. Multilayer perceptron (MLP), geomatic approaches for modeling land change scenarios. New York: Springer International Publishing, 2018.
21.	Ghayoumi M, Bansal A K. Emotion in robots using convolutional neural networks// International Conference on Social Robotics (ICSR). Kansas City: Springer International Publishing, 2016: 285-295.
22.	Wang X, Fouhey D, Gupta A. Designing deep networks for surface normal estimation// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE, 2015: 539-547.
23.	Nair V, Hinton G E. Rectified linear units improve restricted Boltzmann machines// Proceedings of the 27th International Conference on Machine Learning (ICML-10). Haifa: Omnipress, 2010: 807-814.
24.	Saito K, Watanabe K, Ushiku Y, et al. Maximum classifier discrepancy for unsupervised domain adaptation// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 3723-3732.
25.	Laurens V D M, Hinton G. Visualizing Data using t-SNE. J Mach Learn Res, 2008, 9(2605): 2579-2605.

1. Mühl C, Brendan A, Anton N, et al. A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges. Brain-Comput Interfa, 2014, 1(2): 66-84.
2. 蒋小梅, 张俊然, 陈富琴, 等. 基于 J48 决策树分类器的情绪识别与结果分析. 计算机工程与设计, 2017, 38(3): 761-767.
3. 苏建新. 基于脑电信号的情绪识别研究. 南京: 南京邮电大学, 2015.
4. 杨豪, 张俊然, 蒋小梅, 等. 基于深度信念网络脑电信号表征情绪状态的识别研究. 生物医学工程学杂志, 2018, 35(2): 182-190.
5. Jayaram V, Alamgir M, Altun Y, et al. Transfer learning in brain-computer interfaces. IEEE Comput Intell Mag, 2016, 11(1): 20-31.
6. Pan S J, Tsang I W, Kwok J T, et al. Domain adaptation via transfer component analysis. IEEE Trans Neural Netw, 2011, 22(2): 199-210.
7. Ponti M, Kittler J, Riva M, et al. A decision cognizant Kullback-Leibler divergence. Pattern Recog, 2017, 61(SI): 470-478.
8. Fuglede B, Topsøe F. Jensen-Shannon divergence and Hilbert space embedding// International Symposium on Information Theory. Chicago: IEEE, 2005: 30.
9. Sejdinovic D, Sriperumbudur B, Gretton A, et al. Equivalence of distance-based and RKHS-based statistics in hypothesis testing. Ann Stat, 2013, 41(5): 2263-2291.
10. Zheng Weilong, Lu Baoliang. Personalizing EEG-based affective models with transfer learning// International Conference on Artificial Intelligence. New York: AAAI Press, 2016: 2732-2738.
11. Jin Yiming, Luo Yudong, Zheng Weilong, et al. EEG-based emotion recognition using domain adaptation network// 2017 International Conference on Orange Technologies (ICOT). Singapore: IEEE, 2017: 222-225.
12. Ghifary M, Kleijn W B, Zhang M. Domain adaptive neural networks for object recognition// Pacific Rim International Conference on Artificial Intelligence (PRICA). Gold Coast: Springer, 2014: 898-904.
13. Li Jinpeng, Qiu Shuang, Du Changde, et al. Domain adaptation for EEG emotion recognition based on latent representation similarity. IEEE T Cogn Dev Syst, 2020, 12(2): 344-353.
14. Long Mingsheng, Wang Jianmin, Ding Guiguang, et al. Transfer feature learning with joint distribution adaptation// Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV). Washington DC: IEEE, 2013: 2200-2207.
15. Liu Zhaohua, Lu Baoliang, Wei Hualiang, et al. Fault diagnosis for electromechanical drivetrains using a joint distribution optimal deep domain adaptation approach. IEEE Sens J, 2019, 19(24): 12261-12270.
16. Le Q V, Ranzato M, Monga R, et al. Building high-level features using large scale unsupervised learning// 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Edinburgh: IEEE, 2013: 8595-8598.
17. Yosinski J, Clune J, Bengio Y, et al. How transferable are features in deep neural networks?// International Conference on Neural Information Processing Systems (NIPS’14). Montreal: MIT Press, 2014: 1-9.
18. Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks// Advances in Neural Information Processing Systems (NIPS’14). Montreal: MIT Press, 2014, 3: 2672-2680.
19. Zheng Weilong, Lu Baoliang. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE T Auton Ment De, 2017, 7(3): 162-175.
20. Taud H, Mas J F. Multilayer perceptron (MLP), geomatic approaches for modeling land change scenarios. New York: Springer International Publishing, 2018.
21. Ghayoumi M, Bansal A K. Emotion in robots using convolutional neural networks// International Conference on Social Robotics (ICSR). Kansas City: Springer International Publishing, 2016: 285-295.
22. Wang X, Fouhey D, Gupta A. Designing deep networks for surface normal estimation// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE, 2015: 539-547.
23. Nair V, Hinton G E. Rectified linear units improve restricted Boltzmann machines// Proceedings of the 27th International Conference on Machine Learning (ICML-10). Haifa: Omnipress, 2010: 807-814.
24. Saito K, Watanabe K, Ushiku Y, et al. Maximum classifier discrepancy for unsupervised domain adaptation// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 3723-3732.
25. Laurens V D M, Hinton G. Visualizing Data using t-SNE. J Mach Learn Res, 2008, 9(2605): 2579-2605.

Journal of Biomedical Engineering

Cross-subject electroencephalogram emotion recognition based on maximum classifier discrepancy

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content