Control of intelligent car based on electroencephalogram and neurofeedback_Journal of Biomedical Engineering

Authors：

LI Song , XIONG Xin ,  FU Yunfa

School of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, P.R.China;

Corresponding author：

FU Yunfa, Email: fyf@ynu.edu.cn

Keywords：

neurofeedback with electroencephalogram; brain-controlled robot; motor imagery; multi-features fusion; multi-classifiers decision

DOI：

10.7507/1001-5515.201612080

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

To improve the performance of brain-controlled intelligent car based on motor imagery (MI), a method based on neurofeedback (NF) with electroencephalogram (EEG) for controlling intelligent car is proposed. A mental strategy of MI in which the energy column diagram of EEG features related to the mental activity is presented to subjects with visual feedback in real time to train them to quickly master the skills of MI and regulate their EEG activity, and combination of multi-features fusion of MI and multi-classifiers decision were used to control the intelligent car online. The average, maximum and minimum accuracy of identifying instructions achieved by the trained group (trained by the designed feedback system before the experiment) were 85.71%, 90.47% and 76.19%, respectively and the corresponding accuracy achieved by the control group (untrained) were 73.32%, 80.95% and 66.67%, respectively. For the trained group, the average, longest and shortest time consuming were 92 s, 101 s, and 85 s, respectively, while for the control group the corresponding time were 115.7 s, 120 s, and 110 s, respectively. According to the results described above, it is expected that this study may provide a new idea for the follow-up development of brain-controlled intelligent robot by the neurofeedback with EEG related to MI.

Citation： LI Song, XIONG Xin, FU Yunfa. Control of intelligent car based on electroencephalogram and neurofeedback. Journal of Biomedical Engineering, 2018, 35(1): 15-24. doi: 10.7507/1001-5515.201612080 Copy

1.	Schmidt E M, Mcintosh J S, Durelli L, et al. Fine control of operantly conditioned firing patterns of cortical-neurons. Exp Neurol, 1978, 61(2): 349-369.
2.	王行愚, 金晶, 张宇, 等. 脑控: 基于脑-机接口的人机融合控制. 自动化学报, 2013, 39(3): 208-221.
3.	伏云发, 王越超, 李洪谊, 等. 直接脑控机器人接口技术. 自动化学报, 2012, 38(8): 1229-1246.
4.	Miller K J, Schalk G, Fetz E E, et al. Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proc Natl Acad Sci U S A, 2010, 107(15): 7113-7113.
5.	Li J H, Zhang L Q. Bilateral adaptation and neurofeedback for brain computer interface system. J Neurosci Methods, 2010, 193(2): 373-379.
6.	Choi K, Electroencephalography (EEG)-based neurofeedback training for brain-computer interface (BCI). Experimental Brain Research, 2014, 232(3): 1071-1071.
7.	Kondo T, Saeki M, Hayashi Y A, et al. Effect of instructive visual stimuli on neurofeedback training for motor imagery-based brain-computer interface. Hum Mov Sci, 2015, 43: 239-249.
8.	Roberts R, Callow N, Hardy L, et al. Movement imagery ability: development and assessment of a revised version of the vividness of movement imagery questionnaire. J Sport Exerc Psychol, 2008, 30(2): 200-221.
9.	Auer T, Schweizer R, Frahm J. Training efficiency and transfer success in an extended real-time functional MRI neurofeedback training of the somatomotor cortex of healthy subjects. Front Hum Neurosci, 2015, 9: 1-14.
10.	Thibault R T, Lifshitz M, Raz A. The self-regulating brain and neurofeedback: experimental science and clinical promise. Cortex, 2016, 74: 247-261.
11.	Gomez-Pilar J, Corralejo R, Nicolas-Alonso L F, et al. Assessment of neurofeedback training by means of motor imagery based-BCI for cognitive rehabilitation//2014 36th annual international conference of the IEEE engineering in medicine and biology society (EMBC), 2014: 3630-3633.
12.	Neuper C, Scherer R, Wriessnegger S, et al. Motor imagery and action observation: modulation of sensorimotor brain rhythms during mental control of a brain-computer interface. Clin Neurophysiol, 2009, 120(2): 239-247.
13.	Neuper C, Schlogl A, Pfurtscheller G. Enhancement of left-right sensorimotor EEG differences during feedback-regulated motor imagery. Journal of Clinical Neurophysiology, 1999, 16(4): 373-382.
14.	Yu Tianyou, Xiao Jun, Wang Fangyi, et al. Enhanced motor imagery training using a hybrid BCI with feedback. IEEE Trans Biomed Eng, 2015, 62(7): 1706-1717.
15.	Hwang H J, Kwon K, Im C H. Neurofeedback-based motor imagery training for brain-computer interface (BCI). J Neurosci Methods, 2009(1): 150-156.
16.	Alvarez-Meza A M, Velasquez-Martinez L F, Castellanos Dominguez G. Time-series discrimination using feature relevance analysis in motor imagery classification. Neurocomputing, 2015, 151(1): 122-129.
17.	Witte M, Kober S E, Ninaus M, et al. Control beliefs can predict the ability to up-regulate sensorimotor rhythm during neurofeedback training. Front Hum Neurosci, 2013, 7(7): 1-8.
18.	吴小培, 周蚌艳, 张磊. 运动想象脑机-接口中的 ICA 滤波器设计. 生物物理学报, 2014, 30(7): 540-544.
19.	Li Ma, Cui Y, Yang J F, et al. An adaptive multi-domain fusion feature extraction with method HHT and CSSD. Acta Electronica Sinica, 2013, 41(12): 2479-2486.
20.	徐宝国, 彭思, 宋爱国. 基于运动想象脑电的上肢康复机器人. 机器人, 2011, 33(3): 307-313.
21.	Fu K, Qu J F, Chai Y, et al. Classifcation of seizure based on the time-frequency image of EEG signals using HHT and SVM. Biomed Signal Process Control, 2014, 13: 15-22.
22.	孙会文, 伏云发, 熊馨, 等. 基于 HHT 运动想象脑电模式识别研究. 自动化学报, 2015, 41(9): 1686-1692.
23.	Zhao Qibin, Zhang Liqing, Cichocki A. EEG-based asynchronous BCI control of a car in 3D virtual reality environments. Chinese Science Bulletin, 2009, 54(1): 78-87.
24.	Xia B, Zhang Q M, Xie H. A neurofeedback training paradigm for motor imagery based Brain-Computer interface. International Joint Conference on Neural Networks (IJCNN), 2012.
25.	Kus R, Valbuena D, Zygierewicz J, et al. Asynchronous BCI based on motor imagery with automated calibration and neurofeedback training. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2012, 20(6): 823-835.
26.	Lee Y, Kim J, Lee S, et al. Characteristics of motor imagery based EEG-Brain computer interface using combined cue and neuro-feedback//32nd Annual International Conference of the IEEE EMBS Buenos Aires, 2010: 4238-4241.
27.	Kreilinger A, Hiebel H, Mueller-Putz G R. Single versus multiple events error potential detection in a BCI-Controlled car game with continuous and discrete feedback. IEEE Trans Biomed Eng, 2016, 63(3): 519-529.
28.	Lee P L, Chang H C, Hsieh T Y, et al. A brain-wave-actuated small robot car using ensemble empirical mode decomposition-based approach. IEEE Transactions on Systems Man and Cybernetics Part A: Systems and Humans, 2012, 42(5): 1053-1064.
29.	Shu Xiaokang, Yao Lin, Meng Jianjun, et al. Visual stimulus background effects on SSVEP-Based BCI towards a practical robot car control. International Journal of Humanoid Robotics, 2015, 12(2, SI): 155001-155014.

1. Schmidt E M, Mcintosh J S, Durelli L, et al. Fine control of operantly conditioned firing patterns of cortical-neurons. Exp Neurol, 1978, 61(2): 349-369.
2. 王行愚, 金晶, 张宇, 等. 脑控: 基于脑-机接口的人机融合控制. 自动化学报, 2013, 39(3): 208-221.
3. 伏云发, 王越超, 李洪谊, 等. 直接脑控机器人接口技术. 自动化学报, 2012, 38(8): 1229-1246.
4. Miller K J, Schalk G, Fetz E E, et al. Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proc Natl Acad Sci U S A, 2010, 107(15): 7113-7113.
5. Li J H, Zhang L Q. Bilateral adaptation and neurofeedback for brain computer interface system. J Neurosci Methods, 2010, 193(2): 373-379.
6. Choi K, Electroencephalography (EEG)-based neurofeedback training for brain-computer interface (BCI). Experimental Brain Research, 2014, 232(3): 1071-1071.
7. Kondo T, Saeki M, Hayashi Y A, et al. Effect of instructive visual stimuli on neurofeedback training for motor imagery-based brain-computer interface. Hum Mov Sci, 2015, 43: 239-249.
8. Roberts R, Callow N, Hardy L, et al. Movement imagery ability: development and assessment of a revised version of the vividness of movement imagery questionnaire. J Sport Exerc Psychol, 2008, 30(2): 200-221.
9. Auer T, Schweizer R, Frahm J. Training efficiency and transfer success in an extended real-time functional MRI neurofeedback training of the somatomotor cortex of healthy subjects. Front Hum Neurosci, 2015, 9: 1-14.
10. Thibault R T, Lifshitz M, Raz A. The self-regulating brain and neurofeedback: experimental science and clinical promise. Cortex, 2016, 74: 247-261.
11. Gomez-Pilar J, Corralejo R, Nicolas-Alonso L F, et al. Assessment of neurofeedback training by means of motor imagery based-BCI for cognitive rehabilitation//2014 36th annual international conference of the IEEE engineering in medicine and biology society (EMBC), 2014: 3630-3633.
12. Neuper C, Scherer R, Wriessnegger S, et al. Motor imagery and action observation: modulation of sensorimotor brain rhythms during mental control of a brain-computer interface. Clin Neurophysiol, 2009, 120(2): 239-247.
13. Neuper C, Schlogl A, Pfurtscheller G. Enhancement of left-right sensorimotor EEG differences during feedback-regulated motor imagery. Journal of Clinical Neurophysiology, 1999, 16(4): 373-382.
14. Yu Tianyou, Xiao Jun, Wang Fangyi, et al. Enhanced motor imagery training using a hybrid BCI with feedback. IEEE Trans Biomed Eng, 2015, 62(7): 1706-1717.
15. Hwang H J, Kwon K, Im C H. Neurofeedback-based motor imagery training for brain-computer interface (BCI). J Neurosci Methods, 2009(1): 150-156.
16. Alvarez-Meza A M, Velasquez-Martinez L F, Castellanos Dominguez G. Time-series discrimination using feature relevance analysis in motor imagery classification. Neurocomputing, 2015, 151(1): 122-129.
17. Witte M, Kober S E, Ninaus M, et al. Control beliefs can predict the ability to up-regulate sensorimotor rhythm during neurofeedback training. Front Hum Neurosci, 2013, 7(7): 1-8.
18. 吴小培, 周蚌艳, 张磊. 运动想象脑机-接口中的 ICA 滤波器设计. 生物物理学报, 2014, 30(7): 540-544.
19. Li Ma, Cui Y, Yang J F, et al. An adaptive multi-domain fusion feature extraction with method HHT and CSSD. Acta Electronica Sinica, 2013, 41(12): 2479-2486.
20. 徐宝国, 彭思, 宋爱国. 基于运动想象脑电的上肢康复机器人. 机器人, 2011, 33(3): 307-313.
21. Fu K, Qu J F, Chai Y, et al. Classifcation of seizure based on the time-frequency image of EEG signals using HHT and SVM. Biomed Signal Process Control, 2014, 13: 15-22.
22. 孙会文, 伏云发, 熊馨, 等. 基于 HHT 运动想象脑电模式识别研究. 自动化学报, 2015, 41(9): 1686-1692.
23. Zhao Qibin, Zhang Liqing, Cichocki A. EEG-based asynchronous BCI control of a car in 3D virtual reality environments. Chinese Science Bulletin, 2009, 54(1): 78-87.
24. Xia B, Zhang Q M, Xie H. A neurofeedback training paradigm for motor imagery based Brain-Computer interface. International Joint Conference on Neural Networks (IJCNN), 2012.
25. Kus R, Valbuena D, Zygierewicz J, et al. Asynchronous BCI based on motor imagery with automated calibration and neurofeedback training. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2012, 20(6): 823-835.
26. Lee Y, Kim J, Lee S, et al. Characteristics of motor imagery based EEG-Brain computer interface using combined cue and neuro-feedback//32nd Annual International Conference of the IEEE EMBS Buenos Aires, 2010: 4238-4241.
27. Kreilinger A, Hiebel H, Mueller-Putz G R. Single versus multiple events error potential detection in a BCI-Controlled car game with continuous and discrete feedback. IEEE Trans Biomed Eng, 2016, 63(3): 519-529.
28. Lee P L, Chang H C, Hsieh T Y, et al. A brain-wave-actuated small robot car using ensemble empirical mode decomposition-based approach. IEEE Transactions on Systems Man and Cybernetics Part A: Systems and Humans, 2012, 42(5): 1053-1064.
29. Shu Xiaokang, Yao Lin, Meng Jianjun, et al. Visual stimulus background effects on SSVEP-Based BCI towards a practical robot car control. International Journal of Humanoid Robotics, 2015, 12(2, SI): 155001-155014.

Journal of Biomedical Engineering

Control of intelligent car based on electroencephalogram and neurofeedback

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content