Recognition of three different imagined movement of the right foot based on functional near-infrared spectroscopy_Journal of Biomedical Engineering

Authors：

LI Yu ^1,2,3 , XIONG Xin ^1,2,3 , LI Zhaoyang ^1,2,3 ,  FU Yunfa ^1,2,3

1. School of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, P.R.China;
2. Integration and Innovation team of Brain Cognition and Brain Computer Intelligence, Kunming University of Science and Technology, Kunming 650500, P.R.China;
3. Key Laboratory of Computer Technology Application in Yunnan Province, Kunming 650500, P.R.China;

Corresponding author：

FU Yunfa, Email: fyf@ynu.edu.cn

Keywords：

functional near-infrared spectroscopy; motor imagery; correlation coefficient; brain-computer interface; classification and recognition

DOI：

10.7507/1001-5515.201905001

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Brain-computer interface (BCI) based on functional near-infrared spectroscopy (fNIRS) is a new-type human-computer interaction technique. To explore the separability of fNIRS signals in different motor imageries on the single limb, the study measured the fNIRS signals of 15 subjects (amateur football fans) during three different motor imageries of the right foot (passing, stopping and shooting). And the correlation coefficient of the HbO signal during different motor imageries was extracted as features for the input of a three-classification model based on support vector machines. The results found that the classification accuracy of the three motor imageries of the right foot was 78.89%±6.161%. The classification accuracy of the two-classification of motor imageries of the right foot, that is, passing and stopping, passing and shooting, and stopping and shooting was 85.17%±4.768%, 82.33%±6.011%, and 89.33%±6.713%, respectively. The results demonstrate that the fNIRS of different motor imageries of the single limb is separable, which is expected to add new control commands to fNIRS-BCI and also provide a new option for rehabilitation training and control peripherals for unilateral stroke patients. Besides, the study also confirms that the correlation coefficient can be used as an effective feature to classify different motor imageries.

Citation： LI Yu, XIONG Xin, LI Zhaoyang, FU Yunfa. Recognition of three different imagined movement of the right foot based on functional near-infrared spectroscopy. Journal of Biomedical Engineering, 2020, 37(2): 262-270. doi: 10.7507/1001-5515.201905001 Copy

1.	Asensio-Cubero J, Gan J Q, Palaniappan R. Multiresolution analysis over graphs for a motor imagery based online BCI game. Comput Biol Med, 2016, 68: 21-26.
2.	Xu Minpeng, Qi Hongzhi, Wan Baikun, et al. A hybrid BCI speller paradigm combining P300 potential and the SSVEP blocking feature. J Neural Eng, 2013, 10(2): 26001-26013.
3.	Lotte F, Faller J, Guger C, et al. Combining BCI with virtual reality: Towards new applications and improved BCI. Berlin Springer Berlin Heidlberg, 2012: 197-220.
4.	Tumanov K, Goebel R, Möckel R, et al. fNIRS-based BCI for robot control// Proceeding of the 2015 International Conference on Autonomous Agents and Multiagent Systems. Istanbul: International Foundation for Autonomous Agents and Multiagent Systems, 2015: 1953-1954.
5.	Pichiorri F, Morone G, Petti M, et al. Brain-computer interface boots motor imagery practice during stroke recovery. Ann Neurol, 2015, 77(5): 851-865.
6.	Bamdad M, Zarshenas H, Auais M A. Application of BCI systems in neurorehabilitation: a scoping review. Disabil Rehabil Assist Technol, 2015, 10(5): 355-364.
7.	Turnip A, Hong K S, Jeong M Y. Real-time feature extraction of P300 component using adaptive nonlinear principal component analysis. Biomed Eng Online, 2011, 10(1): 83.
8.	Popovich C, Staines W R. Acute aerobic exercise enhances attentional modulation of somatosensory event-related potentials during a tactile discrimination task. Behav Brain Res, 2015, 281: 267-275.
9.	Yuan Zhen, Ye J C. Fusion of fNIRS and fMRI data: identifying when and where hemodynamic signals are changing in human brains. Front Hum Neurosci, 2013, 7: 676.
10.	Ghio M, Schulze P, Suchan B, et al. Neural representations of novel objects associated with olfactory experience. Behav Brain Res, 2016, 308: 143-151.
11.	Gillis M M, Garcia S, Hampstead B M. Working memory contributes to the encoding of object location associations: Support for a 3-part model of object location memory. Behav Brain Res, 2016, 311: 192-200.
12.	Kim J, Chung Y G, Chung S C, et al. Decoding pressure stimulation locations on the fingers from human neural activation patterns. Neuroreport, 2016, 27(16): 1232-1236.
13.	Verriotis M, Chang P, Fitzgerald M, et al. The development of the nociceptive brain. Neuroscience, 2016, 338: 207-219.
14.	Tempest G D, Eston R G, Parfitt G. Prefrontal cortex haemodynamics and affective responses during exercise: a multi-channel near infrared spectroscopy study. PLoS One, 2014, 9(5): 95924.
15.	León-Domínguez U, Martín-Rodríguez J F, León-Carrión J. Executive n-back tasks for the neuropsychological assessment of working memory. Behav Brain Res, 2015, 292: 167-173.
16.	Naseer N, Hong K S. Decoding answers to four-choice questions using functional near infrared spectroscopy. Journal of Near Infrared Spectroscopy, 2015, 23(1): 23-31.
17.	Nguyen H D, Hong K S, Shin Y I. Bundled-optode method in functional near-infrared spectroscopy. PLoS One, 2016, 11(10): 0165146.
18.	Kim C K, Lee S, Koh D, et al. Development of wireless NIRS system with dynamic removal of motion artifacts. Biomed Eng Lett, 2011, 1(4): 254-259.
19.	Khan M J, Hong M J, Hong K S. Decoding of four movement directions using hybrid NIRS-EEG brain-computer interface. Front Hum Neurosci, 2014, 8: 244.
20.	Piper S K, Krueger A, Koch S P, et al. A wearable multi-channel fNIRS system for brain imaging in freely moving subjects. Neuroimage, 2014, 85(1, SI): 64-71.
21.	Bhutta M R, Hong M J, Kim Y H, et al. Single-trial lie detection using a combined fNIRS-polygraph system. Front Psychol, 2015, 6: 709.
22.	Naseer N, Hong K S. Classification of functional near-infrared spectroscopy signals corresponding to the right- and left-wrist motor imagery for development of a brain-computer interface. Neurosci Lett, 2013, 553: 84-89.
23.	Nguyen H D, Hong K S. Bundled-optode implementation for 3D imaging in functional near-infrared spectroscopy. Biomed Opt Express, 2016, 7(9): 3491-3507.
24.	Zafar A, Hong K S. Detection and classification of three-class initial dips from prefrontal cortex. Biomed Opt Express, 2017, 8(1): 367-383.
25.	Quaresima V, Lepanto R, Ferrari M. The use of near infrared spectroscopy in sports medicine. J Sports Med Phys Fitness, 2003, 43(1): 1-13.
26.	Fazli S, Mehnert J, Steinbrink J, et al. Enhanced performance by a hybrid NIRS-EEG brain computer interface. Neuroimage, 2012, 59(1): 519-529.
27.	Naseer N, Hong M J, Hong K S. Online binary decision decoding using functional near-infrared spectroscopy for the development of brain-computer interface. Exp Brain Res, 2014, 232(2): 555-564.
28.	Schudlo L C, Chau T. Single-trial classification of near-infrared spectroscopy signals arising from multiple cortical regions. Behav Brain Res, 2015, 290: 131-142.
29.	Naseer N, Hong K S. fNIRS-based brain-computer interfaces: a review. Front Hum Neurosci, 2015, 9: e3.
30.	焦学军, 张朕, 姜劲, 等. 基于功能性近红外光谱技术的脑机接口. 上海交通大学学报, 2017, 51(12): 1456-1463.
31.	Kaiser V, Bauernfeind G, Kreilinger A, et al. Cortical effects of user training in a motor imagery based brain-computer interface measured by fNIRS and EEG. Neuroimage, 2014, 85(Pt 1): 432-444.
32.	Kirilina E, Jelzow A, Heine A, et al. The physiological origin of task-evoked systemic artefacts in functional near infrared spectroscopy. Neuroimage, 2012, 61(1): 70-81.
33.	Hong K S, Santosa H. Decoding four different sound-categories in the auditory cortex using functional near-infrared spectroscopy. Hear Res, 2016, 333: 157-166.
34.	Jang K E, Tak S, Jung J, et al. Wavelet minimum description length detrending for near-infrared spectroscopy. J Biomed Opt, 2009, 14(3): 659-660.
35.	Nishiyori R, Bisconti S, Ulrich B D. Motor cortex activity during functional motor skills: an fNIRS study. Brain Topography, 2016, 29(1): 42-55.
36.	Cui Xu, Bray S, Reiss A L. Functional near infrared spectroscopy (NIRS) signal improvement based on negative correlation between oxygenated and deoxygenated hemoglobin dynamics. Neuroimage, 2010, 49(4): 3039-3046.
37.	Swami A, Jain R. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 2012, 12(10): 2825-2830.
38.	Dura-Bernal S, Zhou Xianlian, Neymotin S A, et al. Cortical spiking network interfaced with virtual musculoskeletal arm and robotic arm. Front Neurorobot, 2015, 9: 13.
39.	Kocaturk M, Gulcur H O, Canbeyli R. Toward building hybrid biological/in silico neural networks for motor neuroprosthetic control. Front Neurorobot, 2015, 9: 8.
40.	Power S D, Kushki A, Chau T. Automatic single-trial discrimination of mental arithmetic, mental singing and the no-control state from prefrontal activity: toward a three-state NIRS-BCI. BMC Res Notes, 2012, 5(1): 141.
41.	苏杭. 基于大数据背景的相关系数. 电子技术与软件工程, 2018(7): 182-182.
42.	宋廷山. 相关系数统计量的功能及其应用探讨——以SPSS为分析工具. 统计教育, 2008(11): 27-31.
43.	陈珍珍, 罗乐勤. 统计学(精品课程立体教材系列). 北京: 科学出版社发行处出版社, 2006.
44.	徐胜. 三维物体识别研究. 成都: 电子科技大学, 2010.
45.	Hong K S, Naseer N, Kim Y H. Classification of prefrontal and motor cortex signals for three-class fNIRS-BCI. Neurosci Lett, 2015, 587: 87-92.
46.	Hong K S, Bhutta M R, Liu Xiaolong, et al. Classification of somatosensory cortex activities using fNIRS. Behav Brain Res, 2017, 333: 225-234.
47.	Vidal A C, Banca P, Pascoal A G, et al. Modulation of cortical interhemispheric interactions by motor facilitation or restraint. Neural Plasticity, 2014, 2014: 1-8.
48.	Naito M, Michioka Y, Ozawa K, et al. A communication means for totally locked-in ALS patients based on changes in cerebral blood volume measured with near-infrered light. IEICE Trans Inf Syst, 2007, 90(7): 1028-1037.
49.	Wang Gang, Erpelding N, Davis K D. Sex differences in connectivity of the subgenual anterior cingulate cortex. Pain, 2014, 155(4): 755-763.
50.	Brodoehl S, Klingner C, Witte O W. Age-dependent modulation of the somatosensory network upon eye closure. Behav Brain Res, 2016, 298(B): 52-56.
51.	Wang Bitan, Zhang Ming, Bu Lingguo, et al. Posture-related changes in brain functional connectivity as assessed by wavelet phase coherence of NIRS signals in elderly subjects. Behav Brain Res, 2016, 312: 238-245.

1. Asensio-Cubero J, Gan J Q, Palaniappan R. Multiresolution analysis over graphs for a motor imagery based online BCI game. Comput Biol Med, 2016, 68: 21-26.
2. Xu Minpeng, Qi Hongzhi, Wan Baikun, et al. A hybrid BCI speller paradigm combining P300 potential and the SSVEP blocking feature. J Neural Eng, 2013, 10(2): 26001-26013.
3. Lotte F, Faller J, Guger C, et al. Combining BCI with virtual reality: Towards new applications and improved BCI. Berlin Springer Berlin Heidlberg, 2012: 197-220.
4. Tumanov K, Goebel R, Möckel R, et al. fNIRS-based BCI for robot control// Proceeding of the 2015 International Conference on Autonomous Agents and Multiagent Systems. Istanbul: International Foundation for Autonomous Agents and Multiagent Systems, 2015: 1953-1954.
5. Pichiorri F, Morone G, Petti M, et al. Brain-computer interface boots motor imagery practice during stroke recovery. Ann Neurol, 2015, 77(5): 851-865.
6. Bamdad M, Zarshenas H, Auais M A. Application of BCI systems in neurorehabilitation: a scoping review. Disabil Rehabil Assist Technol, 2015, 10(5): 355-364.
7. Turnip A, Hong K S, Jeong M Y. Real-time feature extraction of P300 component using adaptive nonlinear principal component analysis. Biomed Eng Online, 2011, 10(1): 83.
8. Popovich C, Staines W R. Acute aerobic exercise enhances attentional modulation of somatosensory event-related potentials during a tactile discrimination task. Behav Brain Res, 2015, 281: 267-275.
9. Yuan Zhen, Ye J C. Fusion of fNIRS and fMRI data: identifying when and where hemodynamic signals are changing in human brains. Front Hum Neurosci, 2013, 7: 676.
10. Ghio M, Schulze P, Suchan B, et al. Neural representations of novel objects associated with olfactory experience. Behav Brain Res, 2016, 308: 143-151.
11. Gillis M M, Garcia S, Hampstead B M. Working memory contributes to the encoding of object location associations: Support for a 3-part model of object location memory. Behav Brain Res, 2016, 311: 192-200.
12. Kim J, Chung Y G, Chung S C, et al. Decoding pressure stimulation locations on the fingers from human neural activation patterns. Neuroreport, 2016, 27(16): 1232-1236.
13. Verriotis M, Chang P, Fitzgerald M, et al. The development of the nociceptive brain. Neuroscience, 2016, 338: 207-219.
14. Tempest G D, Eston R G, Parfitt G. Prefrontal cortex haemodynamics and affective responses during exercise: a multi-channel near infrared spectroscopy study. PLoS One, 2014, 9(5): 95924.
15. León-Domínguez U, Martín-Rodríguez J F, León-Carrión J. Executive n-back tasks for the neuropsychological assessment of working memory. Behav Brain Res, 2015, 292: 167-173.
16. Naseer N, Hong K S. Decoding answers to four-choice questions using functional near infrared spectroscopy. Journal of Near Infrared Spectroscopy, 2015, 23(1): 23-31.
17. Nguyen H D, Hong K S, Shin Y I. Bundled-optode method in functional near-infrared spectroscopy. PLoS One, 2016, 11(10): 0165146.
18. Kim C K, Lee S, Koh D, et al. Development of wireless NIRS system with dynamic removal of motion artifacts. Biomed Eng Lett, 2011, 1(4): 254-259.
19. Khan M J, Hong M J, Hong K S. Decoding of four movement directions using hybrid NIRS-EEG brain-computer interface. Front Hum Neurosci, 2014, 8: 244.
20. Piper S K, Krueger A, Koch S P, et al. A wearable multi-channel fNIRS system for brain imaging in freely moving subjects. Neuroimage, 2014, 85(1, SI): 64-71.
21. Bhutta M R, Hong M J, Kim Y H, et al. Single-trial lie detection using a combined fNIRS-polygraph system. Front Psychol, 2015, 6: 709.
22. Naseer N, Hong K S. Classification of functional near-infrared spectroscopy signals corresponding to the right- and left-wrist motor imagery for development of a brain-computer interface. Neurosci Lett, 2013, 553: 84-89.
23. Nguyen H D, Hong K S. Bundled-optode implementation for 3D imaging in functional near-infrared spectroscopy. Biomed Opt Express, 2016, 7(9): 3491-3507.
24. Zafar A, Hong K S. Detection and classification of three-class initial dips from prefrontal cortex. Biomed Opt Express, 2017, 8(1): 367-383.
25. Quaresima V, Lepanto R, Ferrari M. The use of near infrared spectroscopy in sports medicine. J Sports Med Phys Fitness, 2003, 43(1): 1-13.
26. Fazli S, Mehnert J, Steinbrink J, et al. Enhanced performance by a hybrid NIRS-EEG brain computer interface. Neuroimage, 2012, 59(1): 519-529.
27. Naseer N, Hong M J, Hong K S. Online binary decision decoding using functional near-infrared spectroscopy for the development of brain-computer interface. Exp Brain Res, 2014, 232(2): 555-564.
28. Schudlo L C, Chau T. Single-trial classification of near-infrared spectroscopy signals arising from multiple cortical regions. Behav Brain Res, 2015, 290: 131-142.
29. Naseer N, Hong K S. fNIRS-based brain-computer interfaces: a review. Front Hum Neurosci, 2015, 9: e3.
30. 焦学军, 张朕, 姜劲, 等. 基于功能性近红外光谱技术的脑机接口. 上海交通大学学报, 2017, 51(12): 1456-1463.
31. Kaiser V, Bauernfeind G, Kreilinger A, et al. Cortical effects of user training in a motor imagery based brain-computer interface measured by fNIRS and EEG. Neuroimage, 2014, 85(Pt 1): 432-444.
32. Kirilina E, Jelzow A, Heine A, et al. The physiological origin of task-evoked systemic artefacts in functional near infrared spectroscopy. Neuroimage, 2012, 61(1): 70-81.
33. Hong K S, Santosa H. Decoding four different sound-categories in the auditory cortex using functional near-infrared spectroscopy. Hear Res, 2016, 333: 157-166.
34. Jang K E, Tak S, Jung J, et al. Wavelet minimum description length detrending for near-infrared spectroscopy. J Biomed Opt, 2009, 14(3): 659-660.
35. Nishiyori R, Bisconti S, Ulrich B D. Motor cortex activity during functional motor skills: an fNIRS study. Brain Topography, 2016, 29(1): 42-55.
36. Cui Xu, Bray S, Reiss A L. Functional near infrared spectroscopy (NIRS) signal improvement based on negative correlation between oxygenated and deoxygenated hemoglobin dynamics. Neuroimage, 2010, 49(4): 3039-3046.
37. Swami A, Jain R. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 2012, 12(10): 2825-2830.
38. Dura-Bernal S, Zhou Xianlian, Neymotin S A, et al. Cortical spiking network interfaced with virtual musculoskeletal arm and robotic arm. Front Neurorobot, 2015, 9: 13.
39. Kocaturk M, Gulcur H O, Canbeyli R. Toward building hybrid biological/in silico neural networks for motor neuroprosthetic control. Front Neurorobot, 2015, 9: 8.
40. Power S D, Kushki A, Chau T. Automatic single-trial discrimination of mental arithmetic, mental singing and the no-control state from prefrontal activity: toward a three-state NIRS-BCI. BMC Res Notes, 2012, 5(1): 141.
41. 苏杭. 基于大数据背景的相关系数. 电子技术与软件工程, 2018(7): 182-182.
42. 宋廷山. 相关系数统计量的功能及其应用探讨——以SPSS为分析工具. 统计教育, 2008(11): 27-31.
43. 陈珍珍, 罗乐勤. 统计学(精品课程立体教材系列). 北京: 科学出版社发行处出版社, 2006.
44. 徐胜. 三维物体识别研究. 成都: 电子科技大学, 2010.
45. Hong K S, Naseer N, Kim Y H. Classification of prefrontal and motor cortex signals for three-class fNIRS-BCI. Neurosci Lett, 2015, 587: 87-92.
46. Hong K S, Bhutta M R, Liu Xiaolong, et al. Classification of somatosensory cortex activities using fNIRS. Behav Brain Res, 2017, 333: 225-234.
47. Vidal A C, Banca P, Pascoal A G, et al. Modulation of cortical interhemispheric interactions by motor facilitation or restraint. Neural Plasticity, 2014, 2014: 1-8.
48. Naito M, Michioka Y, Ozawa K, et al. A communication means for totally locked-in ALS patients based on changes in cerebral blood volume measured with near-infrered light. IEICE Trans Inf Syst, 2007, 90(7): 1028-1037.
49. Wang Gang, Erpelding N, Davis K D. Sex differences in connectivity of the subgenual anterior cingulate cortex. Pain, 2014, 155(4): 755-763.
50. Brodoehl S, Klingner C, Witte O W. Age-dependent modulation of the somatosensory network upon eye closure. Behav Brain Res, 2016, 298(B): 52-56.
51. Wang Bitan, Zhang Ming, Bu Lingguo, et al. Posture-related changes in brain functional connectivity as assessed by wavelet phase coherence of NIRS signals in elderly subjects. Behav Brain Res, 2016, 312: 238-245.

Journal of Biomedical Engineering

Recognition of three different imagined movement of the right foot based on functional near-infrared spectroscopy

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content