Convolutional neural network human gesture recognition algorithm based on phase portrait of surface electromyography energy kernel_Journal of Biomedical Engineering

Authors：

XU Liukai ^1,2 , ZHANG Keqin ³ , XU Zhaohong ³ ,  YANG Genke ^1,2

1. Ningbo Artificial Intelligence Institute of Shanghai Jiao Tong University, Ningbo, Zhejiang 315000, P.R.China;
2. Department of Automation, Shanghai Jiao Tong University, Shanghai 200240, P.R.China;
3. Ningbo Industrial Internet Institute, Ningbo, Zhejiang 315000, P.R.China;

Corresponding author：

YANG Genke, Email: gkyang@sjtu.edu.cn

Keywords：

surface electromyography; gesture recognition; energy kernel; convolutional neural network

DOI：

10.7507/1001-5515.202010080

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Surface electromyography (sEMG) is a weak signal which is non-stationary and non-periodic. The sEMG classification methods based on time domain and frequency domain features have low recognition rate and poor stability. Based on the modeling and analysis of sEMG energy kernel, this paper proposes a new method to recognize human gestures utilizing convolutional neural network (CNN) and phase portrait of sEMG energy kernel. Firstly, the matrix counting method is used to process the sEMG energy kernel phase portrait into a grayscale image. Secondly, the grayscale image is preprocessed by moving average method. Finally, CNN is used to recognize sEMG of gestures. Experiments on gesture sEMG signal data set show that the effectiveness of the recognition framework and the recognition method of CNN combined with the energy kernel phase portrait have obvious advantages in recognition accuracy and computational efficiency over the area extraction methods. The algorithm in this paper provides a new feasible method for sEMG signal modeling analysis and real-time identification.

Citation： XU Liukai, ZHANG Keqin, XU Zhaohong, YANG Genke. Convolutional neural network human gesture recognition algorithm based on phase portrait of surface electromyography energy kernel. Journal of Biomedical Engineering, 2021, 38(4): 621-629. doi: 10.7507/1001-5515.202010080 Copy

1.	丁其川, 熊安斌, 赵新刚, 等. 基于表面肌电的运动意图识别方法研究及应用综述. 自动化学报, 2016, 42(1): 13-25.
2.	Meng Qingyun, Meng Qiaoling, Yu Hongliu, et al. A survey on sEMG control strategies of wearable hand exoskeleton for rehabilitation//2017 2nd Asia-Pacific Conference on Intelligent Robot Systems (ACIRS), Wuhan: IEEE, 2017: 165-169.
3.	Phukpattaranont P, Thongpanja S, Anam K, et al. Evaluation of feature extraction techniques and classifiers for finger movement recognition using surface electromyography signal. Med Biol Eng Comput, 2018, 56(12): 2259-2271.
4.	丁帅, 王亮. 基于块稀疏贝叶斯学习的肌电信号特征提取. 仪器仪表学报, 2014, 35(12): 2731-2738.
5.	Chen X, Yin Y, Fan Y. EMG oscillator model-based energy kernel method for haracterizing muscle intrinsic property under isometric contraction. Chin Sci Bull, 2014, 59(14): 1556-1567.
6.	Chen X, Zeng Y, Yin Y. Improving the transparency of an exoskeleton knee joint based on the understanding of motor intent using energy kernel method of EMG. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(6): 577-588.
7.	Zeng Y, Yang J, Peng C, et al. Evolving gaussian process autoregression based learning of human motion intent using improved energy kernel method of EMG. IEEE Trans Biomed Eng, 2019, 66(9): 2556-2565.
8.	石欣, 朱家庆, 秦鹏杰, 等. 基于改进能量核的下肢表面肌电信号特征提取方法. 仪器仪表学报, 2020, 41(1): 121-128.
9.	Yang Kuo, Zhang Zhen. Real-time pattern recognition for hand gesture based on ANN and surface EMG//2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing: IEEE, 2019: 799-802.
10.	Abdel-Hamid O, Mohamed A, Jiang H, et al. Convolutional neural networks for speech recognition. IEEE/ACM Trans Audio, Speech, Language Process, 2014, 22(10): 1533-1545.
11.	Pinzón-Arenas J O, Jiménez-Moreno R, Herrera-Benavides J E. Convolutional neural network for hand gesture recognition using 8 different EMG signals//2019 XXII Symposium on Image, Signal Processing and Artificial Vision (STSIVA). Bucaramanga: IEEE, 2019: 1-5.
12.	Atzori M, Cognolato M, Müller H. Deep learning with convolutional neural networks applied to electromyography data: a resource for the classification of movements for prosthetic hands. Front Neurorobot, 2016, 10: 9.
13.	Geng W, Du Y, Jin W, et al. Gesture recognition by instantaneous surface EMG images. Sci Rep, 2016, 6: 36571.
14.	Wei W, Wong Y, Du Y, et al. A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface. Pattern Recognit Lett, 2019, 119: 131-138.
15.	Zhai X, Jelfs B, Chan R M, et al. Self-recalibrating surface EMG pattern recognition for neuroprosthesis control based on convolutional neural network. Front Neurosci, 2017, 11: 379.
16.	He Yunan, Fukuda O, Bu Nan, et al. Surface EMG pattern recognition using long short-term memory combined with multilayer perceptron//2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu: IEEE, 2018: 5636-5639.
17.	Wu Yuheng, Zheng Bin, Zhao Yongting. Dynamic gesture recognition based on LSTM-CNN//2018 Chinese Automation Congress (CAC), Xi’an: IEEE, 2018: 2446-2450.
18.	Staudenmann D, Roeleveld K, Stegeman D F, et al. Methodological aspects of SEMG recordings for force estimation--a tutorial and review. J Electromyogr Kinesiol, 2010, 20(3): 375-387.
19.	Du Y C, Lin C H, Shyu L Y, et al. Portable hand motion classifier for multi-channel surface electromyography recognition using grey relational analysis. Expert Syst Appl, 2010, 37(6): 4283-4291.
20.	McComas A J, Mrozek K. The electrical properties of muscle fiber membranes in dystrophia myotonica and myotonia congenita. J Neurol Neurosurg Psychiatry, 1968, 31(5): 441-447.
21.	Trajano G S, Nosaka K, Blazevich A J. Neurophysiological mechanisms underpinning stretch-induced force loss. Sports Med, 2017, 47(8): 1531-1541.
22.	李伟, 杨向东, 陈恳. 基于CNN和RNN联合网络的心音自动分类. 计算机工程与设计, 2020, 41(1): 46-51.
23.	Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks. Commun ACM, 2017, 60(6): 84-90.

1. 丁其川, 熊安斌, 赵新刚, 等. 基于表面肌电的运动意图识别方法研究及应用综述. 自动化学报, 2016, 42(1): 13-25.
2. Meng Qingyun, Meng Qiaoling, Yu Hongliu, et al. A survey on sEMG control strategies of wearable hand exoskeleton for rehabilitation//2017 2nd Asia-Pacific Conference on Intelligent Robot Systems (ACIRS), Wuhan: IEEE, 2017: 165-169.
3. Phukpattaranont P, Thongpanja S, Anam K, et al. Evaluation of feature extraction techniques and classifiers for finger movement recognition using surface electromyography signal. Med Biol Eng Comput, 2018, 56(12): 2259-2271.
4. 丁帅, 王亮. 基于块稀疏贝叶斯学习的肌电信号特征提取. 仪器仪表学报, 2014, 35(12): 2731-2738.
5. Chen X, Yin Y, Fan Y. EMG oscillator model-based energy kernel method for haracterizing muscle intrinsic property under isometric contraction. Chin Sci Bull, 2014, 59(14): 1556-1567.
6. Chen X, Zeng Y, Yin Y. Improving the transparency of an exoskeleton knee joint based on the understanding of motor intent using energy kernel method of EMG. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(6): 577-588.
7. Zeng Y, Yang J, Peng C, et al. Evolving gaussian process autoregression based learning of human motion intent using improved energy kernel method of EMG. IEEE Trans Biomed Eng, 2019, 66(9): 2556-2565.
8. 石欣, 朱家庆, 秦鹏杰, 等. 基于改进能量核的下肢表面肌电信号特征提取方法. 仪器仪表学报, 2020, 41(1): 121-128.
9. Yang Kuo, Zhang Zhen. Real-time pattern recognition for hand gesture based on ANN and surface EMG//2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing: IEEE, 2019: 799-802.
10. Abdel-Hamid O, Mohamed A, Jiang H, et al. Convolutional neural networks for speech recognition. IEEE/ACM Trans Audio, Speech, Language Process, 2014, 22(10): 1533-1545.
11. Pinzón-Arenas J O, Jiménez-Moreno R, Herrera-Benavides J E. Convolutional neural network for hand gesture recognition using 8 different EMG signals//2019 XXII Symposium on Image, Signal Processing and Artificial Vision (STSIVA). Bucaramanga: IEEE, 2019: 1-5.
12. Atzori M, Cognolato M, Müller H. Deep learning with convolutional neural networks applied to electromyography data: a resource for the classification of movements for prosthetic hands. Front Neurorobot, 2016, 10: 9.
13. Geng W, Du Y, Jin W, et al. Gesture recognition by instantaneous surface EMG images. Sci Rep, 2016, 6: 36571.
14. Wei W, Wong Y, Du Y, et al. A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface. Pattern Recognit Lett, 2019, 119: 131-138.
15. Zhai X, Jelfs B, Chan R M, et al. Self-recalibrating surface EMG pattern recognition for neuroprosthesis control based on convolutional neural network. Front Neurosci, 2017, 11: 379.
16. He Yunan, Fukuda O, Bu Nan, et al. Surface EMG pattern recognition using long short-term memory combined with multilayer perceptron//2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu: IEEE, 2018: 5636-5639.
17. Wu Yuheng, Zheng Bin, Zhao Yongting. Dynamic gesture recognition based on LSTM-CNN//2018 Chinese Automation Congress (CAC), Xi’an: IEEE, 2018: 2446-2450.
18. Staudenmann D, Roeleveld K, Stegeman D F, et al. Methodological aspects of SEMG recordings for force estimation--a tutorial and review. J Electromyogr Kinesiol, 2010, 20(3): 375-387.
19. Du Y C, Lin C H, Shyu L Y, et al. Portable hand motion classifier for multi-channel surface electromyography recognition using grey relational analysis. Expert Syst Appl, 2010, 37(6): 4283-4291.
20. McComas A J, Mrozek K. The electrical properties of muscle fiber membranes in dystrophia myotonica and myotonia congenita. J Neurol Neurosurg Psychiatry, 1968, 31(5): 441-447.
21. Trajano G S, Nosaka K, Blazevich A J. Neurophysiological mechanisms underpinning stretch-induced force loss. Sports Med, 2017, 47(8): 1531-1541.
22. 李伟, 杨向东, 陈恳. 基于CNN和RNN联合网络的心音自动分类. 计算机工程与设计, 2020, 41(1): 46-51.
23. Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks. Commun ACM, 2017, 60(6): 84-90.

Next Article
Research on enhancement of mental rotation ability based on transcranial direct current stimulation

Journal of Biomedical Engineering

Convolutional neural network human gesture recognition algorithm based on phase portrait of surface electromyography energy kernel

Abstract Full text Figures/Tables Video References Cited by

Next Article

Format

Content