Enhancement algorithm for surface electromyographic-based gesture recognition based on real-time fusion of muscle fatigue features_Journal of Biomedical Engineering

Authors：

YAN Shijia ^1,2 ,  YANG Ye ^1,2 , YI Peng ^1,2

1. College of Information, Mechanical and Electrical Engineering, Shanghai Normal University, Shanghai 200234, P. R. China;
2. Shanghai Engineering Research Center of Intelligent Education and Bigdata, Shanghai 200234, P. R. China;

Corresponding author：

YANG Ye, Email: yangye0707@shnu.edu.cn

Keywords：

Surface electromyograph; Muscle fatigue; Hand gesture recognition; Convolutional neural networks; Long short-term memory

DOI：

10.7507/1001-5515.202312023

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

This study aims to optimize surface electromyography-based gesture recognition technique, focusing on the impact of muscle fatigue on the recognition performance. An innovative real-time analysis algorithm is proposed in the paper, which can extract muscle fatigue features in real time and fuse them into the hand gesture recognition process. Based on self-collected data, this paper applies algorithms such as convolutional neural networks and long short-term memory networks to provide an in-depth analysis of the feature extraction method of muscle fatigue, and compares the impact of muscle fatigue features on the performance of surface electromyography-based gesture recognition tasks. The results show that by fusing the muscle fatigue features in real time, the algorithm proposed in this paper improves the accuracy of hand gesture recognition at different fatigue levels, and the average recognition accuracy for different subjects is also improved. In summary, the algorithm in this paper not only improves the adaptability and robustness of the hand gesture recognition system, but its research process can also provide new insights into the development of gesture recognition technology in the field of biomedical engineering.

Citation： YAN Shijia, YANG Ye, YI Peng. Enhancement algorithm for surface electromyographic-based gesture recognition based on real-time fusion of muscle fatigue features. Journal of Biomedical Engineering, 2024, 41(5): 958-968. doi: 10.7507/1001-5515.202312023 Copy

1.	Xu H, Xiong A. Advances and disturbances in sEMG-based intentions and movements recognition: a review. IEEE Sensors Journal, 2021, 21(12): 13019-13028.
2.	Staudenmann D, Roeleveld K, Stegeman D F, et al. Methodological aspects of SEMG recordings for force estimation–a tutorial and review. J Electromyogr Kinesiol, 2010, 20(3): 375-387.
3.	Li K, Zhang J, Wang L, et al. A review of the key technologies for sEMG-based human-robot interaction systems. Biomedical Signal Processing and Control, 2020, 62: 102074.
4.	Asif A R, Waris A, Gilani S O, et al. Performance evaluation of convolutional neural network for hand gesture recognition using EMG. Sensors, 2020, 20(6): 1642.
5.	Lee K H, Min J Y, Byun S. Electromyogram-based classification of hand and finger gestures using artificial neural networks. Sensors, 2021, 22(1): 225.
6.	Wu J, Li X, Liu W, et al. sEMG signal processing methods: a review. Journal of Physics: Conference Series, 2019, 1237(3): 032008.
7.	Yu M, Li G, Jiang D, et al. Application of PSO-RBF neural network in gesture recognition of continuous surface EMG signals. Journal of Intelligent & Fuzzy Systems, 2020, 38(3): 2469-2480.
8.	Too J, Abdullah A R, Mohd Saad N. Hybrid binary particle swarm optimization differential evolution-based feature selection for EMG signals classification. Axioms, 2019, 8(3): 79.
9.	Zanghieri M, Benatti S, Burrello A, et al. Robust real-time embedded EMG recognition framework using temporal convolutional networks on a multicore IoT processor. IEEE Transactions on Biomedical Circuits and Systems, 2020, 14(2): 244-256.
10.	Zanghieri M, Benatti S, Burrello A, et al. sEMG-based regression of hand kinematics with temporal convolutional networks on a low-power edge microcontroller//2021 IEEE International Conference on Omni-Layer Intelligent Systems (COINS). IEEE, 2021: 1-6.
11.	Bhushan S, Alshehri M, Keshta I, et al. An experimental analysis of various machine learning algorithms for hand gesture recognition. Electronics, 2022, 11(6): 968.
12.	Sundberg C W, Fitts R H. Bioenergetic basis of skeletal muscle fatigue. Current Opinion in Physiology, 2019, 10: 118-127.
13.	徐瑞, 李志才, 王雯婕, 等. 基于肌电的人机交互控制策略及其应用与挑战. 电子测量与仪器学报, 2020, 34(2): 1-11.
14.	Al-Mulla M R, Sepulveda F, Colley M. A review of non-invasive techniques to detect and predict localised muscle fatigue. Sensors, 2011, 11(4): 3545-3594.
15.	Yousif H A, Zakaria A, Rahim N A, et al. Assessment of muscles fatigue based on surface EMG signals using machine learning and statistical approaches: a review//IOP Conference Series: Materials Science and Engineering. IOP Publishing, 2019, 705(1): 012010.
16.	Ebied A, Awadallah A M, Abbass M A, et al. Upper limb muscle fatigue analysis using multi-channel surface EMG//2020 2nd Novel Intelligent and Leading Emerging Sciences Conference (NILES). IEEE, 2020: 423-427.
17.	Rampichini S, Vieira T M, Castiglioni P, et al. Complexity analysis of surface electromyography for assessing the myoelectric manifestation of muscle fatigue: A review. Entropy, 2020, 22(5): 529.
18.	Su Y, Sun S, Ozturk Y, et al. Measurement of upper limb muscle fatigue using deep belief networks. Journal of Mechanics in Medicine and Biology, 2016, 16(8): 1640032.
19.	Moniri A, Terracina D, Rodriguez-Manzano J, et al. Real-time forecasting of sEMG features for trunk muscle fatigue using machine learning. IEEE Transactions on Biomedical Engineering, 2021, 68(2): 718-727.
20.	Xiong D, Zhang D, Zhao X, et al. Deep learning for EMG-based human-machine interaction: A review. IEEE/CAA Journal of Automatica Sinica, 2021, 8(3): 512-533.
21.	González-Izal M, Malanda A, Gorostiaga E, et al. Electromyographic models to assess muscle fatigue. Journal of Electromyography and Kinesiology, 2012, 22(4): 501-512.
22.	Pizzolato S, Tagliapietra L, Cognolato M, et al. Comparison of six electromyography acquisition setups on hand movement classification tasks. PloS one, 2017, 12(10): e0186132.
23.	Krasoulis A, Kyranou I, Erden M S, et al. Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements. Journal of Neuroengineering and Rehabilitation, 2017, 14(1): 71.
24.	Du Y, Jin W, Wei W, et al. Surface EMG-based inter-session gesture recognition enhanced by deep domain adaptation. Sensors, 2017, 17(3): 458.
25.	Amma C, Krings T, Böer J, et al. Advancing muscle-computer interfaces with high-density electromyography//Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 2015: 929-938.
26.	Jaramillo-Yánez A, Benalcázar M E, Mena-Maldonado E. Real-time hand gesture recognition using surface electromyography and machine learning: A systematic literature review. Sensors, 2020, 20(9): 2467.
27.	Zhu B, Zhang D, Chu Y, et al. SeNic: An open source dataset for sEMG-based gesture recognition in non-ideal conditions. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, 30: 1252-1260.
28.	Wang J, Pang M, Yu P, et al. Effect of muscle fatigue on surface electromyography-based hand grasp force estimation. Applied Bionics and Biomechanics, 2021, 2021: 8817480.
29.	Li W, Shi P, Yu H. Gesture recognition using surface electromyography and deep learning for prostheses hand: state-of-the-art, challenges, and future. Frontiers in Neuroscience, 2021, 15: 621885.
30.	Yang J, Pan J, Li J. sEMG-based continuous hand gesture recognition using GMM-HMM and threshold model//2017 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 2017: 1509-1514.

1. Xu H, Xiong A. Advances and disturbances in sEMG-based intentions and movements recognition: a review. IEEE Sensors Journal, 2021, 21(12): 13019-13028.
2. Staudenmann D, Roeleveld K, Stegeman D F, et al. Methodological aspects of SEMG recordings for force estimation–a tutorial and review. J Electromyogr Kinesiol, 2010, 20(3): 375-387.
3. Li K, Zhang J, Wang L, et al. A review of the key technologies for sEMG-based human-robot interaction systems. Biomedical Signal Processing and Control, 2020, 62: 102074.
4. Asif A R, Waris A, Gilani S O, et al. Performance evaluation of convolutional neural network for hand gesture recognition using EMG. Sensors, 2020, 20(6): 1642.
5. Lee K H, Min J Y, Byun S. Electromyogram-based classification of hand and finger gestures using artificial neural networks. Sensors, 2021, 22(1): 225.
6. Wu J, Li X, Liu W, et al. sEMG signal processing methods: a review. Journal of Physics: Conference Series, 2019, 1237(3): 032008.
7. Yu M, Li G, Jiang D, et al. Application of PSO-RBF neural network in gesture recognition of continuous surface EMG signals. Journal of Intelligent & Fuzzy Systems, 2020, 38(3): 2469-2480.
8. Too J, Abdullah A R, Mohd Saad N. Hybrid binary particle swarm optimization differential evolution-based feature selection for EMG signals classification. Axioms, 2019, 8(3): 79.
9. Zanghieri M, Benatti S, Burrello A, et al. Robust real-time embedded EMG recognition framework using temporal convolutional networks on a multicore IoT processor. IEEE Transactions on Biomedical Circuits and Systems, 2020, 14(2): 244-256.
10. Zanghieri M, Benatti S, Burrello A, et al. sEMG-based regression of hand kinematics with temporal convolutional networks on a low-power edge microcontroller//2021 IEEE International Conference on Omni-Layer Intelligent Systems (COINS). IEEE, 2021: 1-6.
11. Bhushan S, Alshehri M, Keshta I, et al. An experimental analysis of various machine learning algorithms for hand gesture recognition. Electronics, 2022, 11(6): 968.
12. Sundberg C W, Fitts R H. Bioenergetic basis of skeletal muscle fatigue. Current Opinion in Physiology, 2019, 10: 118-127.
13. 徐瑞, 李志才, 王雯婕, 等. 基于肌电的人机交互控制策略及其应用与挑战. 电子测量与仪器学报, 2020, 34(2): 1-11.
14. Al-Mulla M R, Sepulveda F, Colley M. A review of non-invasive techniques to detect and predict localised muscle fatigue. Sensors, 2011, 11(4): 3545-3594.
15. Yousif H A, Zakaria A, Rahim N A, et al. Assessment of muscles fatigue based on surface EMG signals using machine learning and statistical approaches: a review//IOP Conference Series: Materials Science and Engineering. IOP Publishing, 2019, 705(1): 012010.
16. Ebied A, Awadallah A M, Abbass M A, et al. Upper limb muscle fatigue analysis using multi-channel surface EMG//2020 2nd Novel Intelligent and Leading Emerging Sciences Conference (NILES). IEEE, 2020: 423-427.
17. Rampichini S, Vieira T M, Castiglioni P, et al. Complexity analysis of surface electromyography for assessing the myoelectric manifestation of muscle fatigue: A review. Entropy, 2020, 22(5): 529.
18. Su Y, Sun S, Ozturk Y, et al. Measurement of upper limb muscle fatigue using deep belief networks. Journal of Mechanics in Medicine and Biology, 2016, 16(8): 1640032.
19. Moniri A, Terracina D, Rodriguez-Manzano J, et al. Real-time forecasting of sEMG features for trunk muscle fatigue using machine learning. IEEE Transactions on Biomedical Engineering, 2021, 68(2): 718-727.
20. Xiong D, Zhang D, Zhao X, et al. Deep learning for EMG-based human-machine interaction: A review. IEEE/CAA Journal of Automatica Sinica, 2021, 8(3): 512-533.
21. González-Izal M, Malanda A, Gorostiaga E, et al. Electromyographic models to assess muscle fatigue. Journal of Electromyography and Kinesiology, 2012, 22(4): 501-512.
22. Pizzolato S, Tagliapietra L, Cognolato M, et al. Comparison of six electromyography acquisition setups on hand movement classification tasks. PloS one, 2017, 12(10): e0186132.
23. Krasoulis A, Kyranou I, Erden M S, et al. Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements. Journal of Neuroengineering and Rehabilitation, 2017, 14(1): 71.
24. Du Y, Jin W, Wei W, et al. Surface EMG-based inter-session gesture recognition enhanced by deep domain adaptation. Sensors, 2017, 17(3): 458.
25. Amma C, Krings T, Böer J, et al. Advancing muscle-computer interfaces with high-density electromyography//Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 2015: 929-938.
26. Jaramillo-Yánez A, Benalcázar M E, Mena-Maldonado E. Real-time hand gesture recognition using surface electromyography and machine learning: A systematic literature review. Sensors, 2020, 20(9): 2467.
27. Zhu B, Zhang D, Chu Y, et al. SeNic: An open source dataset for sEMG-based gesture recognition in non-ideal conditions. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, 30: 1252-1260.
28. Wang J, Pang M, Yu P, et al. Effect of muscle fatigue on surface electromyography-based hand grasp force estimation. Applied Bionics and Biomechanics, 2021, 2021: 8817480.
29. Li W, Shi P, Yu H. Gesture recognition using surface electromyography and deep learning for prostheses hand: state-of-the-art, challenges, and future. Frontiers in Neuroscience, 2021, 15: 621885.
30. Yang J, Pan J, Li J. sEMG-based continuous hand gesture recognition using GMM-HMM and threshold model//2017 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 2017: 1509-1514.

Journal of Biomedical Engineering

Enhancement algorithm for surface electromyographic-based gesture recognition based on real-time fusion of muscle fatigue features

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content