Multi-modal synergistic quantitative analysis and rehabilitation assessment of lower limbs for exoskeleton_Journal of Biomedical Engineering

Authors：

ZHONG Xu ^1,2,3 , ZHANG Bi ^1,2 , LI Jiwei ^1,2 , ZHANG Liang ⁴ , YUAN Xiangnan ⁵ , ZHANG Peng ³ ,  ZHAO Xingang ^1,2

1. State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, P. R. China;
2. Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110016, P. R. China;
3. Medical Engineering Department, Affiliated Hospital of Yangzhou University, Yangzhou, Jiangsu 225003, P. R. China;
4. Department of Rehabilitation, Liaoning Provincial People's Hospital, Shenyang 110067, P. R. China;
5. Shengjing Hospital of China Medical University, Shenyang 110004, P. R. China;

Corresponding author：

ZHAO Xingang, Email: zhaoxingang@sia.cn

Keywords：

Rehabilitation assessment; Muscle synergy; Modal fusion; Machine learning; Stroke

DOI：

10.7507/1001-5515.202212028

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

In response to the problem that the traditional lower limb rehabilitation scale assessment method is time-consuming and difficult to use in exoskeleton rehabilitation training, this paper proposes a quantitative assessment method for lower limb walking ability based on lower limb exoskeleton robot training with multimodal synergistic information fusion. The method significantly improves the efficiency and reliability of the rehabilitation assessment process by introducing quantitative synergistic indicators fusing electrophysiological and kinematic level information. First, electromyographic and kinematic data of the lower extremity were collected from subjects trained to walk wearing an exoskeleton. Then, based on muscle synergy theory, a synergistic quantification algorithm was used to construct synergistic index features of electromyography and kinematics. Finally, the electrophysiological and kinematic level information was fused to build a modal feature fusion model and output the lower limb motor function score. The experimental results showed that the correlation coefficients of the constructed synergistic features of electromyography and kinematics with the clinical scale were 0.799 and 0.825, respectively. The results of the fused synergistic features in the K-nearest neighbor (KNN) model yielded higher correlation coefficients (r = 0.921, P < 0.01). This method can modify the rehabilitation training mode of the exoskeleton robot according to the assessment results, which provides a basis for the synchronized assessment-training mode of “human in the loop” and provides a potential method for remote rehabilitation training and assessment of the lower extremity.

Citation： ZHONG Xu, ZHANG Bi, LI Jiwei, ZHANG Liang, YUAN Xiangnan, ZHANG Peng, ZHAO Xingang. Multi-modal synergistic quantitative analysis and rehabilitation assessment of lower limbs for exoskeleton. Journal of Biomedical Engineering, 2023, 40(5): 953-964. doi: 10.7507/1001-5515.202212028 Copy

1.	Zhou M, Wang H, Zeng X, et al. Mortality, morbidity, and risk factors in China and its provinces, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet, 2019, 394(10204): 1145-1158.
2.	Hatem S M, Saussez G, Della Faille M, et al. Rehabilitation of motor function after stroke: a multiple systematic review focused on techniques to stimulate upper extremity recovery. Front Hum Neurosci, 2016, 10: 442.
3.	中华医学会神经病学分会神经康复学组, 中华医学会神经病学分会脑血管病学组, 卫生部脑卒中筛查与防治工程委员会办公室, 等. 中国脑卒中康复治疗指南(2011完全版). 中国康复理论与实践, 2012, 18(4): 301-318.
4.	国家卫生健康委员会. 2021中国卫生健康统计年鉴. 北京: 中国协和医科大学出版社, 2021.
5.	Esquenazi A, Talaty M, Packel A, et al. The ReWalk powered exoskeleton to restore ambulatory function to individuals with thoracic-level motor-complete spinal cord injury. American Journal of Physical Medicine Rehabilitation, 2012, 91(11): 911-921.
6.	Nilsson A, Vreede K S, Häglund V, et al. Gait training early after stroke with a new exoskeleton–the hybrid assistive limb: a study of safety and feasibility. J Neuroeng Rehabil, 2014, 11: 92.
7.	Nam K Y, Kim H J, Kwon B S, et al. Robot-assisted gait training (Lokomat) improves walking function and activity in people with spinal cord injury: a systematic review. J Neuroeng Rehabil, 2017, 14(1): 24.
8.	刘京运. 从马拉松到冬奥会,大艾外骨骼机器人为残疾人创造更多可能. 机器人产业, 2022, 2022(4): 35-39.
9.	程洪, 黄瑞, 邱静, 等. 康复机器人及其临床应用综述. 机器人, 2021, 43(5): 606-619.
10.	王天. 一种下肢康复训练用外骨骼机器人. 浙江省: CN215193457U, 2021-12-17.
11.	丁逸苇, 涂利娟, 刘怡希, 等. 可穿戴式下肢外骨骼康复机器人研究进展. 机器人, 2022, 44(5): 522-532.
12.	Zhang J, Fiers P, Witte K A, et al. Human-in-the-loop optimization of exoskeleton assistance during walking. Science, 2017, 356(6344): 1280-1284.
13.	Gordon D F, Mcgreavy C, Christou A, et al. Human-in-the-loop optimization of exoskeleton assistance via online simulation of metabolic cost. IEEE Trans Rob, 2022, 38(3): 1410-1429.
14.	Brunnstrom S. Motor testing procedures in hemiplegia: based on sequential recovery stages. Physical Therapy, 1966, 46(4): 357-375.
15.	Sanford J, Moreland J, Swanson L R, et al. Reliability of the Fugl-Meyer assessment for testing motor performance in patients following stroke. Physical therapy, 1993, 73(7): 447-454.
16.	Carr J H, Shepherd R B, Nordholm L, et al. Investigation of a new motor assessment scale for stroke patients. Physical therapy, 1985, 65(2): 175-180.
17.	Kinoshita S, Abo M, Okamoto T. Effectiveness of ICF-based multidisciplinary rehabilitation approach with serial assessment and discussion using the ICF rehabilitation set in a convalescent rehabilitation ward. Int J Rehabil Res, 2020, 43(3): 255-260.
18.	姜荣荣, 陈艳, 潘翠环. 脑卒中后上肢和手运动功能康复评定的研究进展. 中国康复理论与实践, 2015, 21(10): 1173-1177.
19.	Gao F, Wang L, Lin T. Intelligent wearable rehabilitation robot control system based on mobile communication network. Comput Commun, 2020, 153: 286-293.
20.	Bai J, Song A. Development of a novel home based multi-scene upper limb rehabilitation training and evaluation system for post-stroke patients. IEEE Access, 2019, 7: 9667-9677.
21.	Cerfoglio S, Ferraris C, Vismara L, et al. Kinect-based assessment of lower limbs during gait in post-stroke hemiplegic patients: a narrative review. Sensors, 2022, 22(13): 4910.
22.	Qian C, Li W, Jia T, et al. Quantitative assessment of motor function by an end-effector upper limb rehabilitation robot based on admittance control. Appl Sci, 2021, 11(15): 6854.
23.	Meziani Y, Morère Y, Hadj-Abdelkader A, et al. Towards adaptive and finer rehabilitation assessment: a learning framework for kinematic evaluation of upper limb rehabilitation on an Armeo Spring exoskeleton. Control Engineering Practice, 2021, 111: 104804.
24.	Liparulo L, Zhang Z, Panella M, et al. A novel fuzzy approach for automatic Brunnstrom stage classification using surface electromyography. Med Biol Eng Comput, 2017, 55(8): 1367-1378.
25.	Yu L, Xiong D, Guo L, et al. A remote quantitative Fugl-Meyer assessment framework for stroke patients based on wearable sensor networks. Comput Methods Programs Biomed, 2016, 128: 100-110.
26.	王跃, 郁磊, 傅建明, 等. 基于极限学习机的脑卒中上肢康复 Brunnstrom 远程智能评定系统. 生物医学工程学杂志, 2014, 31(2): 251-256.
27.	Ye F, Yang B, Nam C, et al. A data-driven investigation on surface electromyography based clinical assessment in chronic stroke. Front Neurorobot, 2021, 15: 648855.
28.	李素姣, 吴坤, 孟巧玲, 等. 脑卒中患者上肢功能智能评估系统研究进展. 生物医学工程学杂志, 2022, 39(3): 620-626.
29.	Zhang Z, Fang Q, Gu X. Objective assessment of upper-limb mobility for poststroke rehabilitation. IEEE Trans Biomed Eng, 2015, 63(4): 859-868.
30.	Rahman S, Sarker S, Haque A K M N, et al. AI-driven stroke rehabilitation systems and assessment: a systematic review. IEEE Trans Neural Syst Rehabil Eng, 2023, 31: 192-207.
31.	Woodward R B, Shefelbine S J, Vaidyanathan R. Pervasive monitoring of motion and muscle activation: Inertial and mechanomyography fusion. IEEE/ASME Trans Mechatron, 2017, 22(5): 2022-2033.
32.	Zhang X, Yue Z, Wang J. Robotics in lower-limb rehabilitation after stroke. Behav Neurol, 2017, 2017: 3731802.
33.	束一铭, 钱竞光, 戎科, 等. 偏瘫患者步态特征的动力学仿真分析. 医用生物力学, 2017, 32(6): 535-540.
34.	娄智, 姚博, 杨基海. 基于表面肌电信号的小儿脑瘫步态活动段检测研究. 生物医学工程学杂志, 2017, 34(3): 342-349.
35.	陈万鑫, 张弼, 张庆超等. 面向康复外骨骼的串联弹性关节设计与控制. 机器人, 2023, 45(5): 554-567.
36.	Hermens H J, Freriks B, Disselhorst-Klug C, et al. Development of recommendations for SEMG sensors and sensor placement procedures. J Electromyogr Kinesiol, 2000, 10(5): 361-374.
37.	陈玲玲, 张存, 宋晓伟, 等. 面向外骨骼机器人的肌肉功能网络构建及分析. 生物医学工程学杂志, 2019, 36(4): 565-572.
38.	Wen L, Xu J, Li D, et al. Continuous estimation of upper limb joint angle from sEMG based on multiple decomposition feature and BiLSTM network. Biomed Signal Process Control, 2023, 80: 104303.
39.	Sartori M, Llyod D G, Farina D. Neural data-driven musculoskeletal modeling for personalized neurorehabilitation technologies. IEEE Trans Biomed Eng, 2016, 63(5): 879-893.
40.	Zhang L, Li Z, Hu Y, et al. Ankle joint torque estimation using an EMG-driven neuromusculoskeletal model and an artificial neural network model. IEEE Trans Autom Sci Eng, 2021, 18(2): 564-573.
41.	Yang Y, Chen L, Pang J, et al. Validation of a spatiotemporal gait model using inertial measurement units for early-stage Parkinson’s disease detection during turns. IEEE Trans Biomed Eng, 2022, 69(12): 3591-3699.
42.	Ison M, Artemiadis P. The role of muscle synergies in myoelectric control: trends and challenges for simultaneous multifunction control. J Neural Eng, 2014, 11(5): 051001.
43.	Hong Y N G, Ballekere A N, Fregly B J, et al. Are muscle synergies useful for stroke rehabilitation?. Current Opinion in Biomedical Engineering, 2021, 19: 100315.
44.	Ivanenko Y P, Poppele R E, Lacquaniti F. Five basic muscle activation patterns account for muscle activity during human locomotion. J Physiol, 2004, 556: 267-282.
45.	Frère J, Hug F. Between-subject variability of muscle synergies during a complex motor skill. Front Comput Neurosci, 2012, 6: 99.
46.	Ryait H S, Arora A, Agarwal R. Interpretations of wrist/grip operations from SEMG signals at different locations on arm. IEEE Trans Biomed Circuits Syst, 2010, 4(2): 101-111.
47.	Yang J, Zhang D, Frangi A F, et al. Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell, 2004, 26(1): 131-137.
48.	Hao W. Classification of sport actions using principal component analysis and random forest based on three-dimensional data. Displays, 2022, 72: 102135.
49.	Krüger B, Vögele A, Willig T, et al. Efficient unsupervised temporal segmentation of motion data. IEEE Trans Multimedia, 2016, 19(4): 797-812.
50.	Li C, Zheng S Q, Prabhakaran B. Segmentation and recognition of motion streams by similarity search. ACM Trans Multimedia Comput Commun, 2007, 3(3): 16.

1. Zhou M, Wang H, Zeng X, et al. Mortality, morbidity, and risk factors in China and its provinces, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet, 2019, 394(10204): 1145-1158.
2. Hatem S M, Saussez G, Della Faille M, et al. Rehabilitation of motor function after stroke: a multiple systematic review focused on techniques to stimulate upper extremity recovery. Front Hum Neurosci, 2016, 10: 442.
3. 中华医学会神经病学分会神经康复学组, 中华医学会神经病学分会脑血管病学组, 卫生部脑卒中筛查与防治工程委员会办公室, 等. 中国脑卒中康复治疗指南(2011完全版). 中国康复理论与实践, 2012, 18(4): 301-318.
4. 国家卫生健康委员会. 2021中国卫生健康统计年鉴. 北京: 中国协和医科大学出版社, 2021.
5. Esquenazi A, Talaty M, Packel A, et al. The ReWalk powered exoskeleton to restore ambulatory function to individuals with thoracic-level motor-complete spinal cord injury. American Journal of Physical Medicine Rehabilitation, 2012, 91(11): 911-921.
6. Nilsson A, Vreede K S, Häglund V, et al. Gait training early after stroke with a new exoskeleton–the hybrid assistive limb: a study of safety and feasibility. J Neuroeng Rehabil, 2014, 11: 92.
7. Nam K Y, Kim H J, Kwon B S, et al. Robot-assisted gait training (Lokomat) improves walking function and activity in people with spinal cord injury: a systematic review. J Neuroeng Rehabil, 2017, 14(1): 24.
8. 刘京运. 从马拉松到冬奥会,大艾外骨骼机器人为残疾人创造更多可能. 机器人产业, 2022, 2022(4): 35-39.
9. 程洪, 黄瑞, 邱静, 等. 康复机器人及其临床应用综述. 机器人, 2021, 43(5): 606-619.
10. 王天. 一种下肢康复训练用外骨骼机器人. 浙江省: CN215193457U, 2021-12-17.
11. 丁逸苇, 涂利娟, 刘怡希, 等. 可穿戴式下肢外骨骼康复机器人研究进展. 机器人, 2022, 44(5): 522-532.
12. Zhang J, Fiers P, Witte K A, et al. Human-in-the-loop optimization of exoskeleton assistance during walking. Science, 2017, 356(6344): 1280-1284.
13. Gordon D F, Mcgreavy C, Christou A, et al. Human-in-the-loop optimization of exoskeleton assistance via online simulation of metabolic cost. IEEE Trans Rob, 2022, 38(3): 1410-1429.
14. Brunnstrom S. Motor testing procedures in hemiplegia: based on sequential recovery stages. Physical Therapy, 1966, 46(4): 357-375.
15. Sanford J, Moreland J, Swanson L R, et al. Reliability of the Fugl-Meyer assessment for testing motor performance in patients following stroke. Physical therapy, 1993, 73(7): 447-454.
16. Carr J H, Shepherd R B, Nordholm L, et al. Investigation of a new motor assessment scale for stroke patients. Physical therapy, 1985, 65(2): 175-180.
17. Kinoshita S, Abo M, Okamoto T. Effectiveness of ICF-based multidisciplinary rehabilitation approach with serial assessment and discussion using the ICF rehabilitation set in a convalescent rehabilitation ward. Int J Rehabil Res, 2020, 43(3): 255-260.
18. 姜荣荣, 陈艳, 潘翠环. 脑卒中后上肢和手运动功能康复评定的研究进展. 中国康复理论与实践, 2015, 21(10): 1173-1177.
19. Gao F, Wang L, Lin T. Intelligent wearable rehabilitation robot control system based on mobile communication network. Comput Commun, 2020, 153: 286-293.
20. Bai J, Song A. Development of a novel home based multi-scene upper limb rehabilitation training and evaluation system for post-stroke patients. IEEE Access, 2019, 7: 9667-9677.
21. Cerfoglio S, Ferraris C, Vismara L, et al. Kinect-based assessment of lower limbs during gait in post-stroke hemiplegic patients: a narrative review. Sensors, 2022, 22(13): 4910.
22. Qian C, Li W, Jia T, et al. Quantitative assessment of motor function by an end-effector upper limb rehabilitation robot based on admittance control. Appl Sci, 2021, 11(15): 6854.
23. Meziani Y, Morère Y, Hadj-Abdelkader A, et al. Towards adaptive and finer rehabilitation assessment: a learning framework for kinematic evaluation of upper limb rehabilitation on an Armeo Spring exoskeleton. Control Engineering Practice, 2021, 111: 104804.
24. Liparulo L, Zhang Z, Panella M, et al. A novel fuzzy approach for automatic Brunnstrom stage classification using surface electromyography. Med Biol Eng Comput, 2017, 55(8): 1367-1378.
25. Yu L, Xiong D, Guo L, et al. A remote quantitative Fugl-Meyer assessment framework for stroke patients based on wearable sensor networks. Comput Methods Programs Biomed, 2016, 128: 100-110.
26. 王跃, 郁磊, 傅建明, 等. 基于极限学习机的脑卒中上肢康复 Brunnstrom 远程智能评定系统. 生物医学工程学杂志, 2014, 31(2): 251-256.
27. Ye F, Yang B, Nam C, et al. A data-driven investigation on surface electromyography based clinical assessment in chronic stroke. Front Neurorobot, 2021, 15: 648855.
28. 李素姣, 吴坤, 孟巧玲, 等. 脑卒中患者上肢功能智能评估系统研究进展. 生物医学工程学杂志, 2022, 39(3): 620-626.
29. Zhang Z, Fang Q, Gu X. Objective assessment of upper-limb mobility for poststroke rehabilitation. IEEE Trans Biomed Eng, 2015, 63(4): 859-868.
30. Rahman S, Sarker S, Haque A K M N, et al. AI-driven stroke rehabilitation systems and assessment: a systematic review. IEEE Trans Neural Syst Rehabil Eng, 2023, 31: 192-207.
31. Woodward R B, Shefelbine S J, Vaidyanathan R. Pervasive monitoring of motion and muscle activation: Inertial and mechanomyography fusion. IEEE/ASME Trans Mechatron, 2017, 22(5): 2022-2033.
32. Zhang X, Yue Z, Wang J. Robotics in lower-limb rehabilitation after stroke. Behav Neurol, 2017, 2017: 3731802.
33. 束一铭, 钱竞光, 戎科, 等. 偏瘫患者步态特征的动力学仿真分析. 医用生物力学, 2017, 32(6): 535-540.
34. 娄智, 姚博, 杨基海. 基于表面肌电信号的小儿脑瘫步态活动段检测研究. 生物医学工程学杂志, 2017, 34(3): 342-349.
35. 陈万鑫, 张弼, 张庆超等. 面向康复外骨骼的串联弹性关节设计与控制. 机器人, 2023, 45(5): 554-567.
36. Hermens H J, Freriks B, Disselhorst-Klug C, et al. Development of recommendations for SEMG sensors and sensor placement procedures. J Electromyogr Kinesiol, 2000, 10(5): 361-374.
37. 陈玲玲, 张存, 宋晓伟, 等. 面向外骨骼机器人的肌肉功能网络构建及分析. 生物医学工程学杂志, 2019, 36(4): 565-572.
38. Wen L, Xu J, Li D, et al. Continuous estimation of upper limb joint angle from sEMG based on multiple decomposition feature and BiLSTM network. Biomed Signal Process Control, 2023, 80: 104303.
39. Sartori M, Llyod D G, Farina D. Neural data-driven musculoskeletal modeling for personalized neurorehabilitation technologies. IEEE Trans Biomed Eng, 2016, 63(5): 879-893.
40. Zhang L, Li Z, Hu Y, et al. Ankle joint torque estimation using an EMG-driven neuromusculoskeletal model and an artificial neural network model. IEEE Trans Autom Sci Eng, 2021, 18(2): 564-573.
41. Yang Y, Chen L, Pang J, et al. Validation of a spatiotemporal gait model using inertial measurement units for early-stage Parkinson’s disease detection during turns. IEEE Trans Biomed Eng, 2022, 69(12): 3591-3699.
42. Ison M, Artemiadis P. The role of muscle synergies in myoelectric control: trends and challenges for simultaneous multifunction control. J Neural Eng, 2014, 11(5): 051001.
43. Hong Y N G, Ballekere A N, Fregly B J, et al. Are muscle synergies useful for stroke rehabilitation?. Current Opinion in Biomedical Engineering, 2021, 19: 100315.
44. Ivanenko Y P, Poppele R E, Lacquaniti F. Five basic muscle activation patterns account for muscle activity during human locomotion. J Physiol, 2004, 556: 267-282.
45. Frère J, Hug F. Between-subject variability of muscle synergies during a complex motor skill. Front Comput Neurosci, 2012, 6: 99.
46. Ryait H S, Arora A, Agarwal R. Interpretations of wrist/grip operations from SEMG signals at different locations on arm. IEEE Trans Biomed Circuits Syst, 2010, 4(2): 101-111.
47. Yang J, Zhang D, Frangi A F, et al. Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell, 2004, 26(1): 131-137.
48. Hao W. Classification of sport actions using principal component analysis and random forest based on three-dimensional data. Displays, 2022, 72: 102135.
49. Krüger B, Vögele A, Willig T, et al. Efficient unsupervised temporal segmentation of motion data. IEEE Trans Multimedia, 2016, 19(4): 797-812.
50. Li C, Zheng S Q, Prabhakaran B. Segmentation and recognition of motion streams by similarity search. ACM Trans Multimedia Comput Commun, 2007, 3(3): 16.

Journal of Biomedical Engineering

Multi-modal synergistic quantitative analysis and rehabilitation assessment of lower limbs for exoskeleton

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content