Research progress and application of transfer entropy algorithm_Journal of Biomedical Engineering

Authors：

LI Tianxiang ^1,2,3 ,  LI Shuangyan ^1,2,3

1. State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Electrical Engineering, Hebei University of Technology, Tianjin 300130, P. R. China;
2. Key Laboratory of Electromagnetic Field and Electrical Apparatus Reliability of Hebei Province, School of Electrical Engineering, Hebei University of Technology, Tianjin 300130, P. R. China;
3. Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Hebei University of Technology, Tianjin 300130, P. R. China;

Corresponding author：

Keywords：

Transfer entropy; Brain functional network; Neural signal analysis

DOI：

10.7507/1001-5515.202109067

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

In recent years, exploring the physiological and pathological mechanisms of brain functional integration from the neural network level has become one of the focuses of neuroscience research. Due to the non-stationary and nonlinear characteristics of neural signals, its linear characteristics are not sufficient to fully explain the potential neurophysiological activity mechanism in the implementation of complex brain functions. In order to overcome the limitation that the linear algorithm cannot effectively analyze the nonlinear characteristics of signals, researchers proposed the transfer entropy (TE) algorithm. In recent years, with the introduction of the concept of brain functional network, TE has been continuously optimized as a powerful tool for nonlinear time series multivariate analysis. This paper first introduces the principle of TE algorithm and the research progress of related improved algorithms, discusses and compares their respective characteristics, and then summarizes the application of TE algorithm in the field of electrophysiological signal analysis. Finally, combined with the research progress in recent years, the existing problems of TE are discussed, and the future development direction is prospected.

Citation： LI Tianxiang, LI Shuangyan. Research progress and application of transfer entropy algorithm. Journal of Biomedical Engineering, 2022, 39(3): 612-619. doi: 10.7507/1001-5515.202109067 Copy

1.	覃国萍, 李双燕. 多尺度熵算法研究进展及其在神经信号分析中的应用. 生物医学工程学杂志, 2020, 37(3): 541-548.
2.	Chen Xiangbin, Liu Mengting, Wu Zhibing, et al. Topological abnormalities of functional brain network in early-stage parkinson's disease patients with mild cognitive impairment. Front Neurosci-Switz, 2020, 14: 616872.
3.	Li Fei, Wang Xuewei, Lin Qiang, et al. Unified model selection approach based on minimum description length principle in granger causality analysis. IEEE Access, 2020, 8: 68400-68416.
4.	Olejarczyk E, Zuchowicz U, Wozniak-Kwasniewska A, et al. The impact of repetitive transcranial magnetic stimulation on functional connectivity in major depressive disorder and bipolar disorder evaluated by directed transfer function and indices based on graph theory. Int J Neural Syst, 2020, 30(4): 2050015.
5.	Jafarian A, Litvak V, Cagnan H, et al. Comparing dynamic causal models of neurovascular coupling with fMRI and EEG/MEG. Neuroimage, 2020, 216(2-3): 116734.
6.	Nalatore H, Sasikumar N, Rangarajan G. Effect of measurement noise on granger causality. Phys Rev E, 2014, 90(6-1): 062127.
7.	Gong Xiajing, Li Wu, Liang Hualou. Spike-field granger causality for hybrid neural data analysis. J Neurophysiol, 2019, 122(2): 809-822.
8.	Fei He, Yuan Yang. Nonlinear system identification of neural systems from neurophysiological signals. Neuroscience, 2021, 458: 213-228.
9.	Joko O, Vidakovic M R, Lorincz J, et al. A novel latency estimation algorithm of motor evoked potential signals. IEEE Access, 2020, 8: 193356-193374.
10.	Talib I, Sundaraj K, Lam C K, et al. A review on crosstalk in myographic signals. Eur J Appl Physiol, 2018, 119(1): 9-28.
11.	Schreiber T. Measuring information transfer. Phys Rev Lett, 2000, 85(2): 461-464.
12.	Novelli L, Lizier J T. Inferring network properties from time series via transfer entropy and mutual information: validation of bivariate versus multivariate approaches. Netw Neurosci, 5(2): 373-404.
13.	Mao Xuegeng, Shang Pengjian. Transfer entropy between multivariate time series. Commun Nonlinear Sci, 2016, 47: 338-347.
14.	Cao Jianqin, Li Yang, Yu Hong, et al. Investigation of brain networks in children with attention deficit/hyperactivity disorder using a graph theoretical approach. Biomed Signal Proces, 2018, 40(1): 351-358.
15.	Olejarczyk E, Marzetti L, Pizzella V, et al. Comparison of connectivity analyses for resting state EEG data. J Neural Eng, 2017, 14(3): 036017.
16.	Wang Tingting, Yang Juan, Song Yingjie, et al. Interactions of central and autonomic nervous systems in patients with sleep apnea-hypopnea syndrome during sleep. Sleep Breath, 2021. DOI: 10.1007/s11325-021-02429-6.
17.	Zhang Yuanyuan, Chen Xiaoling, Pang Xiaohui, et al. Multiscale multivariate transfer entropy and application to functional corticocortical coupling. J Neural Eng, 2021, 18(4): 046056.
18.	Bing Pingping, Liu Wei, Wang Zhong, et al. Noise reduction in ECG signal using an effective hybrid scheme. IEEE Access, 2020, 8: 160790-160801.
19.	Staniek M, Lehnertz K. Symbolic transfer entropy: inferring directionality in biosignals. Nat Biomed Eng, 2009, 54(6): 323-328.
20.	Zhang Yu, Zhang Yiling, Chen Junwei, et al. Magnetoencephalogram analysis of depression based on multivariable sign transfer entropy. J Phys Conf Ser, 2020, 1592(1): 012093.
21.	Wessel N, Ziehmann C, Kurths J, et al. Short-term forecasting of life-threatening cardiac arrhythmias based on symbolic dynamics and finite-time growth rates. Phys Rev A, 2000, 61(1): 733-739.
22.	Ye S, Kitago K, Kitano K. Information-theoretic approach to detect directional information flow in EEG signals induced by TMS. Neurosci Res, 2020, 156(1): 197-205.
23.	Wang Yalin, Chen Wei. Effective brain connectivity for fNIRS data analysis based on multi-delays symbolic phase transfer entropy. J Neural Eng, 2020, 17(5): 056024.
24.	Fang Pan, Dai Liming, Hou Yongjun, et al. The study of identification method for dynamic behavior of high-dimensional nonlinear system. Shock Vib, 2019, 2: 1-9.
25.	Porta A, Bari V, Cario B, et al. Comparison of symbolization strategies for complexity assessment of spontaneous variability in individuals with signs of cardiovascular control impairment. Biomed Signal Proces, 2020, 62(1): 102128.
26.	Zhang Boyi, Shang Pengjian. Measuring information transfer by dispersion transfer entropy. Commun Nonlinear Sci, 2020, 89(2): 105329.
27.	Rostaghi M, Azami H. Dispersion entropy: a measure for time-series analysis. IEEE Signal Proc Lett, 23(5): 610-614.
28.	Rogers N, Thunemann M, Devor A, et al. Impact of brain surface boundary conditions on electrophysiology and implications for electrocorticography. Front Neurosci-Switz, 2020, 14(1): 763.
29.	Faes L, Erla S, Nollo G. Compensating for instantaneous signal mixing in transfer entropy analysis of neurobiological time series// Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Shenzhen: IEEE, 2012: 3672-3675.
30.	Gu Danlei, Mi Yujia, Lin Aijing. Application of time-delay multiscale symbolic phase compensated transfer entropy in analyzing cyclic alternating pattern (CAP) in sleep-related pathological data. Commun Nonlinear Sci, 2021, 99(9): 105835.
31.	Mckenna D, Peever J. Degeneration of rapid eye movement sleep circuitry underlies rapid eye movement sleep behavior disorder. Mov Disord, 2017, 32(5): 636-644.
32.	Wakasugi N, Togo H, Mukai Y, et al. Prefrontal network dysfunctions in rapid eye movement sleep behavior disorder. Parkinsonism Relat D, 2021, 85: 72-77.
33.	Fonseca P, Gilst M, Radha M, et al. Automatic sleep staging using heart rate variability, body movements, and recurrent neural networks in a sleep disordered population. Sleep, 2020, 43(9): zsaa048.
34.	Sammani F, Sim K S, Tan S C. Encoding rich frequencies for classification of stroke patients EEG signals. IEEE Access, 2020, 8: 135811-135820.
35.	张园园, 邹策, 陈晓玲. 基于Gabor小波-传递熵的脑-肌电信号同步耦合分析. 生物医学工程学杂志, 2017, 34(6): 850-856.
36.	Valenza G, Fase L, Citi L, et al. Instantaneous transfer entropy for the study of cardiovascular and cardio-respiratory nonstationary dynamics. IEEE Trans Biomed Eng, 2018, 65(5): 1077-1085.
37.	Blanchard M, Bennouna M A. The representation power of neural networks: breaking the curse of dimensionality. ArXiv preprint, 2020: arxiv-2012.05451.
38.	Mostafa S S, Baptista D, Garcia A G, et al. Greedy based convolutional neural network optimization for detecting apnea. Comput Meth Prog Bio, 2020, 197: 105640.
39.	Han Chunxiao, Sun Xiaozhou, Yang Yaru, et al. Brain complex network characteristic analysis of fatigue during simulated driving based on electroencephalogram signals. Entropy-Switz, 2019, 21(4): 353.
40.	Runge J. Causal network reconstruction from time series: From theoretical assumptions to practical estimation. Chaos, 2019, 28(7): 075310.
41.	Kim P, Rogers J, Sun J, et al. Causation entropy identifies sparsity structure for parameter estimation of dynamic systems. J Comput Nonlin Dyn, 2016, 12: 011008.
42.	Wollstadt P, Lizier J T, Vicente R, et al. IDTxl: the information dynamics toolkit xl: a python package for the efficient analysis of multivariate information dynamics in networks. J Syst Softw, 2019, 4(34): 1081.
43.	Yin Guimei, Li Haifang, Tan Shuping, et al. Synchronization stability model of complex brain networks: an EEG study. Front Psychiatry, 2020, 11: 571068.
44.	Li Haifang, Yao Rong, Xia Xiaoluan, et al. Adjustment of synchronization stability of dynamic brain-networks based on feature fusion. Front Hum Neurosci, 2019, 13: 98.
45.	Harmah D J, Li C, Li F, et al. Measuring the non-linear directed information flow in schizophrenia by multivariate transfer entropy. Front Comput Neurosc, 2020, 13: 85.
46.	Li Zhaohui, Li Shuaifei, Yu Tao, et al. Measuring the coupling direction between neural oscillations with weighted symbolic transfer entropy. Entropy-Switz, 2020, 22(12): 1442.
47.	Ciprian C, Masychev K, Ravan M, et al. Diagnosing schizophrenia using effective connectivity of resting-state EEG data. Algorithms, 2021, 14(5): 139.
48.	Wu Zhanxiong, Chen Xnmin, Gao Mingyu, et al. Effective connectivity extracted from resting-state fMRI images using transfer entropy. IRBM, 2021, 42(6): 457-465.
49.	Moldovan A, Caţaron A, Andonie R. Learning in feedforward neural networks accelerated by transfer entropy. Entropy-Switz. 2020, 22(1): 102.
50.	Dourado J R, Junior J, Maciel C. Parallelism strategies for big data delayed transfer entropy evaluation. Algorithms, 2019, 12(9): 190.
51.	Silini R, Masoller C. Fast and effective pseudo transfer entropy for bivariate data-driven causal inference. Sci Rep-UK, 2021, 11(1): 8423.
52.	Li Zhaohui, Gao Mengyu, Wang Yongtian. The orientation selectivity of spike-LFP synchronization in macaque V1 and V4. Front Comput Neurosc, 2019, 13: 47.

1. 覃国萍, 李双燕. 多尺度熵算法研究进展及其在神经信号分析中的应用. 生物医学工程学杂志, 2020, 37(3): 541-548.
2. Chen Xiangbin, Liu Mengting, Wu Zhibing, et al. Topological abnormalities of functional brain network in early-stage parkinson's disease patients with mild cognitive impairment. Front Neurosci-Switz, 2020, 14: 616872.
3. Li Fei, Wang Xuewei, Lin Qiang, et al. Unified model selection approach based on minimum description length principle in granger causality analysis. IEEE Access, 2020, 8: 68400-68416.
4. Olejarczyk E, Zuchowicz U, Wozniak-Kwasniewska A, et al. The impact of repetitive transcranial magnetic stimulation on functional connectivity in major depressive disorder and bipolar disorder evaluated by directed transfer function and indices based on graph theory. Int J Neural Syst, 2020, 30(4): 2050015.
5. Jafarian A, Litvak V, Cagnan H, et al. Comparing dynamic causal models of neurovascular coupling with fMRI and EEG/MEG. Neuroimage, 2020, 216(2-3): 116734.
6. Nalatore H, Sasikumar N, Rangarajan G. Effect of measurement noise on granger causality. Phys Rev E, 2014, 90(6-1): 062127.
7. Gong Xiajing, Li Wu, Liang Hualou. Spike-field granger causality for hybrid neural data analysis. J Neurophysiol, 2019, 122(2): 809-822.
8. Fei He, Yuan Yang. Nonlinear system identification of neural systems from neurophysiological signals. Neuroscience, 2021, 458: 213-228.
9. Joko O, Vidakovic M R, Lorincz J, et al. A novel latency estimation algorithm of motor evoked potential signals. IEEE Access, 2020, 8: 193356-193374.
10. Talib I, Sundaraj K, Lam C K, et al. A review on crosstalk in myographic signals. Eur J Appl Physiol, 2018, 119(1): 9-28.
11. Schreiber T. Measuring information transfer. Phys Rev Lett, 2000, 85(2): 461-464.
12. Novelli L, Lizier J T. Inferring network properties from time series via transfer entropy and mutual information: validation of bivariate versus multivariate approaches. Netw Neurosci, 5(2): 373-404.
13. Mao Xuegeng, Shang Pengjian. Transfer entropy between multivariate time series. Commun Nonlinear Sci, 2016, 47: 338-347.
14. Cao Jianqin, Li Yang, Yu Hong, et al. Investigation of brain networks in children with attention deficit/hyperactivity disorder using a graph theoretical approach. Biomed Signal Proces, 2018, 40(1): 351-358.
15. Olejarczyk E, Marzetti L, Pizzella V, et al. Comparison of connectivity analyses for resting state EEG data. J Neural Eng, 2017, 14(3): 036017.
16. Wang Tingting, Yang Juan, Song Yingjie, et al. Interactions of central and autonomic nervous systems in patients with sleep apnea-hypopnea syndrome during sleep. Sleep Breath, 2021. DOI: 10.1007/s11325-021-02429-6.
17. Zhang Yuanyuan, Chen Xiaoling, Pang Xiaohui, et al. Multiscale multivariate transfer entropy and application to functional corticocortical coupling. J Neural Eng, 2021, 18(4): 046056.
18. Bing Pingping, Liu Wei, Wang Zhong, et al. Noise reduction in ECG signal using an effective hybrid scheme. IEEE Access, 2020, 8: 160790-160801.
19. Staniek M, Lehnertz K. Symbolic transfer entropy: inferring directionality in biosignals. Nat Biomed Eng, 2009, 54(6): 323-328.
20. Zhang Yu, Zhang Yiling, Chen Junwei, et al. Magnetoencephalogram analysis of depression based on multivariable sign transfer entropy. J Phys Conf Ser, 2020, 1592(1): 012093.
21. Wessel N, Ziehmann C, Kurths J, et al. Short-term forecasting of life-threatening cardiac arrhythmias based on symbolic dynamics and finite-time growth rates. Phys Rev A, 2000, 61(1): 733-739.
22. Ye S, Kitago K, Kitano K. Information-theoretic approach to detect directional information flow in EEG signals induced by TMS. Neurosci Res, 2020, 156(1): 197-205.
23. Wang Yalin, Chen Wei. Effective brain connectivity for fNIRS data analysis based on multi-delays symbolic phase transfer entropy. J Neural Eng, 2020, 17(5): 056024.
24. Fang Pan, Dai Liming, Hou Yongjun, et al. The study of identification method for dynamic behavior of high-dimensional nonlinear system. Shock Vib, 2019, 2: 1-9.
25. Porta A, Bari V, Cario B, et al. Comparison of symbolization strategies for complexity assessment of spontaneous variability in individuals with signs of cardiovascular control impairment. Biomed Signal Proces, 2020, 62(1): 102128.
26. Zhang Boyi, Shang Pengjian. Measuring information transfer by dispersion transfer entropy. Commun Nonlinear Sci, 2020, 89(2): 105329.
27. Rostaghi M, Azami H. Dispersion entropy: a measure for time-series analysis. IEEE Signal Proc Lett, 23(5): 610-614.
28. Rogers N, Thunemann M, Devor A, et al. Impact of brain surface boundary conditions on electrophysiology and implications for electrocorticography. Front Neurosci-Switz, 2020, 14(1): 763.
29. Faes L, Erla S, Nollo G. Compensating for instantaneous signal mixing in transfer entropy analysis of neurobiological time series// Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Shenzhen: IEEE, 2012: 3672-3675.
30. Gu Danlei, Mi Yujia, Lin Aijing. Application of time-delay multiscale symbolic phase compensated transfer entropy in analyzing cyclic alternating pattern (CAP) in sleep-related pathological data. Commun Nonlinear Sci, 2021, 99(9): 105835.
31. Mckenna D, Peever J. Degeneration of rapid eye movement sleep circuitry underlies rapid eye movement sleep behavior disorder. Mov Disord, 2017, 32(5): 636-644.
32. Wakasugi N, Togo H, Mukai Y, et al. Prefrontal network dysfunctions in rapid eye movement sleep behavior disorder. Parkinsonism Relat D, 2021, 85: 72-77.
33. Fonseca P, Gilst M, Radha M, et al. Automatic sleep staging using heart rate variability, body movements, and recurrent neural networks in a sleep disordered population. Sleep, 2020, 43(9): zsaa048.
34. Sammani F, Sim K S, Tan S C. Encoding rich frequencies for classification of stroke patients EEG signals. IEEE Access, 2020, 8: 135811-135820.
35. 张园园, 邹策, 陈晓玲. 基于Gabor小波-传递熵的脑-肌电信号同步耦合分析. 生物医学工程学杂志, 2017, 34(6): 850-856.
36. Valenza G, Fase L, Citi L, et al. Instantaneous transfer entropy for the study of cardiovascular and cardio-respiratory nonstationary dynamics. IEEE Trans Biomed Eng, 2018, 65(5): 1077-1085.
37. Blanchard M, Bennouna M A. The representation power of neural networks: breaking the curse of dimensionality. ArXiv preprint, 2020: arxiv-2012.05451.
38. Mostafa S S, Baptista D, Garcia A G, et al. Greedy based convolutional neural network optimization for detecting apnea. Comput Meth Prog Bio, 2020, 197: 105640.
39. Han Chunxiao, Sun Xiaozhou, Yang Yaru, et al. Brain complex network characteristic analysis of fatigue during simulated driving based on electroencephalogram signals. Entropy-Switz, 2019, 21(4): 353.
40. Runge J. Causal network reconstruction from time series: From theoretical assumptions to practical estimation. Chaos, 2019, 28(7): 075310.
41. Kim P, Rogers J, Sun J, et al. Causation entropy identifies sparsity structure for parameter estimation of dynamic systems. J Comput Nonlin Dyn, 2016, 12: 011008.
42. Wollstadt P, Lizier J T, Vicente R, et al. IDTxl: the information dynamics toolkit xl: a python package for the efficient analysis of multivariate information dynamics in networks. J Syst Softw, 2019, 4(34): 1081.
43. Yin Guimei, Li Haifang, Tan Shuping, et al. Synchronization stability model of complex brain networks: an EEG study. Front Psychiatry, 2020, 11: 571068.
44. Li Haifang, Yao Rong, Xia Xiaoluan, et al. Adjustment of synchronization stability of dynamic brain-networks based on feature fusion. Front Hum Neurosci, 2019, 13: 98.
45. Harmah D J, Li C, Li F, et al. Measuring the non-linear directed information flow in schizophrenia by multivariate transfer entropy. Front Comput Neurosc, 2020, 13: 85.
46. Li Zhaohui, Li Shuaifei, Yu Tao, et al. Measuring the coupling direction between neural oscillations with weighted symbolic transfer entropy. Entropy-Switz, 2020, 22(12): 1442.
47. Ciprian C, Masychev K, Ravan M, et al. Diagnosing schizophrenia using effective connectivity of resting-state EEG data. Algorithms, 2021, 14(5): 139.
48. Wu Zhanxiong, Chen Xnmin, Gao Mingyu, et al. Effective connectivity extracted from resting-state fMRI images using transfer entropy. IRBM, 2021, 42(6): 457-465.
49. Moldovan A, Caţaron A, Andonie R. Learning in feedforward neural networks accelerated by transfer entropy. Entropy-Switz. 2020, 22(1): 102.
50. Dourado J R, Junior J, Maciel C. Parallelism strategies for big data delayed transfer entropy evaluation. Algorithms, 2019, 12(9): 190.
51. Silini R, Masoller C. Fast and effective pseudo transfer entropy for bivariate data-driven causal inference. Sci Rep-UK, 2021, 11(1): 8423.
52. Li Zhaohui, Gao Mengyu, Wang Yongtian. The orientation selectivity of spike-LFP synchronization in macaque V1 and V4. Front Comput Neurosc, 2019, 13: 47.

Previous Article
Key technology of brain-computer interaction based on speech imagery
Next Article
Research progress on intelligent assessment system for upper limb function of stroke patients

Journal of Biomedical Engineering

Research progress and application of transfer entropy algorithm

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content