A Gaussian mixture-hidden Markov model of human visual behavior_Journal of Biomedical Engineering

Authors：

LIU Huaqian ¹ ,  ZHENG Xiujuan ¹ , WANG Yan ¹ , ZHANG Yun ² , LIU Kai ¹

1. School of Electrical Engineering, Sichuan University, Chengdu 610065, P.R.China;
2. School of Electronics and Information Engineering, Xi'an Jiaotong University, Xi’an 710049, P.R.China;

Corresponding author：

ZHENG Xiujuan, Email: xiujuanzheng@scu.edu.cn

Keywords：

Gaussian mixture-hidden Markov model; scanpath; time-shifting segmentation method; pattern recognition; visual behavior

DOI：

10.7507/1001-5515.202008022

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Vision is an important way for human beings to interact with the outside world and obtain information. In order to research human visual behavior under different conditions, this paper uses a Gaussian mixture-hidden Markov model (GMM-HMM) to model the scanpath, and proposes a new model optimization method, time-shifting segmentation (TSS). The TSS method can highlight the characteristics of the time dimension in the scanpath, improve the pattern recognition results, and enhance the stability of the model. In this paper, a linear discriminant analysis (LDA) method is used for multi-dimensional feature pattern recognition to evaluates the rationality and the accuracy of the proposed model. Four sets of comparative trials were carried out for the model evaluation. The first group applied the GMM-HMM to model the scanpath, and the average accuracy of the classification could reach 0.507, which is greater than the opportunity probability of three classification (0.333). The second set of trial applied TSS method, and the mean accuracy of classification was raised to 0.610. The third group combined GMM-HMM with TSS method, and the mean accuracy of classification reached 0.602, which was more stable than the second model. Finally, comparing the model analysis results with the saccade amplitude (SA) characteristics analysis results, the modeling analysis method is much better than the basic information analysis method. Via analyzing the characteristics of three types of tasks, the results show that the free viewing task have higher specificity value and a higher sensitivity to the cued object search task. In summary, the application of GMM-HMM model has a good performance in scanpath pattern recognition, and the introduction of TSS method can enhance the difference of scanpath characteristics. Especially for the recognition of the scanpath of search-type tasks, the model has better advantages. And it also provides a new solution for a single state eye movement sequence.

Citation： LIU Huaqian, ZHENG Xiujuan, WANG Yan, ZHANG Yun, LIU Kai. A Gaussian mixture-hidden Markov model of human visual behavior. Journal of Biomedical Engineering, 2021, 38(3): 512-519. doi: 10.7507/1001-5515.202008022 Copy

1.	Kollmorgen S, Nortmann N, Schröder S, et al. Influence of low-level stimulus features, task dependent factors, and spatial biases on overt visual attention. PLoS Comput Biol, 2010, 6(5): e1000791.
2.	闫国利, 熊建萍, 臧传丽, 等. 阅读研究中的主要眼动指标评述. 心理科学进展, 2013, 21(4): 589-605.
3.	Henderson J M, Shinkareva S V, Wang J, et al. Predicting cognitive state from eye movements. PLoS One, 2013, 8(5): e64937.
4.	Kang Z, Landry S J. An eye movement analysis algorithm for a multielement target tracking task: maximum transition-based agglomerative hierarchical clustering. IEEE Trans Hum Mach Syst, 2015, 45(1): 13-24.
5.	Mcclung S N, Kang Z. Characterization of visual scanning patterns in air traffic control. Comput Intell Neurosci, 2016, 2016: 8343842.
6.	Chihara T, Kobayashi F, Sakamoto J. Evaluation of mental workload during automobile driving using one-class support vector machine with eye movement data. Appl Ergon, 2020, 89: 103201.
7.	Itti L. New eye-tracking techniques may revolutionize mental health screening. Neuron, 2015, 88(3): 442-444.
8.	Wang S, Jiang M, Duchesne X M, et al. Atypical visual saliency in autism spectrum disorder quantified through model-based eye tracking. Neuron, 2015, 88(3): 604-616.
9.	Crabb D P, Smith N D, ZHU H. What's on TV? Detecting age-related neurodegenerative eye disease using eye movement scanpaths. Front Aging Neurosci, 2014, 6: 312.
10.	Caldara R, Miellet S. iMap: a novel method for statistical fixation mapping of eye movement data. Behav Res Methods, 2011, 43(3): 864-878.
11.	Cristino F, Mathôt S, Theeuwes J, et al. ScanMatch: a novel method for comparing fixation sequences. Behav Res Methods, 2010, 42(3): 692-700.
12.	Kübler T C, Kasneci E, Rosenstiel W. SubsMatch: scanpath similarity in dynamic scenes based on subsequence frequencies//Eye Tracking Research and Applications Symposium, Florida: Association For Computing Machinery, 2014: 319-322.
13.	Kübler T C, Rothe C, Schiefer U, et al. SubsMatch 2.0: scanpath comparison and classification based on subsequence frequencies. Behav Res Methods, 2017, 49(3): 1048-1064.
14.	Couronne T, Guerindugue A, Dubois M, et al. A statistical mixture method to reveal bottom-up and top-down factors guiding the eye-movements. J Eye Mov Res, 2010, 3(2): 1-13.
15.	梁铭炬. 基于嵌入式隐马尔科夫模型的人眼注视方向识别. 电子技术与软件工程, 2019, 163(17): 182-183.
16.	Coutrot A, Hsiao J H, Chan A B. Scanpath modeling and classification with hidden Markov models. Behav Res Methods, 2018, 50(1): 362-379.
17.	Koehler K, Guo F, Zhang S, et al. What do saliency models predict?. Journal of Vision, 2014, 14(3): 14.
18.	University of California, Vision & Image Understanding. Mendeley Data. (2020.8.27)[2020.9.1]. https://data.mendeley.com/datasets/8rj98pp6km/1.

1. Kollmorgen S, Nortmann N, Schröder S, et al. Influence of low-level stimulus features, task dependent factors, and spatial biases on overt visual attention. PLoS Comput Biol, 2010, 6(5): e1000791.
2. 闫国利, 熊建萍, 臧传丽, 等. 阅读研究中的主要眼动指标评述. 心理科学进展, 2013, 21(4): 589-605.
3. Henderson J M, Shinkareva S V, Wang J, et al. Predicting cognitive state from eye movements. PLoS One, 2013, 8(5): e64937.
4. Kang Z, Landry S J. An eye movement analysis algorithm for a multielement target tracking task: maximum transition-based agglomerative hierarchical clustering. IEEE Trans Hum Mach Syst, 2015, 45(1): 13-24.
5. Mcclung S N, Kang Z. Characterization of visual scanning patterns in air traffic control. Comput Intell Neurosci, 2016, 2016: 8343842.
6. Chihara T, Kobayashi F, Sakamoto J. Evaluation of mental workload during automobile driving using one-class support vector machine with eye movement data. Appl Ergon, 2020, 89: 103201.
7. Itti L. New eye-tracking techniques may revolutionize mental health screening. Neuron, 2015, 88(3): 442-444.
8. Wang S, Jiang M, Duchesne X M, et al. Atypical visual saliency in autism spectrum disorder quantified through model-based eye tracking. Neuron, 2015, 88(3): 604-616.
9. Crabb D P, Smith N D, ZHU H. What's on TV? Detecting age-related neurodegenerative eye disease using eye movement scanpaths. Front Aging Neurosci, 2014, 6: 312.
10. Caldara R, Miellet S. iMap: a novel method for statistical fixation mapping of eye movement data. Behav Res Methods, 2011, 43(3): 864-878.
11. Cristino F, Mathôt S, Theeuwes J, et al. ScanMatch: a novel method for comparing fixation sequences. Behav Res Methods, 2010, 42(3): 692-700.
12. Kübler T C, Kasneci E, Rosenstiel W. SubsMatch: scanpath similarity in dynamic scenes based on subsequence frequencies//Eye Tracking Research and Applications Symposium, Florida: Association For Computing Machinery, 2014: 319-322.
13. Kübler T C, Rothe C, Schiefer U, et al. SubsMatch 2.0: scanpath comparison and classification based on subsequence frequencies. Behav Res Methods, 2017, 49(3): 1048-1064.
14. Couronne T, Guerindugue A, Dubois M, et al. A statistical mixture method to reveal bottom-up and top-down factors guiding the eye-movements. J Eye Mov Res, 2010, 3(2): 1-13.
15. 梁铭炬. 基于嵌入式隐马尔科夫模型的人眼注视方向识别. 电子技术与软件工程, 2019, 163(17): 182-183.
16. Coutrot A, Hsiao J H, Chan A B. Scanpath modeling and classification with hidden Markov models. Behav Res Methods, 2018, 50(1): 362-379.
17. Koehler K, Guo F, Zhang S, et al. What do saliency models predict?. Journal of Vision, 2014, 14(3): 14.
18. University of California, Vision & Image Understanding. Mendeley Data. (2020.8.27)[2020.9.1]. https://data.mendeley.com/datasets/8rj98pp6km/1.

Previous Article
Bacomics——a new discipline integrating brain and the outside
Next Article
Monitoring microwave ablation using ultrasound backscatter homodyned K imaging: Comparison of estimators

Journal of Biomedical Engineering

A Gaussian mixture-hidden Markov model of human visual behavior

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content