Vision is an important way for human beings to interact with the outside world and obtain information. In order to research human visual behavior under different conditions, this paper uses a Gaussian mixture-hidden Markov model (GMM-HMM) to model the scanpath, and proposes a new model optimization method, time-shifting segmentation (TSS). The TSS method can highlight the characteristics of the time dimension in the scanpath, improve the pattern recognition results, and enhance the stability of the model. In this paper, a linear discriminant analysis (LDA) method is used for multi-dimensional feature pattern recognition to evaluates the rationality and the accuracy of the proposed model. Four sets of comparative trials were carried out for the model evaluation. The first group applied the GMM-HMM to model the scanpath, and the average accuracy of the classification could reach 0.507, which is greater than the opportunity probability of three classification (0.333). The second set of trial applied TSS method, and the mean accuracy of classification was raised to 0.610. The third group combined GMM-HMM with TSS method, and the mean accuracy of classification reached 0.602, which was more stable than the second model. Finally, comparing the model analysis results with the saccade amplitude (SA) characteristics analysis results, the modeling analysis method is much better than the basic information analysis method. Via analyzing the characteristics of three types of tasks, the results show that the free viewing task have higher specificity value and a higher sensitivity to the cued object search task. In summary, the application of GMM-HMM model has a good performance in scanpath pattern recognition, and the introduction of TSS method can enhance the difference of scanpath characteristics. Especially for the recognition of the scanpath of search-type tasks, the model has better advantages. And it also provides a new solution for a single state eye movement sequence.