The ultrasound Doppler fetal heart rate measurement is the gold standard of fetal heart rate counting. However, the existing fetal heart rate extraction algorithms are not designed specifically to suppress the high maternal interference during the second stage of labor, and false detection occurrences are common during labor. With this background, a method combining time-frequency frame template library optimal selecting and non-linear template matching is proposed. The method contributes a template library, and the optimal template can be selected to match the signal frame. After the short-time Fourier transform of the signal, the difference between the signal and the template is optimized by leaky rectified linear unit (LReLU) function frame by frame. The heart rate was calculated from the peak of the matching curve and the heart rate was calculated. By comparing the proposed method with the autocorrelation method, the results show that the detection accuracy of the proposed method is improved by 20% on average, and the non-linear template matching of 23% samples is at least 50% higher than the autocorrelation method. This paper designs the algorithm by analyzing the characteristics of the interference and signal mixing. We hope that this paper will provide a new idea for fetal heart rate extraction which not only focuses on the original signal.