This study aims to determine the salient brain regions with abnormal changes in white matter structures from diffusion tensor imaging (DTI) images of the patients with temporal lobe epilepsy (TLE), and to discriminate the patients with TLE from normal controls (NCs). Firstly, the DTI images from 50 subjects (28 NCs and 22 TLE) were acquired. Secondly, the four measures including the fractional anisotropy (FA), the mean diffusivity (MD), the axial diffusivity (AD) and the radial diffusivity (RD) were calculated. Thirdly, the tract-based spatial statistics (TBSS) was adopted to extract the measures in brain regions with significant differences between the two compared groups. Fourthly, the obtained measures were used as input features of the support vector machine (SVM) for classification, and the support vector machine-recursive feature elimination (SVM-RFE) was compared with the support vector machine-tract-based spatial statistics (SVM-TBSS) method. Finally, the essential brain regions and their spatial distribution were analyzed and discussed. The experimental results showed that the FA measures of the TLE group decreased significantly in the corpus callosum, superior longitudinal fasciculus, corona radiata, external capsule, internal capsule, inferior fronto-occipital fasciculus, fasciculus uncinatus and sagittal stratum, which were nearly bilaterally distributed, while the MD and RD increased significantly in most of these brain regions of the TLE group. Although the AD also increased, the differences were not statistically significant. The SVM-TBSS classifier obtained accuracies of 82%, 76% and 76% using the FA, MD and RD for classification, respectively, and 80% using combined measures. The SVM-RFE classifier obtained accuracies of 90%, 90% and 92% using the FA, MD and RD respectively, while the highest accuracy was 100% using combined measures. These results demonstrated that the SVM-RFE outperformed the SVM-TBSS, and the dominant characteristic influencing classification in brain regions were in associative and commissural fibers. These results illustrated that the measures of DTI images could reveal the abnormal changes in white matter structure of patients with TLE, providing effective information to clarify its pathological mechanism, localize the focus and diagnose automatically.