MAO Bochun 1 , CHEN Shengkai 1 , XIE Yu 1 , YAO Pan 2 , LI Chunjie 1,3
  • 1. State Key Laboratory of Oral Disease, National Clinical Research Center for Oral Diseases, West China Hospital of Stomatology, Sichuan University, Chengdu, 610041, P.R.China;
  • 2. College of Computer Science, Sichuan University, Chengdu, 610065, P.R.China;
  • 3. Department of Oral and Maxillofacial Surgery, Department of Evidence Based Dentistry, West China Hospital of Stomatology, Sichuan University, Chengdu, 610041, P.R.China;
LI Chunjie, Email: lichunjie07@qq.com
Deep learning; Randomized controlled trial; Literature screening; Evidence-based medicine
10.7507/1672-2531.201906050
Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables References Cited by

Objectives To explore the effect of the deep learning algorithm convolutional neural network (CNN) in screening of randomized controlled trials (RCTs) in Chinese medical literatures.Methods Literature with the topic " oral science” published in 2014 were retrieved from CNKI and exported citations containing title and abstract. RCTs screening was conducted by double independent screening, checking and peer discussion. The final results of the citations were used for CNN algorithm model training. After completing the algorithm model training, a prospective comparative trial was organized by searching all literature with the topic "oral science" published in CNKI from January to March 2018 to compare the sensitivity (SEN) and specificity (SPE) of algorithm with manual screening. The initial results of a single screener represented the performance of manual screening, and the final results after peer discussion were used as the gold standard. The best thresholds of algorithm were determined with the receptor operative characteristic (ROC) curve.Results A total of 1 246 RCTs and 4 754 non-RCTs were eventually included for training and testing of CNN algorithm model. 249 RCTs and 949 non-RCTs were included in the prospective trial. The SEN and SPE of manual screening were 98.01% and 98.82%. For the algorithm model, the SEN of RCTs screening decreased with the increase of threshold value while the SPE increased with the increase of threshold value. After 27 changes of threshold value, ROC curve were obtained. The area under the ROC curve was 0.9977, unveiling the optimal accuracy threshold (Threshold=0.4, SEN=98.39%, SPE=98.84%) and high sensitivity threshold (Threshold=0.06, SEN=99.60%, SPE=94.10%).Conclusions A CNN algorithm model is trained with Chinese RCTs classification database established in this study and shows an excellent classification performance in screening RCTs of Chinese medical literature, which is proved to be comparable to the manual screening performance in the prospective controlled trial.

Citation： MAO Bochun, CHEN Shengkai, XIE Yu, YAO Pan, LI Chunjie. Exploration of classical deep learning algorithm in intelligent classification of Chinese randomized controlled trials. Chinese Journal of Evidence-Based Medicine, 2019, 19(11): 1262-1267. doi: 10.7507/1672-2531.201906050 Copy

  • Previous Article

    Strengthening clinical research source data management in hospitals to promote data quality of clinical research in China

  • Next Article

    The prevalence of high risk human papillomavirus of females in western China: a meta-analysis