Machine learning-based diagnostic test accuracy (1): study design_Chinese Journal of Evidence-Based Medicine

Authors：

ZHANG Yunan ¹ , ZHU Tao ¹ , ZENG Wei ¹ , GUO Jixiang ² , ZHANG Tao ³ , GOU Pan ⁴ ,  TANG Wei ¹ ,  LIU Chang ^1,4

1. State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases & Department of Oral and Maxillofacial Surgery & Department of Evidence-Based Dentistry & "Medicine+Manufacturing" Center, West China Hospital of Stomatology, Sichuan University, Chengdu 610041, P. R. China;
2. Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu 610065, P. R. China;
3. Department of Epidemiology and Health Statistics, West China School of Public Health, Sichuan University, Chengdu 610041, P. R. China;
4. Department of Information Management & Department of Dental Informatics, West China Hospital of Stomatology, Sichuan University, Chengdu 610041, P. R. China;

Corresponding author：

TANG Wei, Email: mydrtw@vip.sina.com; LIU Chang, Email: liu_chang_92@sina.com

Keywords：

Diagnostic test accuracy study; Clinical epidemiology; Artificial intelligence; Machine learning; Deep learning

DOI：

10.7507/1672-2531.202302047

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

With the development of artificial intelligence, machine learning has been widely used in diagnosis of diseases. It is crucial to conduct diagnostic test accuracy studies and evaluate the performance of models reasonably to improve the accuracy of diagnosis. For machine learning-based diagnostic test accuracy studies, this paper introduces the principles of study design in the aspects of target conditions, selection of participants, diagnostic tests, reference standards and ethics.

Citation： ZHANG Yunan, ZHU Tao, ZENG Wei, GUO Jixiang, ZHANG Tao, GOU Pan, TANG Wei, LIU Chang. Machine learning-based diagnostic test accuracy (1): study design. Chinese Journal of Evidence-Based Medicine, 2023, 23(6): 725-730. doi: 10.7507/1672-2531.202302047 Copy

1.	田晨, 杨秋玉, 赖鸿皓, 等. 诊断性试验准确性比较研究. 中国循证医学杂志, 2022, 22(5): 590-594.
2.	Whiting P, Savović J, Higgins JP, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol, 2016, 69: 225-234.
3.	Landini N, Colzani G, Ciet P, et al. Chest radiography findings of COVID-19 pneumonia: a specific pattern for a confident differential diagnosis. Acta Radiol, 2022, 63(12): 1619-1626.
4.	Bi Y, Abrol A, Fu Z, et al. Prediction of gender from longitudinal MRI data via deep learning on adolescent data reveals unique patterns associated with brain structure and change over a two-year period. J Neurosci Methods, 2023, 384: 109744.
5.	Rezapour M, Elmshaeuser SK. Artificial intelligence-based analytics for impacts of COVID-19 and online learning on college students' mental health. PLoS One, 2022, 17(11): e0276767.
6.	Koutsouleris N, Dwyer DB, Degenhardt F, et al. Multimodal machine learning workflows for prediction of psychosis in patients with clinical high-risk syndromes and recent-onset depression. JAMA Psychiatry, 2021, 78(2): 195-209.
7.	Nishiyama M, Ishibashi K, Ariji Y, et al. Performance of deep learning models constructed using panoramic radiographs from two hospitals to diagnose fractures of the mandibular condyle. Dentomaxillofac Radiol, 2021, 50(7): 20200611.
8.	Chang HJ, Lee SJ, Yong TH, et al. Deep learning hybrid method to automatically diagnose periodontal bone loss and stage periodontitis. Sci Rep, 2020, 10(1): 7531.
9.	Hu Z, Bhattacharya S, Butte AJ. Application of machine learning for cytometry data. Front Immunol, 2022, 12: 787574.
10.	Ozer G, Akca A, Yuksel B, et al. Prediction of risk factors for first trimester pregnancy loss in frozen-thawed good-quality embryo transfer cycles using machine learning algorithms. J Assist Reprod Genet, 2023, 40(2): 279-288.
11.	Cruz Rivera S, Liu X, Chan AW, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med, 2020, 26(9): 1351-1363.
12.	Hajian-Tilaki K. Sample size estimation in diagnostic test studies of biomedical informatics. J Biomed Inform, 2014, 48: 193-204.
13.	Yu D, Hu J, Feng Z, et al. Deep learning based diagnosis for cysts and tumors of jaw with massive healthy samples. Sci Rep, 2022, 12(1): 1855.
14.	Hou HF, Liu Y, Zhang X, et al. The value of postoperative HLA-DR expression and high mobility group box 1 level in predictive diagnosis of sepsis in percutaneous nephrolithotomy surgery. Ren Fail, 2022, 44(1): 1338-1344.
15.	Baştanlar Y, Ozuysal M. Introduction to machine learning. Methods Mol Biol, 2014, 1107: 105-128.
16.	杨秋玉, 陆瑶, 田晨, 等. 诊断性试验准确性比较研究: 研究设计. 中国循证医学杂志, 2022, 22(6): 739-744.
17.	Glasziou P, Irwig L, Deeks JJ. When should a new test become the current reference standard. Ann Intern Med, 2008, 149(11): 816-822.
18.	Zhou J, Wang W, Lei B, et al. Automatic detection and classification of focal liver lesions based on deep convolutional neural networks: a preliminary study. Front Oncol, 2021, 10: 581210.
19.	Yang B, Olsen M, Vali Y, et al. Study designs for comparative diagnostic test accuracy: a methodological review and classification scheme. J Clin Epidemiol, 2021, 138: 128-138.
20.	World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA, 2013, 310(20): 2191-2194.
21.	涉及人的生物医学研究伦理审查办法. 中华人民共和国国务院公报, 2017, (27): 44-50.
22.	Mörch CM, Atsu S, Cai W, et al. Artificial intelligence and ethics in dentistry: a scoping review. J Dent Res, 2021, 100(13): 1452-1460.
23.	He JX, Baxter SL, Xu J, et al. The practical implementation of artificial intelligence technologies in medicine. Nat Med, 2019, 25(1): 30-36.
24.	Lacoste A, Luccioni A, Schmidt V, et al. Quantifying the carbon emissions of machine learning. arXiv preprint, 2019.

1. 田晨, 杨秋玉, 赖鸿皓, 等. 诊断性试验准确性比较研究. 中国循证医学杂志, 2022, 22(5): 590-594.
2. Whiting P, Savović J, Higgins JP, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol, 2016, 69: 225-234.
3. Landini N, Colzani G, Ciet P, et al. Chest radiography findings of COVID-19 pneumonia: a specific pattern for a confident differential diagnosis. Acta Radiol, 2022, 63(12): 1619-1626.
4. Bi Y, Abrol A, Fu Z, et al. Prediction of gender from longitudinal MRI data via deep learning on adolescent data reveals unique patterns associated with brain structure and change over a two-year period. J Neurosci Methods, 2023, 384: 109744.
5. Rezapour M, Elmshaeuser SK. Artificial intelligence-based analytics for impacts of COVID-19 and online learning on college students' mental health. PLoS One, 2022, 17(11): e0276767.
6. Koutsouleris N, Dwyer DB, Degenhardt F, et al. Multimodal machine learning workflows for prediction of psychosis in patients with clinical high-risk syndromes and recent-onset depression. JAMA Psychiatry, 2021, 78(2): 195-209.
7. Nishiyama M, Ishibashi K, Ariji Y, et al. Performance of deep learning models constructed using panoramic radiographs from two hospitals to diagnose fractures of the mandibular condyle. Dentomaxillofac Radiol, 2021, 50(7): 20200611.
8. Chang HJ, Lee SJ, Yong TH, et al. Deep learning hybrid method to automatically diagnose periodontal bone loss and stage periodontitis. Sci Rep, 2020, 10(1): 7531.
9. Hu Z, Bhattacharya S, Butte AJ. Application of machine learning for cytometry data. Front Immunol, 2022, 12: 787574.
10. Ozer G, Akca A, Yuksel B, et al. Prediction of risk factors for first trimester pregnancy loss in frozen-thawed good-quality embryo transfer cycles using machine learning algorithms. J Assist Reprod Genet, 2023, 40(2): 279-288.
11. Cruz Rivera S, Liu X, Chan AW, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med, 2020, 26(9): 1351-1363.
12. Hajian-Tilaki K. Sample size estimation in diagnostic test studies of biomedical informatics. J Biomed Inform, 2014, 48: 193-204.
13. Yu D, Hu J, Feng Z, et al. Deep learning based diagnosis for cysts and tumors of jaw with massive healthy samples. Sci Rep, 2022, 12(1): 1855.
14. Hou HF, Liu Y, Zhang X, et al. The value of postoperative HLA-DR expression and high mobility group box 1 level in predictive diagnosis of sepsis in percutaneous nephrolithotomy surgery. Ren Fail, 2022, 44(1): 1338-1344.
15. Baştanlar Y, Ozuysal M. Introduction to machine learning. Methods Mol Biol, 2014, 1107: 105-128.
16. 杨秋玉, 陆瑶, 田晨, 等. 诊断性试验准确性比较研究: 研究设计. 中国循证医学杂志, 2022, 22(6): 739-744.
17. Glasziou P, Irwig L, Deeks JJ. When should a new test become the current reference standard. Ann Intern Med, 2008, 149(11): 816-822.
18. Zhou J, Wang W, Lei B, et al. Automatic detection and classification of focal liver lesions based on deep convolutional neural networks: a preliminary study. Front Oncol, 2021, 10: 581210.
19. Yang B, Olsen M, Vali Y, et al. Study designs for comparative diagnostic test accuracy: a methodological review and classification scheme. J Clin Epidemiol, 2021, 138: 128-138.
20. World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA, 2013, 310(20): 2191-2194.
21. 涉及人的生物医学研究伦理审查办法. 中华人民共和国国务院公报, 2017, (27): 44-50.
22. Mörch CM, Atsu S, Cai W, et al. Artificial intelligence and ethics in dentistry: a scoping review. J Dent Res, 2021, 100(13): 1452-1460.
23. He JX, Baxter SL, Xu J, et al. The practical implementation of artificial intelligence technologies in medicine. Nat Med, 2019, 25(1): 30-36.
24. Lacoste A, Luccioni A, Schmidt V, et al. Quantifying the carbon emissions of machine learning. arXiv preprint, 2019.

Chinese Journal of Evidence-Based Medicine

Machine learning-based diagnostic test accuracy (1): study design

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content