Interpretation of the DECIDE-AI guideline: a reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence_Chinese Journal of Evidence-Based Medicine

Authors：

CHEN Ningsu ^1,2 , ZHAO Kai ² , XUE Xinyu ² , QI Yana ² ,  YU Jiajie ^1,2

1. Department of Clinical Nutrition, West China Hospital of Sichuan University, Chengdu 610041, P. R. China;
2. Research Center of Clinical Epidemiology and Evidence Based Medicine/Chinese Cochrane Centre, West China Hospital of Sichuan University, Chengdu 610041, P. R. China;

Corresponding author：

YU Jiajie, Email: 2003xiong@163.com

Keywords：

Reporting guideline; Artificial intelligence; Decision support systems; Early-stage clinical evaluation; Interpretation

DOI：

10.7507/1672-2531.202401188

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Artificial intelligence has been extensively applied in healthcare services recently, and clinical decision support systems driven by artificial intelligence are one of the applications. Early-stage clinical evaluation of artificial intelligence (AI)-based clinical decision support systems lies between preclinical development (in silico), offline validation, and large-scale trials, but few AI-related clinical studies have addressed human factors evaluations and reported the implementation environment, user characteristics, selection process and algorithm identification of AI systems. In order to bridge the development-to-implementation gap in clinical artificial intelligence and to promote the transparent and standardized reporting of early-stage clinical studies of AI-based decision support systems. A reporting guideline for the developmental and exploratory clinical investigations of decision support systems driven by artificial intelligence (DECIDE-AI) was published in 2022. This paper aimed to interpret the background, development process and key items of the DECIDE-AI guideline and promote its understanding as well as dissemination in China.

Citation： CHEN Ningsu, ZHAO Kai, XUE Xinyu, QI Yana, YU Jiajie. Interpretation of the DECIDE-AI guideline: a reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence. Chinese Journal of Evidence-Based Medicine, 2024, 24(9): 1100-1107. doi: 10.7507/1672-2531.202401188 Copy

1.	Liu X, Faes L, Kale AU, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health, 2019, 1(6): e271-e297.
2.	Vasey B, Ursprung S, Beddoe B, et al. Association of clinician diagnostic performance with machine learning-based decision support systems: a systematic review. JAMA Netw Open, 2021, 4(3): e211276.
3.	Freeman K, Geppert J, Stinton C, et al. Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy. BMJ, 2021, 374: n1872.
4.	Keane PA, Topol EJ. With an eye to AI and autonomous diagnosis. NPJ Digit Med, 2018, 1: 40.
5.	喻佳洁, 陕飞, McCulloch P, 等. 外科创新技术/器械临床研究方法学—IDEAL框架与推荐系列文章之一: IDEAL框架与推荐介绍. 中国胸心血管外科临床杂志, 2021, 28(2): 131-136.
6.	喻佳洁, Hirst A, McCulloch P, 等. 外科创新技术/器械临床研究方法学—IDEAL框架与推荐系列文章之二: IDEAL报告规范解读. 中国胸心血管外科临床杂志, 2021, 28(3): 263-270.
7.	Collins GS, Moons KGM. Reporting of artificial intelligence prediction models. Lancet, 2019, 393(10181): 1577-1579.
8.	Sounderajah V, Ashrafian H, Aggarwal R, et al. Developing specific reporting guidelines for diagnostic accuracy studies assessing AI interventions: the STARD-AI steering group. Nat Med, 2020, 26(6): 807-808.
9.	Liu X, Cruz Rivera S, Moher D, et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Lancet Digit Health, 2020, 2(10): e537-e548.
10.	Cruz RS, Liu X, Chan AW, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med, 2020, 26(9): 1351-1363.
11.	Vasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ, 2022, 377: e070904.
12.	Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med, 2010, 7(2): e1000217.
13.	Dias R, Torkamani A. Artificial intelligence in clinical and genomic diagnostics. Genome Med, 2019, 11(1): 70.
14.	International Organization for Standardization. Information technology - artificial intelligence (AI) - bias in AI systems and AI aided decision making (ISO/IEC TR 24027:2021). 2021.
15.	US Food and Drug Administration (FDA). Clinical decision support software: draft guidance for industry and Food and Drug Administration staff. 2019.
16.	Vasey B, Nagendran M, Mcculloch P. DECIDE-AI 2022. 2022.
17.	Hopper AN, Jamison MH, Lewis WG. Learning curves in surgical practice. Postgrad Med J, 2007, 83(986): 777-779.
18.	Bilbro NA, Hirst A, Paez A, et al. The IDEAL reporting guidelines: a Delphi consensus statement stage specific recommendations for reporting the evaluation of surgical innovation. Ann Surg, 2021, 273(1): 82-85.
19.	Abràmoff MD, Lavin PT, Birch M, et al. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med, 2018, 1: 39.
20.	International Organization for Standardization. Ergonomics of human-system interaction-usability methods supporting human-centred design. ISO/TR 16982, 2002.
21.	International Organization for Standardization. Ergonomics of human-system interaction-part 11: usability: definitions and concepts. ISO 9241-11, 2018.
22.	International Electrotechnical Commission. Medical devices-part 1: application of usability engineering to medical devices. IEC 62366-1, 2015.
23.	International Electrotechnical Commission. Medical devices-part 2: guidance on the application of usability engineering to medical devices. IEC TR 62366-2, 2016.
24.	International Electrotechnical Commission. Medical devices-part 2: guidance on the application of usability engineering to medical devices-amendment 1. IEC 62366-1, 2015.
25.	Wu G, Yang P, Xie Y, et al. Development of a clinical decision support system for severity risk prediction and triage of COVID-19 patients at hospital admission: an international multicentre study. Eur Respir J, 2020, 56(2): 2001104.
26.	Li Z, Zhang X, Ding L, et al. Rationale and design of the GOLDEN BRIDGE II: a cluster-randomised multifaceted intervention trial of an artificial intelligence-based cerebrovascular disease clinical decision support system to improve stroke outcomes and care quality in China. Stroke Vasc Neurol, 2024.
27.	李健斌, 江泽飞. 中国临床肿瘤学会人工智能决策系统的建立与应用. 中华医学杂志, 2020, 100(6): 411-415.
28.	Boel A, Navarro-Compán V, Landewé R, et al. Two different invitation approaches for consecutive rounds of a Delphi survey led to comparable final outcome. J Clin Epidemiol, 2021, 129: 31-39.
29.	Sujan M, Furniss D, Grundy K, et al. Human factors challenges for the safe use of artificial intelligence in patient care. BMJ Health Care Inform, 2019, 26(1): e100081.

1. Liu X, Faes L, Kale AU, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health, 2019, 1(6): e271-e297.
2. Vasey B, Ursprung S, Beddoe B, et al. Association of clinician diagnostic performance with machine learning-based decision support systems: a systematic review. JAMA Netw Open, 2021, 4(3): e211276.
3. Freeman K, Geppert J, Stinton C, et al. Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy. BMJ, 2021, 374: n1872.
4. Keane PA, Topol EJ. With an eye to AI and autonomous diagnosis. NPJ Digit Med, 2018, 1: 40.
5. 喻佳洁, 陕飞, McCulloch P, 等. 外科创新技术/器械临床研究方法学—IDEAL框架与推荐系列文章之一: IDEAL框架与推荐介绍. 中国胸心血管外科临床杂志, 2021, 28(2): 131-136.
6. 喻佳洁, Hirst A, McCulloch P, 等. 外科创新技术/器械临床研究方法学—IDEAL框架与推荐系列文章之二: IDEAL报告规范解读. 中国胸心血管外科临床杂志, 2021, 28(3): 263-270.
7. Collins GS, Moons KGM. Reporting of artificial intelligence prediction models. Lancet, 2019, 393(10181): 1577-1579.
8. Sounderajah V, Ashrafian H, Aggarwal R, et al. Developing specific reporting guidelines for diagnostic accuracy studies assessing AI interventions: the STARD-AI steering group. Nat Med, 2020, 26(6): 807-808.
9. Liu X, Cruz Rivera S, Moher D, et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Lancet Digit Health, 2020, 2(10): e537-e548.
10. Cruz RS, Liu X, Chan AW, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med, 2020, 26(9): 1351-1363.
11. Vasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ, 2022, 377: e070904.
12. Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med, 2010, 7(2): e1000217.
13. Dias R, Torkamani A. Artificial intelligence in clinical and genomic diagnostics. Genome Med, 2019, 11(1): 70.
14. International Organization for Standardization. Information technology - artificial intelligence (AI) - bias in AI systems and AI aided decision making (ISO/IEC TR 24027:2021). 2021.
15. US Food and Drug Administration (FDA). Clinical decision support software: draft guidance for industry and Food and Drug Administration staff. 2019.
16. Vasey B, Nagendran M, Mcculloch P. DECIDE-AI 2022. 2022.
17. Hopper AN, Jamison MH, Lewis WG. Learning curves in surgical practice. Postgrad Med J, 2007, 83(986): 777-779.
18. Bilbro NA, Hirst A, Paez A, et al. The IDEAL reporting guidelines: a Delphi consensus statement stage specific recommendations for reporting the evaluation of surgical innovation. Ann Surg, 2021, 273(1): 82-85.
19. Abràmoff MD, Lavin PT, Birch M, et al. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med, 2018, 1: 39.
20. International Organization for Standardization. Ergonomics of human-system interaction-usability methods supporting human-centred design. ISO/TR 16982, 2002.
21. International Organization for Standardization. Ergonomics of human-system interaction-part 11: usability: definitions and concepts. ISO 9241-11, 2018.
22. International Electrotechnical Commission. Medical devices-part 1: application of usability engineering to medical devices. IEC 62366-1, 2015.
23. International Electrotechnical Commission. Medical devices-part 2: guidance on the application of usability engineering to medical devices. IEC TR 62366-2, 2016.
24. International Electrotechnical Commission. Medical devices-part 2: guidance on the application of usability engineering to medical devices-amendment 1. IEC 62366-1, 2015.
25. Wu G, Yang P, Xie Y, et al. Development of a clinical decision support system for severity risk prediction and triage of COVID-19 patients at hospital admission: an international multicentre study. Eur Respir J, 2020, 56(2): 2001104.
26. Li Z, Zhang X, Ding L, et al. Rationale and design of the GOLDEN BRIDGE II: a cluster-randomised multifaceted intervention trial of an artificial intelligence-based cerebrovascular disease clinical decision support system to improve stroke outcomes and care quality in China. Stroke Vasc Neurol, 2024.
27. 李健斌, 江泽飞. 中国临床肿瘤学会人工智能决策系统的建立与应用. 中华医学杂志, 2020, 100(6): 411-415.
28. Boel A, Navarro-Compán V, Landewé R, et al. Two different invitation approaches for consecutive rounds of a Delphi survey led to comparable final outcome. J Clin Epidemiol, 2021, 129: 31-39.
29. Sujan M, Furniss D, Grundy K, et al. Human factors challenges for the safe use of artificial intelligence in patient care. BMJ Health Care Inform, 2019, 26(1): e100081.

Chinese Journal of Evidence-Based Medicine

Interpretation of the DECIDE-AI guideline: a reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content