Interpretation of standards for conducting and reporting Delphi studies (CREDES) in healthcare research_Chinese Journal of Evidence-Based Medicine

Authors：

JING Chengyang ¹ , LIU Ruixue ² , CHU Hongling ³ ,  SUN Weiwei ² ,  LIAO Xing ¹

1. Center for Evidence Based Chinese Medicine, Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing 100700, P. R. China;
2. Guang'anmen Hospital, China Academy of Chinese Medical Sciences, Beijing 100053, P. R. China;
3. Research Center of Clinical Epidemiology, Peking University Third Hospital, Beijing 100191, P. R. China;

Corresponding author：

SUN Weiwei, Email: sunweiweibj@163.com; LIAO Xing, Email: okfrom2008@hotmail.com

Keywords：

Delphi technique; CREDES standards; Medical research; Consensus; Conducting; Reporting

DOI：

10.7507/1672-2531.202210047

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

As a tool for building consensus among groups, Delphi technique has been widely used in healthcare research which is appropriate for clinical questions where quantitative methods are unlikely to yield results that can be successfully implemented in practice. Researchers in palliative care developed standards for conducting and reporting Delphi studies (CREDES). This paper introduces and interprets the specific content of CREDES standards, with a view to providing reference for the evaluation of Delphi research design quality and report transparency.

Citation： JING Chengyang, LIU Ruixue, CHU Hongling, SUN Weiwei, LIAO Xing. Interpretation of standards for conducting and reporting Delphi studies (CREDES) in healthcare research. Chinese Journal of Evidence-Based Medicine, 2023, 23(2): 233-239. doi: 10.7507/1672-2531.202210047 Copy

1.	Powell C. The Delphi technique: myths and realities. J Adv Nurs, 2003, 41(4): 376-382.
2.	Trevelyan EG, Robinson PN. Delphi methodology in health research: how to do it. Europ J Integ Med, 2015, 7(4): 423-428.
3.	Biondo PD, Nekolaichuk CL, Stiles C, et al. Applying the Delphi process to palliative care tool development: lessons learned. Support Care Cancer, 2008, 16(8): 935-942.
4.	Geist MR. Using the Delphi method to engage stakeholders: a comparison of two studies. Eval Program Plann, 2010, 33(2): 147-154.
5.	McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm, 2016, 38(3): 655-662.
6.	Nair R, Aggarwal R, Khanna D. Methods of formal consensus in classification/diagnostic criteria and guideline development. Semin Arthritis Rheum, 2011, 41(2): 95-105.
7.	Needham DM, Sepulveda KA, Dinglas VD, et al. Core outcome measures for clinical research in acute respiratory failure survivors. An international modified delphi consensus study. Am J Respir Crit Care Med, 2017, 196(9): 1122-1130.
8.	Yoshida S. Approaches, tools and methods used for setting priorities in health research in the 21(st) century. J Glob Health, 2016, 6(1): 010507.
9.	Terwee CB, Prinsen CAC, Chiarotto A, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res, 2018, 27(5): 1159-1170.
10.	管慧, 戴国华, 高武霖, 等. 中医药临床研究中应用德尔菲法的系统评价. 中国循证医学杂志, 2021, 21(2): 186-190.
11.	Diamond IR, Grant RC, Feldman BM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol, 2014, 67(4): 401-409.
12.	Jünger S, Payne SA, Brine J, et al. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliat Med, 2017, 31(8): 684-706.
13.	Knaul FM, Farmer PE, Krakauer EL, et al. Alleviating the access abyss in palliative care and pain relief-an imperative of universal health coverage: the Lancet Commission report. Lancet, 2018, 391(10128): 1391-1454.
14.	Greenhalgh T, Howick J, Maskrey N, et al. Evidence based medicine: a movement in crisis. BMJ, 2014, 348: g3725.
15.	Burns PB, Rohrich RJ, Chung KC. The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg, 2011, 128(1): 305-310.
16.	Aoun SM, Nekolaichuk C. Improving the evidence base in palliative care to inform practice and policy: thinking outside the box. J Pain Symptom Manage, 2014, 48(6): 1222-1235.
17.	Visser C, Hadley G, Wee B. Reality of evidence-based practice in palliative care. Cancer Biol Med, 2015, 12(3): 193-200.
18.	Birko S, Dove ES, Özdemir V. Evaluation of nine consensus indices in Delphi foresight research and their dependency on Delphi survey characteristics: a simulation study and debate on Delphi design and interpretation. PLoS One, 2015, 10(8): e0135162.
19.	Hasson F, Keeney S, Mckenna H. Research guidelines for the Delphi survey technique. J Adv Nurs, 2000, 32(4): 1008-1015.
20.	Veugelers R, Gaakeer MI, Patka P, et al. Improving design choices in Delphi studies in medicine: the case of an exemplary physician multi-round panel study with 100% response. BMC Med Res Methodol, 2020, 20(1): 156.
21.	de Kleijn RJCMF, Schropp L, van Hattum ES, et al. Post-thrombotic syndrome after upper extremity deep vein thrombosis: An international Delphi consensus study. J Thromb Haemost, 2022, 20(8): 1880-1886.
22.	Heuzenroeder L, Ibrahim F, Khadka J, et al. A Delphi study to identify content for a new questionnaire based on the 10 Principles of Dignity in Care. J Clin Nurs, 2022, 31(13-14): 1960-1971.
23.	Nutbeam T, Fenwick R, Smith JE, et al. A Delphi study of rescue and clinical subject matter experts on the extrication of patients following a motor vehicle collision. Scand J Trauma Resusc Emerg Med, 2022, 30(1): 41.
24.	Earle-Payne K, Forsyth P, Johnson CF, et al. The standards of practice for delivery of polypharmacy and chronic disease medication reviews by general practice clinical pharmacists: a consensus study. Int J Clin Pharm, 2022, 44(3): 663-672.
25.	Schirghuber J, Köck-Hódi S, Schrems B. "If you're no longer able to get out and about …" Validity and differentiation of the concepts of being homebound, wheelchair-bound and bedridden: A Delphi study. Z Evid Fortbild Qual Gesundhwes, 2022, 173: 1-16.
26.	Fakha A, de Boer B, van Achterberg T, et al. Fostering the implementation of transitional care innovations for older persons: prioritizing the influencing key factors using a modified Delphi technique. BMC Geriatr, 2022, 22(1): 131.
27.	Krause R, Barnard A, Burger H, et al. A Delphi study to guide the development of a clinical indicator tool for palliative care in South Africa. Afr J Prim Health Care Fam Med, 2022, 14(1): e1-e7.
28.	Karlsson P, Griffiths T, Clarke MT, et al. Stakeholder consensus for decision making in eye-gaze control technology for children, adolescents and adults with cerebral palsy service provision: findings from a Delphi study. BMC Neurol, 2021, 21(1): 63.
29.	Graham-Clarke E, Rushton A, Marriott J. A Delphi study to explore and gain consensus regarding the most important barriers and facilitators affecting physiotherapist and pharmacist non-medical prescribing. PLoS One, 2021, 16(2): e0246273.
30.	Wu C, Wu P, Li P, et al. Construction of an index system of core competence assessment for infectious disease specialist nurse in China: a Delphi study. BMC Infect Dis, 2021, 21(1): 791.
31.	陈丽玲, 周春兰, 吴艳妮, 等. 临床护士循证实践培训体系的构建. 护理学报, 2021, 28(11): 7-11.
32.	Wagner A, Zúñiga F, Rüesch P, et al. Selecting home care quality indicators based on the Resident Assessment Instrument-Home Care (RAI-HC) for Switzerland: A public health and healthcare providers' perspective. PLoS One, 2020, 15(12): e0244577.
33.	Kallio H, Pietilä AM, Kangasniemi M. Environmental responsibility in nursing in hospitals: A modified Delphi study of nurses' views. J Clin Nurs, 2020, 29(21-22): 4045-4056.
34.	Shawahna R. Merits, features, and desiderata to be considered when developing electronic health records with embedded clinical decision support systems in Palestinian hospitals: a consensus study. BMC Med Inform Decis Mak, 2019, 19(1): 216.
35.	Khan Y, Brown AD, Gagliardi AR, et al. Are we prepared. The development of performance indicators for public health emergency preparedness using a modified Delphi approach. PLoS One, 2019, 14(12): e0226489.
36.	Dai F, Wei K, Chen Y, et al. Construction of an index system for qualitative evaluation of undergraduate nursing students innovative ability: A Delphi study. J Clin Nurs, 2019, 28(23-24): 4379-4388.
37.	Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med, 2010, 7(2): e1000217.
38.	Benzon HT, Joshi GP, Gan TJ, et al. Development, reporting, and evaluation of clinical practice guidelines. Anesth Analg, 2019, 129(6): 1771-1777.
39.	Banno M, Tsujimoto Y, Kataoka Y. The majority of reporting guidelines are not developed with the Delphi method: a systematic review of reporting guidelines. J Clin Epidemiol, 2020, 124: 50-57.
40.	Langereis EJ, Borgo A, Crushell E, et al. Treatment of hip dysplasia in patients with mucopolysaccharidosis type I after hematopoietic stem cell transplantation: results of an international consensus procedure. Orphanet J Rare Dis, 2013, 8: 155.
41.	Kojima M, Shimazaki H, Iwaya K, et al. Pathological diagnostic criterion of blood and lymphatic vessel invasion in colorectal cancer: a framework for developing an objective pathological diagnostic system using the Delphi method, from the Pathology Working Group of the Japanese Society for Cancer of the Colon and Rectum. J Clin Pathol, 2013, 66(7): 551-558.
42.	Fernandes L, Hagen KB, Bijlsma JW, et al. EULAR recommendations for the non-pharmacological core management of hip and knee osteoarthritis. Ann Rheum Dis, 2013, 72(7): 1125-1135.
43.	Thompson B, MacAuley D, McNally O, et al. Defining the sports medicine specialist in the United Kingdom: a Delphi study. Br J Sports Med, 2004, 38(2): 214-217.

1. Powell C. The Delphi technique: myths and realities. J Adv Nurs, 2003, 41(4): 376-382.
2. Trevelyan EG, Robinson PN. Delphi methodology in health research: how to do it. Europ J Integ Med, 2015, 7(4): 423-428.
3. Biondo PD, Nekolaichuk CL, Stiles C, et al. Applying the Delphi process to palliative care tool development: lessons learned. Support Care Cancer, 2008, 16(8): 935-942.
4. Geist MR. Using the Delphi method to engage stakeholders: a comparison of two studies. Eval Program Plann, 2010, 33(2): 147-154.
5. McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm, 2016, 38(3): 655-662.
6. Nair R, Aggarwal R, Khanna D. Methods of formal consensus in classification/diagnostic criteria and guideline development. Semin Arthritis Rheum, 2011, 41(2): 95-105.
7. Needham DM, Sepulveda KA, Dinglas VD, et al. Core outcome measures for clinical research in acute respiratory failure survivors. An international modified delphi consensus study. Am J Respir Crit Care Med, 2017, 196(9): 1122-1130.
8. Yoshida S. Approaches, tools and methods used for setting priorities in health research in the 21(st) century. J Glob Health, 2016, 6(1): 010507.
9. Terwee CB, Prinsen CAC, Chiarotto A, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res, 2018, 27(5): 1159-1170.
10. 管慧, 戴国华, 高武霖, 等. 中医药临床研究中应用德尔菲法的系统评价. 中国循证医学杂志, 2021, 21(2): 186-190.
11. Diamond IR, Grant RC, Feldman BM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol, 2014, 67(4): 401-409.
12. Jünger S, Payne SA, Brine J, et al. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliat Med, 2017, 31(8): 684-706.
13. Knaul FM, Farmer PE, Krakauer EL, et al. Alleviating the access abyss in palliative care and pain relief-an imperative of universal health coverage: the Lancet Commission report. Lancet, 2018, 391(10128): 1391-1454.
14. Greenhalgh T, Howick J, Maskrey N, et al. Evidence based medicine: a movement in crisis. BMJ, 2014, 348: g3725.
15. Burns PB, Rohrich RJ, Chung KC. The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg, 2011, 128(1): 305-310.
16. Aoun SM, Nekolaichuk C. Improving the evidence base in palliative care to inform practice and policy: thinking outside the box. J Pain Symptom Manage, 2014, 48(6): 1222-1235.
17. Visser C, Hadley G, Wee B. Reality of evidence-based practice in palliative care. Cancer Biol Med, 2015, 12(3): 193-200.
18. Birko S, Dove ES, Özdemir V. Evaluation of nine consensus indices in Delphi foresight research and their dependency on Delphi survey characteristics: a simulation study and debate on Delphi design and interpretation. PLoS One, 2015, 10(8): e0135162.
19. Hasson F, Keeney S, Mckenna H. Research guidelines for the Delphi survey technique. J Adv Nurs, 2000, 32(4): 1008-1015.
20. Veugelers R, Gaakeer MI, Patka P, et al. Improving design choices in Delphi studies in medicine: the case of an exemplary physician multi-round panel study with 100% response. BMC Med Res Methodol, 2020, 20(1): 156.
21. de Kleijn RJCMF, Schropp L, van Hattum ES, et al. Post-thrombotic syndrome after upper extremity deep vein thrombosis: An international Delphi consensus study. J Thromb Haemost, 2022, 20(8): 1880-1886.
22. Heuzenroeder L, Ibrahim F, Khadka J, et al. A Delphi study to identify content for a new questionnaire based on the 10 Principles of Dignity in Care. J Clin Nurs, 2022, 31(13-14): 1960-1971.
23. Nutbeam T, Fenwick R, Smith JE, et al. A Delphi study of rescue and clinical subject matter experts on the extrication of patients following a motor vehicle collision. Scand J Trauma Resusc Emerg Med, 2022, 30(1): 41.
24. Earle-Payne K, Forsyth P, Johnson CF, et al. The standards of practice for delivery of polypharmacy and chronic disease medication reviews by general practice clinical pharmacists: a consensus study. Int J Clin Pharm, 2022, 44(3): 663-672.
25. Schirghuber J, Köck-Hódi S, Schrems B. "If you're no longer able to get out and about …" Validity and differentiation of the concepts of being homebound, wheelchair-bound and bedridden: A Delphi study. Z Evid Fortbild Qual Gesundhwes, 2022, 173: 1-16.
26. Fakha A, de Boer B, van Achterberg T, et al. Fostering the implementation of transitional care innovations for older persons: prioritizing the influencing key factors using a modified Delphi technique. BMC Geriatr, 2022, 22(1): 131.
27. Krause R, Barnard A, Burger H, et al. A Delphi study to guide the development of a clinical indicator tool for palliative care in South Africa. Afr J Prim Health Care Fam Med, 2022, 14(1): e1-e7.
28. Karlsson P, Griffiths T, Clarke MT, et al. Stakeholder consensus for decision making in eye-gaze control technology for children, adolescents and adults with cerebral palsy service provision: findings from a Delphi study. BMC Neurol, 2021, 21(1): 63.
29. Graham-Clarke E, Rushton A, Marriott J. A Delphi study to explore and gain consensus regarding the most important barriers and facilitators affecting physiotherapist and pharmacist non-medical prescribing. PLoS One, 2021, 16(2): e0246273.
30. Wu C, Wu P, Li P, et al. Construction of an index system of core competence assessment for infectious disease specialist nurse in China: a Delphi study. BMC Infect Dis, 2021, 21(1): 791.
31. 陈丽玲, 周春兰, 吴艳妮, 等. 临床护士循证实践培训体系的构建. 护理学报, 2021, 28(11): 7-11.
32. Wagner A, Zúñiga F, Rüesch P, et al. Selecting home care quality indicators based on the Resident Assessment Instrument-Home Care (RAI-HC) for Switzerland: A public health and healthcare providers' perspective. PLoS One, 2020, 15(12): e0244577.
33. Kallio H, Pietilä AM, Kangasniemi M. Environmental responsibility in nursing in hospitals: A modified Delphi study of nurses' views. J Clin Nurs, 2020, 29(21-22): 4045-4056.
34. Shawahna R. Merits, features, and desiderata to be considered when developing electronic health records with embedded clinical decision support systems in Palestinian hospitals: a consensus study. BMC Med Inform Decis Mak, 2019, 19(1): 216.
35. Khan Y, Brown AD, Gagliardi AR, et al. Are we prepared. The development of performance indicators for public health emergency preparedness using a modified Delphi approach. PLoS One, 2019, 14(12): e0226489.
36. Dai F, Wei K, Chen Y, et al. Construction of an index system for qualitative evaluation of undergraduate nursing students innovative ability: A Delphi study. J Clin Nurs, 2019, 28(23-24): 4379-4388.
37. Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med, 2010, 7(2): e1000217.
38. Benzon HT, Joshi GP, Gan TJ, et al. Development, reporting, and evaluation of clinical practice guidelines. Anesth Analg, 2019, 129(6): 1771-1777.
39. Banno M, Tsujimoto Y, Kataoka Y. The majority of reporting guidelines are not developed with the Delphi method: a systematic review of reporting guidelines. J Clin Epidemiol, 2020, 124: 50-57.
40. Langereis EJ, Borgo A, Crushell E, et al. Treatment of hip dysplasia in patients with mucopolysaccharidosis type I after hematopoietic stem cell transplantation: results of an international consensus procedure. Orphanet J Rare Dis, 2013, 8: 155.
41. Kojima M, Shimazaki H, Iwaya K, et al. Pathological diagnostic criterion of blood and lymphatic vessel invasion in colorectal cancer: a framework for developing an objective pathological diagnostic system using the Delphi method, from the Pathology Working Group of the Japanese Society for Cancer of the Colon and Rectum. J Clin Pathol, 2013, 66(7): 551-558.
42. Fernandes L, Hagen KB, Bijlsma JW, et al. EULAR recommendations for the non-pharmacological core management of hip and knee osteoarthritis. Ann Rheum Dis, 2013, 72(7): 1125-1135.
43. Thompson B, MacAuley D, McNally O, et al. Defining the sports medicine specialist in the United Kingdom: a Delphi study. Br J Sports Med, 2004, 38(2): 214-217.

Previous Article
Assessment tool ArRoWS for the quality of real world observational studies: an interpretation
Next Article
Definition, characteristics and methodological interpretation of Cochrane rapid reviews

Chinese Journal of Evidence-Based Medicine

Interpretation of standards for conducting and reporting Delphi studies (CREDES) in healthcare research

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content