Methodological issues in grading the quality of evidence for public health decision-making: a qualitative systematic review_Chinese Journal of Evidence-Based Medicine

Authors：

YANG Chaoqun ^1,2,3 , LIU Wendi ^1,2,3 , WANG Ziyi ^1,2,3 , LIANG Shanshan ^1,2,3 , DENG Xinxin ^1,2,3 , WANG Yongsheng ^1,2,3 , XU Zihan ⁴ ,  LI Xiuxia ^1,2,3 ,  YANG Kehu ^1,2,3

1. Evidence Based Social Science Research Center/Health Technology Assessment Center, School of Public Health, Lanzhou University, Lanzhou 730000, P. R. China;
2. Evidence Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, P. R. China;
3. Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou University, Lanzhou 730000, P. R. China;
4. School of Integrated Traditional Chinese and Western Clinical Medicine, Gansu University of Traditional Chinese Medicine, Lanzhou 730000, P. R. China;

Corresponding author：

LI Xiuxia, Email: lixiuxia@lzu.edu.cn; YANG Kehu, Email: kehuyangebm2006@126.com

Keywords：

Public health decision making; Quality grading of evidence; GRADE; Methodology

DOI：

10.7507/1672-2531.202306114

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To systematically review the research issues related to evidence quality grading methods for public health decision making. Methods PubMed, Web of Science, CNKI, WanFang Data, CBM and VIP databases were electronically searched to collect studies related to the application of evidence quality grading methods for public health decision making from inception to December 2022. The questions were constructed according to the SPIDER model. The quality of the included literature was evaluated by using the CASP checklist, and a three-level interpretation analysis of the questions on the application of quality rating methods for public health decision making was conducted using the thematic synthesis method to establish a pool of question entries. Results A total of 14 papers were included, covering seven countries. GRADE was the commonly used method for grading the quality of evidence. CASP evaluation results showed eight high quality studies, four medium quality studies and two low quality studies. The thematic synthesis method summarized 13 question entries in 7 categories. Conclusion The existing methodology for grading the quality of evidence for public health decision making suffers from the diversity of evidence sources and the underestimation of the level of evidence from complex intervention studies.

Citation： YANG Chaoqun, LIU Wendi, WANG Ziyi, LIANG Shanshan, DENG Xinxin, WANG Yongsheng, XU Zihan, LI Xiuxia, YANG Kehu. Methodological issues in grading the quality of evidence for public health decision-making: a qualitative systematic review. Chinese Journal of Evidence-Based Medicine, 2023, 23(12): 1413-1418. doi: 10.7507/1672-2531.202306114 Copy

1.	张薇, 许吉, 邓宏勇. 国际医学证据分级与推荐体系发展及现状. 中国循证医学杂志, 2019, 19(11): 1373-1378.
2.	陈耀龙, 姚亮, SusanNorris, 等. GRADE在系统评价中应用的必要性及注意事项. 中国循证医学杂志, 2013, 13(12): 1401-1404.
3.	黄桥, 任相颖, 张蓉, 等. GRADE在我国临床实践指南/专家共识中的应用研究. 中国循证医学杂志, 2021, 21(12): 1457-1462.
4.	Irving M, Eramudugolla R, Cherbuin N, et al. A critical review of grading systems: implications for public health policy. Eval Health Prof, 2017, 40(2): 244-262.
5.	Durrheim DN, Reingold A. Modifying the GRADE framework could benefit public health. J Epidemiol Community Health, 2010, 64(5): 387.
6.	Burford BJ, Rehfuess E, Schünemann HJ, et al. Assessing evidence in public health: the added value of GRADE. J Public Health (Oxf), 2012, 34(4): 631-635.
7.	Rehfuess EA, Akl EA. Current experience with applying the GRADE approach to public health interventions: an empirical study. BMC Public Health, 2013, 13: 9.
8.	Movsisyan A, Melendez-Torres GJ, Montgomery P. Users identified challenges in applying GRADE to complex interventions and suggested an extension to GRADE. J Clin Epidemiol, 2016, 70: 191-199.
9.	Casp U. Critical appraisal skills programme (CASP) check lists. 2017.
10.	Fox JR, Dean M, Whittlesea A. The experience of caring for or living with an individual with an eating disorder: a meta-synthesis of qualitative studies. Clin Psychol Psychother, 2017, 24(1): 103-125.
11.	Hilton Boon M, Thomson H, Shaw B, et al. Challenges in applying the GRADE approach in public health guidelines and systematic reviews: a concept article from the GRADE Public Health Group. J Clin Epidemiol, 2021, 135: 42-53.
12.	Akl EA, Kennedy C, Konda K, et al. Using GRADE methodology for the development of public health guidelines for the prevention and treatment of HIV and other STIs among men who have sex with men and transgender people. BMC Public Health, 2012, 12: 386.
13.	Movsisyan A, Melendez-Torres GJ, Montgomery P. Outcomes in systematic reviews of complex interventions never reached "high" GRADE ratings when compared with those of simple interventions. J Clin Epidemiol, 2016, 78: 22-33.
14.	Barbui C, Dua T, van Ommeren M, et al. Challenges in developing evidence-based recommendations using the GRADE approach: the case of mental, neurological, and substance use disorders. PLoS Med, 2010, 7(8): e1000322.
15.	Pottie K, Connor Gorber S, Singh H, et al. Estimating benefits and harms of screening across subgroups: the Canadian Task Force on Preventive Health Care integrates the GRADE approach and overcomes minor challenges. J Clin Epidemiol, 2012, 65(12): 1245-1248.
16.	Forland F, De Carvalho Gomes H, Nokleby H, et al. Applicability of evidence-based practice in public health: risk assessment on Q fever under an ongoing outbreak. Euro Surveill, 2012, 17(3): 20060.
17.	Harder T, Abu Sin M, Bosch-Capblanch X, et al. Towards a framework for evaluating and grading evidence in public health. Health Policy, 2015, 119(6): 732-736.
18.	Hartling L, Fernandes RM, Seida J, et al. From the trenches: a cross-sectional study applying the GRADE tool in systematic reviews of healthcare interventions. PLoS One, 2012, 7(4): e34697.
19.	Tobias DK, Wittenbecher C, Hu FB. Grading nutrition evidence: where to go from here. Am J Clin Nutr, 2021, 113(6): 1385-1387.
20.	张萍, 张崇凡. 随机临床试验报告统一标准声明2022扩展版建议的清单条目翻译和解读. 中国循证儿科杂志, 2023, 18(2): 142-147.
21.	王子君, 姚亮, 刘练, 等. 推荐分级的评估、制订与评价(GRADE)方法学家的培训与认证. 中国循证儿科杂志, 2017, 12(5): 388-391.
22.	Brownson RC, Fielding JE, Maylahn CM. Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health, 2009, 30: 175-201.
23.	Petruzzi L, Ewald B, Covington E, et al. Exploring the efficacy of social work interventions in hospital settings: a scoping review. Soc Work Public Health, 2023, 38(2): 147-160.
24.	Walugembe DR, Sibbald S, Le Ber MJ, et al. Sustainability of public health interventions: where are the gaps. Health Res Policy Syst, 2019, 17(1): 8.
25.	Waters E, Hall BJ, Armstrong R, et al. Essential components of public health evidence reviews: capturing intervention complexity, implementation, economics and equity. J Public Health (Oxf), 2011, 33(3): 462-465.
26.	Murad MH, Almasri J, Alsawas M, et al. Grading the quality of evidence in complex interventions: a guide for evidence-based practitioners. Evid Based Med, 2017, 22(1): 20-22.
27.	Guyatt GH, Oxman AD, Schünemann HJ, et al. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol, 2011, 64(4): 380-382.
28.	Baker A, Potter J, Young K, et al. The applicability of grading systems for guidelines. J Eval Clin Pract, 2011, 17(4): 758-762.
29.	Goldkuhle M, Bender R, Akl EA, et al. GRADE Guidelines: 29. Rating the certainty in time-to-event outcomes-study limitations due to censoring of participants with missing data in intervention studies. J Clin Epidemiol, 2021, 129: 126-137.

1. 张薇, 许吉, 邓宏勇. 国际医学证据分级与推荐体系发展及现状. 中国循证医学杂志, 2019, 19(11): 1373-1378.
2. 陈耀龙, 姚亮, SusanNorris, 等. GRADE在系统评价中应用的必要性及注意事项. 中国循证医学杂志, 2013, 13(12): 1401-1404.
3. 黄桥, 任相颖, 张蓉, 等. GRADE在我国临床实践指南/专家共识中的应用研究. 中国循证医学杂志, 2021, 21(12): 1457-1462.
4. Irving M, Eramudugolla R, Cherbuin N, et al. A critical review of grading systems: implications for public health policy. Eval Health Prof, 2017, 40(2): 244-262.
5. Durrheim DN, Reingold A. Modifying the GRADE framework could benefit public health. J Epidemiol Community Health, 2010, 64(5): 387.
6. Burford BJ, Rehfuess E, Schünemann HJ, et al. Assessing evidence in public health: the added value of GRADE. J Public Health (Oxf), 2012, 34(4): 631-635.
7. Rehfuess EA, Akl EA. Current experience with applying the GRADE approach to public health interventions: an empirical study. BMC Public Health, 2013, 13: 9.
8. Movsisyan A, Melendez-Torres GJ, Montgomery P. Users identified challenges in applying GRADE to complex interventions and suggested an extension to GRADE. J Clin Epidemiol, 2016, 70: 191-199.
9. Casp U. Critical appraisal skills programme (CASP) check lists. 2017.
10. Fox JR, Dean M, Whittlesea A. The experience of caring for or living with an individual with an eating disorder: a meta-synthesis of qualitative studies. Clin Psychol Psychother, 2017, 24(1): 103-125.
11. Hilton Boon M, Thomson H, Shaw B, et al. Challenges in applying the GRADE approach in public health guidelines and systematic reviews: a concept article from the GRADE Public Health Group. J Clin Epidemiol, 2021, 135: 42-53.
12. Akl EA, Kennedy C, Konda K, et al. Using GRADE methodology for the development of public health guidelines for the prevention and treatment of HIV and other STIs among men who have sex with men and transgender people. BMC Public Health, 2012, 12: 386.
13. Movsisyan A, Melendez-Torres GJ, Montgomery P. Outcomes in systematic reviews of complex interventions never reached "high" GRADE ratings when compared with those of simple interventions. J Clin Epidemiol, 2016, 78: 22-33.
14. Barbui C, Dua T, van Ommeren M, et al. Challenges in developing evidence-based recommendations using the GRADE approach: the case of mental, neurological, and substance use disorders. PLoS Med, 2010, 7(8): e1000322.
15. Pottie K, Connor Gorber S, Singh H, et al. Estimating benefits and harms of screening across subgroups: the Canadian Task Force on Preventive Health Care integrates the GRADE approach and overcomes minor challenges. J Clin Epidemiol, 2012, 65(12): 1245-1248.
16. Forland F, De Carvalho Gomes H, Nokleby H, et al. Applicability of evidence-based practice in public health: risk assessment on Q fever under an ongoing outbreak. Euro Surveill, 2012, 17(3): 20060.
17. Harder T, Abu Sin M, Bosch-Capblanch X, et al. Towards a framework for evaluating and grading evidence in public health. Health Policy, 2015, 119(6): 732-736.
18. Hartling L, Fernandes RM, Seida J, et al. From the trenches: a cross-sectional study applying the GRADE tool in systematic reviews of healthcare interventions. PLoS One, 2012, 7(4): e34697.
19. Tobias DK, Wittenbecher C, Hu FB. Grading nutrition evidence: where to go from here. Am J Clin Nutr, 2021, 113(6): 1385-1387.
20. 张萍, 张崇凡. 随机临床试验报告统一标准声明2022扩展版建议的清单条目翻译和解读. 中国循证儿科杂志, 2023, 18(2): 142-147.
21. 王子君, 姚亮, 刘练, 等. 推荐分级的评估、制订与评价(GRADE)方法学家的培训与认证. 中国循证儿科杂志, 2017, 12(5): 388-391.
22. Brownson RC, Fielding JE, Maylahn CM. Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health, 2009, 30: 175-201.
23. Petruzzi L, Ewald B, Covington E, et al. Exploring the efficacy of social work interventions in hospital settings: a scoping review. Soc Work Public Health, 2023, 38(2): 147-160.
24. Walugembe DR, Sibbald S, Le Ber MJ, et al. Sustainability of public health interventions: where are the gaps. Health Res Policy Syst, 2019, 17(1): 8.
25. Waters E, Hall BJ, Armstrong R, et al. Essential components of public health evidence reviews: capturing intervention complexity, implementation, economics and equity. J Public Health (Oxf), 2011, 33(3): 462-465.
26. Murad MH, Almasri J, Alsawas M, et al. Grading the quality of evidence in complex interventions: a guide for evidence-based practitioners. Evid Based Med, 2017, 22(1): 20-22.
27. Guyatt GH, Oxman AD, Schünemann HJ, et al. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol, 2011, 64(4): 380-382.
28. Baker A, Potter J, Young K, et al. The applicability of grading systems for guidelines. J Eval Clin Pract, 2011, 17(4): 758-762.
29. Goldkuhle M, Bender R, Akl EA, et al. GRADE Guidelines: 29. Rating the certainty in time-to-event outcomes-study limitations due to censoring of participants with missing data in intervention studies. J Clin Epidemiol, 2021, 129: 126-137.

Chinese Journal of Evidence-Based Medicine

Methodological issues in grading the quality of evidence for public health decision-making: a qualitative systematic review

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content