Robustness assessment of pediatrics Meta-analysis using fragility index_Chinese Journal of Evidence-Based Medicine

Authors：

ZHANG Rui ^1,2,3 , XIA Mingxin ^1,2,3 , WU Jingting ^1,2,3 , RUI Xiaoya ^1,2,3 ,  XU Chang ²

1. Department of Maternal, Child and Adolescent Health, School of Public Health, Anhui Medical University, Hefei 230032, P. R. China;
2. Key Laboratory of Population Health Across Life Cycle (Anhui Medical University), Ministry of Education of the People’s Republic of China, Hefei 230032, P. R. China;
3. Anhui Provincial Key Laboratory of Environment and Population Health across the Life Course, Hefei 230032, P. R. China;

Corresponding author：

XU Chang, Email: xuchang2016@runbox.com

Keywords：

Fragility index; Pediatrics; Systematic review; Meta-analysis; Robustness

DOI：

10.7507/1672-2531.202405017

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To evaluate the robustness of pediatrics Clinical evidence-based evidence using fragility index and to explore the factors influencing fragility index. Methods We searched the PubMed, Embase, and Scopus databases to collect relevant literature on systematic reviews and meta-analyses in the field of pediatrics, and calculated the fragility index. The rank sum test was used to compare differences between groups with different outcome types, different levels of statistical significance, and different sample sizes. Spearman correlation analysis was used to explore the association between the fragility index and sample size, as well as the year of publication. Results A total of 152 systematic reviews, including 573 meta-analyses, were included, with a median fragility index of 6 (3, 10). Most meta-analyses chose the risk ratio (RR) as the effect measure (387/573, 67.5%), the Mantel-Haenszel method (412/573, 71.9%) as the synthesis method, and the fixed-effect model (300/573, 57.4%) as the assumed model. The Mann-Whitney test showed no statistically significant difference in the fragility index between meta-analyses with safety outcomes and those with efficacy outcomes (P=0.397), and no statistically significant difference between meta-analyses with significant results and those with non-significant results (P=0.520). The Kruskal-Wallis test found a statistically significant difference in sample size among groups with different fragility indices (P<0.001). Spearman correlation analysis found a positive correlation between the fragility index and sample size (ρ=0.39, P<0.001), but no statistically significant correlation with the year of publication (P=0.235). Conclusion The fragility index of clinical evidence-based evidence published in pediatrics journals is generally low, and the robustness of the results is not high, so it is necessary to be cautious when making evidence-based decisions. Furthermore, the larger the sample size included in the meta-analysis, the higher the fragility index, and incorporating more trials and populations can facilitate the increase in the robustness of the meta-analysis results.

1.	Guyatt GH, Oxman AD, Kunz R, et al. What is "quality of evidence" and why is it important to clinicians. BMJ, 2008, 336(7651): 995-998.
2.	Sutton AJ, Higgins JP. Recent developments in meta-analysis. Stat Med, 2008, 27(5): 625-650.
3.	Averitt AJ, Weng C, Ryan P, et al. Translating evidence into practice: eligibility criteria fail to eliminate clinically significant differences between real-world and study populations. NPJ Digit Med, 2020, 3: 67.
4.	Moyé LA. P-value interpretation and alpha allocation in clinical trials. Ann Epidemiol, 1998, 8(6): 351-357.
5.	Benjamin DJ, Berger JO, Johannesson M, et al. Redefine statistical significance. Nat Hum Behav, 2018, 2(1): 6-10.
6.	Amrhein V, Greenland S, McShane B. Scientists rise up against statistical significance. Nature, 2019, 567(7748): 305-307.
7.	Turner EH, Matthews AM, Linardatos E, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med, 2008, 358(3): 252-260.
8.	Eyding D, Lelgemann M, Grouven U, et al. Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ, 2010, 341: c4737.
9.	Chavalarias D, Wallach JD, Li AH, et al. Evolution of reporting P values in the biomedical literature, 1990-2015. JAMA, 2016, 315(11): 1141-1148.
10.	Walsh M, Srinathan SK, McAuley DF, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index. J Clin Epidemiol, 2014, 67(6): 622-628.
11.	Del Paggio JC, Tannock IF. The fragility of phase 3 trials supporting FDA-approved anticancer medicines: a retrospective analysis. Lancet Oncol, 2019, 20(8): 1065-1069.
12.	Shen Y, Cheng X, Zhang W. The fragility of randomized controlled trials in intracranial hemorrhage. Neurosurg Rev, 2019, 42(1): 9-14.
13.	Javidan A, Benipal H, Vi L, et al. Assessing the robustness of positive vascular surgery randomized controlled trials using their fragility index. J Vasc Surg, 2024, 79(1): 148-158.
14.	Volovici V, Vogels VI, Dammers R, et al. Neurosurgical evidence and randomized trials: the fragility index. World Neurosurg, 2022, 161: 224-229.
15.	Vukadinović D, Abdin A, Emrich I, et al. Efficacy and safety of intravenous iron repletion in patients with heart failure: a systematic review and meta-analysis. Clin Res Cardiol, 2023, 112(7): 954-966.
16.	Berti A, Cornec D, Medina Inojosa JR, et al. Treatments for giant cell arteritis: meta-analysis and assessment of estimates reliability using the fragility index. Semin Arthritis Rheum, 2018, 48(1): 77-82.
17.	Lin L, Xing A, Chu H, et al. Assessing the robustness of results from clinical trials and meta-analyses with the fragility index. Am J Obstet Gynecol, 2023, 228(3): 276-282.
18.	Lin L, Chu H. Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package. PLoS One, 2022, 17(6): e0268754.
19.	Xu C, Furuya-Kanamori L, Zorzela L, et al. A proposed framework to guide evidence synthesis practice for meta-analysis with zero-events studies. J Clin Epidemiol, 2021, 135: 70-78.
20.	Higgins JPT, Thomas J, Chandler J, et al. Cochrane handbook for systematic reviews of interventions. Version 6.2. Chichester (UK): Cochrane, 2021.
21.	Atal I, Porcher R, Boutron I, et al. The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses. J Clin Epidemiol, 2019, 111: 32-40.
22.	DeVito NJ, Goldacre B. Catalogue of bias: publication bias. BMJ Evid Based Med, 2019, 24(2): 53-54.
23.	Lin L. Factors that impact fragility index and their visualizations. J Eval Clin Pract, 2021, 27(2): 356-364.
24.	Desnoyers A, Wilson BE, Nadler MB, et al. Fragility index of trials supporting approval of anti-cancer drugs in common solid tumours. Cancer Treat Rev, 2021, 94: 102167.
25.	Khan MS, Ochani RK, Shaikh A, et al. Fragility index in cardiovascular randomized controlled trials. Circ Cardiovasc Qual Outcomes, 2019, 12(12): e005755.
26.	Maldonado DR, Go CC, Huang BH, et al. The fragility index of hip arthroscopy randomized controlled trials: a systematic survey. Arthroscopy, 2021, 37(6): 1983-1989.

1. Guyatt GH, Oxman AD, Kunz R, et al. What is "quality of evidence" and why is it important to clinicians. BMJ, 2008, 336(7651): 995-998.
2. Sutton AJ, Higgins JP. Recent developments in meta-analysis. Stat Med, 2008, 27(5): 625-650.
3. Averitt AJ, Weng C, Ryan P, et al. Translating evidence into practice: eligibility criteria fail to eliminate clinically significant differences between real-world and study populations. NPJ Digit Med, 2020, 3: 67.
4. Moyé LA. P-value interpretation and alpha allocation in clinical trials. Ann Epidemiol, 1998, 8(6): 351-357.
5. Benjamin DJ, Berger JO, Johannesson M, et al. Redefine statistical significance. Nat Hum Behav, 2018, 2(1): 6-10.
6. Amrhein V, Greenland S, McShane B. Scientists rise up against statistical significance. Nature, 2019, 567(7748): 305-307.
7. Turner EH, Matthews AM, Linardatos E, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med, 2008, 358(3): 252-260.
8. Eyding D, Lelgemann M, Grouven U, et al. Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ, 2010, 341: c4737.
9. Chavalarias D, Wallach JD, Li AH, et al. Evolution of reporting P values in the biomedical literature, 1990-2015. JAMA, 2016, 315(11): 1141-1148.
10. Walsh M, Srinathan SK, McAuley DF, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a fragility index. J Clin Epidemiol, 2014, 67(6): 622-628.
11. Del Paggio JC, Tannock IF. The fragility of phase 3 trials supporting FDA-approved anticancer medicines: a retrospective analysis. Lancet Oncol, 2019, 20(8): 1065-1069.
12. Shen Y, Cheng X, Zhang W. The fragility of randomized controlled trials in intracranial hemorrhage. Neurosurg Rev, 2019, 42(1): 9-14.
13. Javidan A, Benipal H, Vi L, et al. Assessing the robustness of positive vascular surgery randomized controlled trials using their fragility index. J Vasc Surg, 2024, 79(1): 148-158.
14. Volovici V, Vogels VI, Dammers R, et al. Neurosurgical evidence and randomized trials: the fragility index. World Neurosurg, 2022, 161: 224-229.
15. Vukadinović D, Abdin A, Emrich I, et al. Efficacy and safety of intravenous iron repletion in patients with heart failure: a systematic review and meta-analysis. Clin Res Cardiol, 2023, 112(7): 954-966.
16. Berti A, Cornec D, Medina Inojosa JR, et al. Treatments for giant cell arteritis: meta-analysis and assessment of estimates reliability using the fragility index. Semin Arthritis Rheum, 2018, 48(1): 77-82.
17. Lin L, Xing A, Chu H, et al. Assessing the robustness of results from clinical trials and meta-analyses with the fragility index. Am J Obstet Gynecol, 2023, 228(3): 276-282.
18. Lin L, Chu H. Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package. PLoS One, 2022, 17(6): e0268754.
19. Xu C, Furuya-Kanamori L, Zorzela L, et al. A proposed framework to guide evidence synthesis practice for meta-analysis with zero-events studies. J Clin Epidemiol, 2021, 135: 70-78.
20. Higgins JPT, Thomas J, Chandler J, et al. Cochrane handbook for systematic reviews of interventions. Version 6.2. Chichester (UK): Cochrane, 2021.
21. Atal I, Porcher R, Boutron I, et al. The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses. J Clin Epidemiol, 2019, 111: 32-40.
22. DeVito NJ, Goldacre B. Catalogue of bias: publication bias. BMJ Evid Based Med, 2019, 24(2): 53-54.
23. Lin L. Factors that impact fragility index and their visualizations. J Eval Clin Pract, 2021, 27(2): 356-364.
24. Desnoyers A, Wilson BE, Nadler MB, et al. Fragility index of trials supporting approval of anti-cancer drugs in common solid tumours. Cancer Treat Rev, 2021, 94: 102167.
25. Khan MS, Ochani RK, Shaikh A, et al. Fragility index in cardiovascular randomized controlled trials. Circ Cardiovasc Qual Outcomes, 2019, 12(12): e005755.
26. Maldonado DR, Go CC, Huang BH, et al. The fragility index of hip arthroscopy randomized controlled trials: a systematic survey. Arthroscopy, 2021, 37(6): 1983-1989.

Chinese Journal of Evidence-Based Medicine

Latest ArticlesRobustness assessment of pediatrics Meta-analysis using fragility index

Abstract Full text Figures/Tables Video References Cited by

Format

Content