Individual treatment effects models based on randomized controlled trials: a systematic review_Chinese Journal of Evidence-Based Medicine

Authors：

YAN Minghai , ZHU Yingxuan , LIN Xiaoying , LI Wei ,  WANG Yang

Chinese Academy of Medical Sciences, Peking Union Medical College, National Center for Cardiovascular Diseases, National Clinical Medical Research Center for Cardiovascular Diseases, FuWai Hospital Medical Research and Statistics Center, Beijing 100037, P. R. China;

Corresponding author：

WANG Yang, Email: wangyang@mrbc-nccd.com

Keywords：

Randomized controlled trial; Prediction model; Individual treatment effect; Systematic review

DOI：

10.7507/1672-2531.202403114

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To review individual treatment effect (ITE) models developed from randomized controlled trials, with the aim of systematically summarizing the current state of model development and assessing the risk of bias. Methods PubMed and Embase databases were searched for studies published between 1990 and 14 June 2024. Data were extracted using the CHARMS inventory, and the PROBAST risk of bias tool was used to assess model quality. Results A total of 11 publications were included, containing 19 ITE models. The ITE modelling methods were regression models with interaction terms (n=8, 42.1%), dual-range models (n=5, 26.3%) and machine learning (n=6, 31.6%). The ITE models had a reporting rate of 78.9%, 73.2% and 10.5% for differentiation, calibration and clinical validity, respectively. Fourteen models were assessed as having a high risk of bias (73.7%), particularly in the area of statistical analysis, due to inappropriate handling of missing data (n=15, 78.9%), inappropriate consideration of model fit issues (n=5, 26.3%), etc. Conclusion Common approaches to ITE model development include constructing interaction terms, dual procedure theory, and machine learning, but suffer from a low number of model developments, more complex modeling methods, and non-standardized reporting. In the future, emphasis should be placed on further exploration of ITE models, promoting diversified modeling methods and standardized reporting to improve the clinical promotion and practical application value of the models.

1.	Kwakman JJM, van Kruijsdijk RCM, Elias SG, et al. Choosing the right strategy based on individualized treatment effect predictions: combination versus sequential chemotherapy in patients with metastatic colorectal cancer. Acta Oncol, 2019, 58(3): 326-333.
2.	Wilson FP, Parikh CR. Translational methods in nephrology: individual treatment effect modeling. J Am Soc Nephrol, 2018, 29(11): 2615-2618.
3.	何文静, 尤东方, 张汝阳, 等. 利用因果森林估计异质性人群下个体的处理效应. 中华流行病学杂志, 2019, 40(6): 707-712.
4.	Blackstone EH. Precision medicine versus evidence-based medicine: individual treatment effect versus average treatment effect. Circulation, 2019, 140(15): 1236-1238.
5.	Mori M, Spertus JA, Krumholz HM. Data-driven individualized surgical decision-making: beyond "better on average" clinical trial results. JAMA Surg, 2022, 157(2): 93-94.
6.	Moons KG, de Groot JA, Bouwmeester W, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med, 2014, 11(10): e1001744.
7.	Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ, 2015, 350: g7594.
8.	Moons KGM, Wolff RF, Riley RD, et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med, 2019, 170(1): W1-W33.
9.	Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ, 2009, 339: b2535.
10.	van Kruijsdijk RC, Visseren FL, Boni L, et al. Pemetrexed plus carboplatin versus pemetrexed in pretreated patients with advanced non-squamous non-small-cell lung cancer: treating the right patients based on individualized treatment effect prediction. Ann Oncol, 2016, 27(7): 1280-1286.
11.	van Bronswijk SC, Lemmens LHJM, Huibers MJH, et al. Selecting the optimal treatment for a depressed individual: clinical judgment or statistical prediction. J Affect Disord, 2021, 279: 149-157.
12.	Furukawa TA, Debray TPA, Akechi T, et al. Can personalized treatment prediction improve the outcomes, compared with the group average approach, in a randomized trial. Developing and validating a multivariable prediction model in a pragmatic megatrial of acute treatment for major depression. J Affect Disord, 2020, 274: 690-697.
13.	Farooq V, van Klaveren D, Steyerberg EW, et al. Anatomical and clinical characteristics to guide decision making between coronary artery bypass surgery and percutaneous coronary intervention for individual patients: development and validation of SYNTAX score II. Lancet, 2013, 381(9867): 639-650.
14.	Nguyen TL, Collins GS, Landais P, et al. Counterfactual clinical prediction models could help to infer individualized treatment effects in randomized controlled trials-an illustration with the International Stroke Trial. J Clin Epidemiol, 2020, 125: 47-56.
15.	Venema E, Mulder MJHL, Roozenbeek B, et al. Selection of patients for intra-arterial treatment for acute ischaemic stroke: development and validation of a clinical decision tool in two randomised trials. BMJ, 2017, 357: j1710.
16.	Kent DM, Selker HP, Ruthazer R, et al. The stroke-thrombolytic predictive instrument: a predictive instrument for intravenous thrombolysis in acute ischemic stroke. Stroke, 2006, 37(12): 2957-2962.
17.	Costa F, van Klaveren D, James S, et al. Derivation and validation of the predicting bleeding complications in patients undergoing stent implantation and subsequent dual antiplatelet therapy (PRECISE-DAPT) score: a pooled analysis of individual-patient datasets from clinical trials. Lancet, 2017, 389(10073): 1025-1034.
18.	Yeh RW, Secemsky EA, Kereiakes DJ, et al. Development and validation of a prediction rule for benefit and harm of dual antiplatelet therapy beyond 1 year after percutaneous coronary intervention. JAMA, 2016, 315(16): 1735-1749.
19.	Rothwell PM, Warlow CP. Prediction of benefit from carotid endarterectomy in individual patients: a risk-modelling study. European Carotid Surgery Trialists' Collaborative Group. Lancet, 1999, 353(9170): 2105-2110.
20.	Kent DM, Steyerberg E, van Klaveren D. Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects. BMJ, 2018, 363: k4245.
21.	van Klaveren D, Varadhan R, Kent DM. The predictive approaches to treatment effect heterogeneity (PATH) statement. Ann Intern Med, 2020, 172(11): 776.
22.	Tabib S, Larocque D. Non-parametric individual treatment effect estimation for survival data with random forests. Bioinformatics, 2020, 36(2): 629-636.
23.	Wager S, Athey S. Estimation and inference of heterogeneous treatment effects using random forests. J Am Stat Assoc, 2018, 113(523): 1228-1242.
24.	Jaskowski M, Jaroszewicz S. Uplift modeling for clinical trial data. Presented at the ICML Workshop on Clinical Data Analysis. 2012.
25.	Wang G, Heagerty PJ, Dahabreh IJ. Using effect scores to characterize heterogeneity of treatment effects. JAMA, 2024, 331(14): 1225-1226.
26.	van Klaveren D, Steyerberg EW, Serruys PW, et al. The proposed 'concordance-statistic for benefit' provided a useful metric when modeling heterogeneous treatment effects. J Clin Epidemiol, 2018, 94: 59-68.
27.	Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ, 2020, 369: m1328.

1. Kwakman JJM, van Kruijsdijk RCM, Elias SG, et al. Choosing the right strategy based on individualized treatment effect predictions: combination versus sequential chemotherapy in patients with metastatic colorectal cancer. Acta Oncol, 2019, 58(3): 326-333.
2. Wilson FP, Parikh CR. Translational methods in nephrology: individual treatment effect modeling. J Am Soc Nephrol, 2018, 29(11): 2615-2618.
3. 何文静, 尤东方, 张汝阳, 等. 利用因果森林估计异质性人群下个体的处理效应. 中华流行病学杂志, 2019, 40(6): 707-712.
4. Blackstone EH. Precision medicine versus evidence-based medicine: individual treatment effect versus average treatment effect. Circulation, 2019, 140(15): 1236-1238.
5. Mori M, Spertus JA, Krumholz HM. Data-driven individualized surgical decision-making: beyond "better on average" clinical trial results. JAMA Surg, 2022, 157(2): 93-94.
6. Moons KG, de Groot JA, Bouwmeester W, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med, 2014, 11(10): e1001744.
7. Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ, 2015, 350: g7594.
8. Moons KGM, Wolff RF, Riley RD, et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med, 2019, 170(1): W1-W33.
9. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ, 2009, 339: b2535.
10. van Kruijsdijk RC, Visseren FL, Boni L, et al. Pemetrexed plus carboplatin versus pemetrexed in pretreated patients with advanced non-squamous non-small-cell lung cancer: treating the right patients based on individualized treatment effect prediction. Ann Oncol, 2016, 27(7): 1280-1286.
11. van Bronswijk SC, Lemmens LHJM, Huibers MJH, et al. Selecting the optimal treatment for a depressed individual: clinical judgment or statistical prediction. J Affect Disord, 2021, 279: 149-157.
12. Furukawa TA, Debray TPA, Akechi T, et al. Can personalized treatment prediction improve the outcomes, compared with the group average approach, in a randomized trial. Developing and validating a multivariable prediction model in a pragmatic megatrial of acute treatment for major depression. J Affect Disord, 2020, 274: 690-697.
13. Farooq V, van Klaveren D, Steyerberg EW, et al. Anatomical and clinical characteristics to guide decision making between coronary artery bypass surgery and percutaneous coronary intervention for individual patients: development and validation of SYNTAX score II. Lancet, 2013, 381(9867): 639-650.
14. Nguyen TL, Collins GS, Landais P, et al. Counterfactual clinical prediction models could help to infer individualized treatment effects in randomized controlled trials-an illustration with the International Stroke Trial. J Clin Epidemiol, 2020, 125: 47-56.
15. Venema E, Mulder MJHL, Roozenbeek B, et al. Selection of patients for intra-arterial treatment for acute ischaemic stroke: development and validation of a clinical decision tool in two randomised trials. BMJ, 2017, 357: j1710.
16. Kent DM, Selker HP, Ruthazer R, et al. The stroke-thrombolytic predictive instrument: a predictive instrument for intravenous thrombolysis in acute ischemic stroke. Stroke, 2006, 37(12): 2957-2962.
17. Costa F, van Klaveren D, James S, et al. Derivation and validation of the predicting bleeding complications in patients undergoing stent implantation and subsequent dual antiplatelet therapy (PRECISE-DAPT) score: a pooled analysis of individual-patient datasets from clinical trials. Lancet, 2017, 389(10073): 1025-1034.
18. Yeh RW, Secemsky EA, Kereiakes DJ, et al. Development and validation of a prediction rule for benefit and harm of dual antiplatelet therapy beyond 1 year after percutaneous coronary intervention. JAMA, 2016, 315(16): 1735-1749.
19. Rothwell PM, Warlow CP. Prediction of benefit from carotid endarterectomy in individual patients: a risk-modelling study. European Carotid Surgery Trialists' Collaborative Group. Lancet, 1999, 353(9170): 2105-2110.
20. Kent DM, Steyerberg E, van Klaveren D. Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects. BMJ, 2018, 363: k4245.
21. van Klaveren D, Varadhan R, Kent DM. The predictive approaches to treatment effect heterogeneity (PATH) statement. Ann Intern Med, 2020, 172(11): 776.
22. Tabib S, Larocque D. Non-parametric individual treatment effect estimation for survival data with random forests. Bioinformatics, 2020, 36(2): 629-636.
23. Wager S, Athey S. Estimation and inference of heterogeneous treatment effects using random forests. J Am Stat Assoc, 2018, 113(523): 1228-1242.
24. Jaskowski M, Jaroszewicz S. Uplift modeling for clinical trial data. Presented at the ICML Workshop on Clinical Data Analysis. 2012.
25. Wang G, Heagerty PJ, Dahabreh IJ. Using effect scores to characterize heterogeneity of treatment effects. JAMA, 2024, 331(14): 1225-1226.
26. van Klaveren D, Steyerberg EW, Serruys PW, et al. The proposed 'concordance-statistic for benefit' provided a useful metric when modeling heterogeneous treatment effects. J Clin Epidemiol, 2018, 94: 59-68.
27. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ, 2020, 369: m1328.

Chinese Journal of Evidence-Based Medicine

Latest ArticlesIndividual treatment effects models based on randomized controlled trials: a systematic review

Abstract Full text Figures/Tables Video References Cited by

Format

Content