Application of covariate-adjusted ROC curves to evaluate the prediction and classification accuracy of markers_Chinese Journal of Evidence-Based Medicine

Authors：

WU Jiawei ¹ , LIU Jiahao ² , CHEN Qi ³ ,  QIN Yuchen ³

1. Basic Medical College, Naval Medical University, Shanghai 200433, P. R. China;
2. Naval Healthcare Information Center, Naval Medical University, Shanghai 200433, P. R. China;
3. Department of Health Statistics, Naval Medical University, Shanghai 200433, P. R. China;

Corresponding author：

QIN Yuchen, Email: qinyc10@163.com

Keywords：

Receiver operating characteristic curve; Covariate-adjusted ROC curve; Marker; Prediction; Area under curve

DOI：

10.7507/1672-2531.202202023

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

This paper introduced the fundamental theory, method advantages, application scenario and R software implementation method of the covariate-adjusted receiver operating characteristic (ROC) curve. Compared with the traditional univariate ROC curve, the covariate-adjusted ROC curve has distinct methodological advantages and wider application scenarios, which can help to evaluate the ability of markers to predict the targeted outcome more scientifically. It merits more widespread and prior adoption in practical research.

Citation： WU Jiawei, LIU Jiahao, CHEN Qi, QIN Yuchen. Application of covariate-adjusted ROC curves to evaluate the prediction and classification accuracy of markers. Chinese Journal of Evidence-Based Medicine, 2022, 22(9): 1085-1089. doi: 10.7507/1672-2531.202202023 Copy

1.	刘利, 田攀文. 生物标志物在肺癌筛查中的研究进展. 华西医学, 2020, 35(1): 78-83.
2.	Hu X, Li C, Chen J, et al. Confidence intervals for the Youden index and its optimal cut-off point in the presence of covariates. J Biopharm Stat, 2021, 31(3): 251-272.
3.	李太顺, 刘沛. ROC曲线绘制和曲线下面积比较的SAS宏包. 中国卫生统计, 2018, 35(2): 302-304,309.
4.	Muschelli J. ROC and AUC with a binary predictor: a potentially misleading metric. J Classif, 2020, 37(3): 696-708.
5.	Wang Z, Zhou X. Biomarker assessment and combination with differential covariate effects and an unknown gold standard, with an application to Alzheimer’s disease. Ann Appl Stat, 2018, 12(2): 1204-1227.
6.	Janes H, Pepe MS. Adjusting for covariates in studies of diagnostic, screening, or prognostic markers: an old concept in a new setting. Am J Epidemiol, 2008, 168(1): 89-97.
7.	Le Borgne F, Combescure C, Gillaizeau F, et al. Standardized and weighted time-dependent receiver operating characteristic curves to evaluate the intrinsic prognostic capacities of a marker by taking into account confounding factors. Stat Methods Med Res, 2018, 27(11): 3397-3410.
8.	王茹月, 叶雨. PSA衍生指标诊断前列腺癌的研究进展. 中华男科学杂志, 2019, 25(7): 655-659.
9.	Liu Y, Xiao G, Zhou JW, et al. Optimal starting age and baseline level for repeat tests: economic concerns of psa screening for chinese men - 10-year experience of a single center. Urol Int, 2020, 104(3-4): 230-238.
10.	Janes H, Pepe MS. Adjusting for covariate effects on classification accuracy using the covariate-adjusted receiver operating characteristic curve. Biometrika, 2009, 96(2): 371-382.
11.	Janes H, Longton G, Pepe M. Accommodating covariates in ROC analysis. Stat J, 2009, 9(1): 17-39.
12.	Inácio V, M Lourenço V, de Carvalho M, et al. Robust and flexible inference for the covariate-specific receiver operating characteristic curve. Stat Med, 2021, 40(26): 5779-5795.
13.	Pepe M, Longton G, Janes H. Estimation and comparison of receiver operating characteristic curves. Stata J, 2009, 9(1): 1.
14.	Anoke SC, Normand SL, Zigler CM. Approaches to treatment effect heterogeneity in the presence of confounding. Stat Med, 2019, 38(15): 2797-2815.
15.	Wu Z, Chen Q, Djaladat H, et al. A preoperative nomogram to predict renal function insufficiency for cisplatin-based adjuvant chemotherapy following minimally invasive radical nephroureterectomy (ROBUUST collaborative group). Eur Urol Focus, 2022, 8(1): 173-181.
16.	Momota M, Hatakeyama S, Tokui N, et al. The impact of preoperative severe renal insufficiency on poor postsurgical oncological prognosis in patients with urothelial carcinoma. Eur Urol Focus, 2019, 5(6): 1066-1073.
17.	Rouer M, Monnot A, Bubenheim M, et al. Early postoperative renal dysfunction predicts long-term renal function degradation after type Ⅳ thoracoabdominal aortic aneurysm surgical repair. Ann Vasc Surg, 2020, 68: 316-325.
18.	Cohen JS, Gu A, Wei C, et al. Preoperative estimated glomerular filtration rate is a marker for postoperative complications following revision total knee arthroplasty. J Arthroplasty, 2019, 34(4): 750-754.
19.	Onohara T, Takagi T, Yoshida K, et al. Assessment of postoperative renal function after adrenalectomy in patients with primary aldosteronism. Int J Urol, 2019, 26(2): 229-233.
20.	Oto J, Fernández-Pardo Á, Royo M, et al. A predictive model for prostate cancer incorporating PSA molecular forms and age. Sci Rep, 2020, 10(1): 2463.
21.	Tao T, Shen D, Yuan L, et al. Establishing a novel prediction model for improving the positive rate of prostate biopsy. Transl Androl Urol, 2020, 9(2): 574-582.

1. 刘利, 田攀文. 生物标志物在肺癌筛查中的研究进展. 华西医学, 2020, 35(1): 78-83.
2. Hu X, Li C, Chen J, et al. Confidence intervals for the Youden index and its optimal cut-off point in the presence of covariates. J Biopharm Stat, 2021, 31(3): 251-272.
3. 李太顺, 刘沛. ROC曲线绘制和曲线下面积比较的SAS宏包. 中国卫生统计, 2018, 35(2): 302-304,309.
4. Muschelli J. ROC and AUC with a binary predictor: a potentially misleading metric. J Classif, 2020, 37(3): 696-708.
5. Wang Z, Zhou X. Biomarker assessment and combination with differential covariate effects and an unknown gold standard, with an application to Alzheimer’s disease. Ann Appl Stat, 2018, 12(2): 1204-1227.
6. Janes H, Pepe MS. Adjusting for covariates in studies of diagnostic, screening, or prognostic markers: an old concept in a new setting. Am J Epidemiol, 2008, 168(1): 89-97.
7. Le Borgne F, Combescure C, Gillaizeau F, et al. Standardized and weighted time-dependent receiver operating characteristic curves to evaluate the intrinsic prognostic capacities of a marker by taking into account confounding factors. Stat Methods Med Res, 2018, 27(11): 3397-3410.
8. 王茹月, 叶雨. PSA衍生指标诊断前列腺癌的研究进展. 中华男科学杂志, 2019, 25(7): 655-659.
9. Liu Y, Xiao G, Zhou JW, et al. Optimal starting age and baseline level for repeat tests: economic concerns of psa screening for chinese men - 10-year experience of a single center. Urol Int, 2020, 104(3-4): 230-238.
10. Janes H, Pepe MS. Adjusting for covariate effects on classification accuracy using the covariate-adjusted receiver operating characteristic curve. Biometrika, 2009, 96(2): 371-382.
11. Janes H, Longton G, Pepe M. Accommodating covariates in ROC analysis. Stat J, 2009, 9(1): 17-39.
12. Inácio V, M Lourenço V, de Carvalho M, et al. Robust and flexible inference for the covariate-specific receiver operating characteristic curve. Stat Med, 2021, 40(26): 5779-5795.
13. Pepe M, Longton G, Janes H. Estimation and comparison of receiver operating characteristic curves. Stata J, 2009, 9(1): 1.
14. Anoke SC, Normand SL, Zigler CM. Approaches to treatment effect heterogeneity in the presence of confounding. Stat Med, 2019, 38(15): 2797-2815.
15. Wu Z, Chen Q, Djaladat H, et al. A preoperative nomogram to predict renal function insufficiency for cisplatin-based adjuvant chemotherapy following minimally invasive radical nephroureterectomy (ROBUUST collaborative group). Eur Urol Focus, 2022, 8(1): 173-181.
16. Momota M, Hatakeyama S, Tokui N, et al. The impact of preoperative severe renal insufficiency on poor postsurgical oncological prognosis in patients with urothelial carcinoma. Eur Urol Focus, 2019, 5(6): 1066-1073.
17. Rouer M, Monnot A, Bubenheim M, et al. Early postoperative renal dysfunction predicts long-term renal function degradation after type Ⅳ thoracoabdominal aortic aneurysm surgical repair. Ann Vasc Surg, 2020, 68: 316-325.
18. Cohen JS, Gu A, Wei C, et al. Preoperative estimated glomerular filtration rate is a marker for postoperative complications following revision total knee arthroplasty. J Arthroplasty, 2019, 34(4): 750-754.
19. Onohara T, Takagi T, Yoshida K, et al. Assessment of postoperative renal function after adrenalectomy in patients with primary aldosteronism. Int J Urol, 2019, 26(2): 229-233.
20. Oto J, Fernández-Pardo Á, Royo M, et al. A predictive model for prostate cancer incorporating PSA molecular forms and age. Sci Rep, 2020, 10(1): 2463.
21. Tao T, Shen D, Yuan L, et al. Establishing a novel prediction model for improving the positive rate of prostate biopsy. Transl Androl Urol, 2020, 9(2): 574-582.

Chinese Journal of Evidence-Based Medicine

Application of covariate-adjusted ROC curves to evaluate the prediction and classification accuracy of markers

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content