Reliability and validity analysis of guideline clinical applicability evaluation tools_Chinese Journal of Evidence-Based Medicine

Authors：

LI Hailong ^1,2 ^# , ZHANG Chuan ^1,2 ^# , HUANG Chao ³ , ZENG Linan ^1,2 , YANG Chunsong ^1,2 , HUANG Zongyao ^1,2,4 , YI Qiusha ^1,2,5 , XU Peipei ^1,2,5 , LIU Guanjian ⁶ , KANG Deying ⁶ , LIU Qiaolan ⁷ , ZHAO Rongsheng ^8,9 , LIU Hanmin ^2,10 , ZHANG Junhua ¹¹ , LI Youping ⁶ ,  WANG Qiang ³ ,  ZHANG Lingli ^1,2,9

1. Department of Pharmacy Evidence-Based Pharmacy Center, West China Second University Hospital, Sichuan University, Chengdu 610041, P.R.China;
2. Key Laboratory of Birth Defects and Related Diseases of Women and Children (Sichuan University), Ministry of Education, Chengdu 610041, P.R.China;
3. Medical Management Service Guidance Center, National Health Commission of the People’s Republic of China, Beijing 100044, P.R.China;
4. West China School of Pharmacy, Sichuan University, Chengdu 610041, P.R.China;
5. West China School of Medicine, Sichuan University, Chengdu 610041 P.R.China;
6. Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu 610041, P.R.China;
7. West China School of Public Health, Sichuan University, Chengdu 610041, P.R.China;
8. Department of Pharmacy, The Third Hospital of Peking University, Beijing 100191, P.R.China;
9. Evidence-Based Pharmacy Committee, Chinese Pharmaceutical Association, Beijing 100044, P.R.China;
10. Department of Pediatrics, West China Second University Hospital, Sichuan University, Chengdu 610041, P.R.China;
11. Centre for Evidence-Based Medicine, Tianjin University of TCM, Tianjin 301617, P.R.China;

Corresponding author：

WANG Qiang, Email: jason2019@sina.cn; ZHANG Lingli, Email: zhanglingli@scu.edu.cn

Keywords：

Guideline applicability tool; Reliability; Validity; Evaluation

DOI：

10.7507/1672-2531.201909124

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objectives This study aimed to evaluate the validity and reliability of the clinical practice guidelines (CPGs) applicability evaluation tool, a preliminary revised tool, by using it to appraise specific clinical guidelines.Methods Medical staffs were sampled from relevant departments in domestic medical institutions to use tool to evaluate the two guidelines. Spearman-Brown coefficient of odd-even split-half method and Cronbach's alpha coefficient were used to evaluate the split-half reliability and internal consistency reliability. The convergent and discriminant validity were evaluated by correlation analysis and correlation coefficient comparison hypothesis test, and the structural validity was investigated by confirmatory factor analysis based on structural equation.Results The split-half reliability of the evaluation tool was 0.86, and the Cronbach's coefficient of the whole tool and each dimension were greater than 0.7 for two guidelines. The success rates of tool convergent and discriminant validity calibration were 100%. In the second-order confirmatory factor analysis model, the χ²anddf were 3.38 and 2.46, the comparative fit index (CFI) were 0.872 and 0.974, the goodness of fit index (GFI) were 0.954 and 0.983, and the adjusted goodness of fit index (AGFI) were 0.846 and 0.959 for two guidelines respectively. Both standard root mean square residual (SRMR) and root mean square error of approximation (RMSEA) were less than 0.09. Both P values of RMSEA hypothesis test were greater than 0.05.Conclusions The evaluation scale is a valid and reliable instrument for assessing the applicability of CPGs, which should be further evaluated in practical applications in the future.

Citation： LI Hailong, ZHANG Chuan, HUANG Chao, ZENG Linan, YANG Chunsong, HUANG Zongyao, YI Qiusha, XU Peipei, LIU Guanjian, KANG Deying, LIU Qiaolan, ZHAO Rongsheng, LIU Hanmin, ZHANG Junhua, LI Youping, WANG Qiang, ZHANG Lingli. Reliability and validity analysis of guideline clinical applicability evaluation tools. Chinese Journal of Evidence-Based Medicine, 2020, 20(4): 378-382. doi: 10.7507/1672-2531.201909124 Copy

1.	史静琤, 莫显昆, 孙振球. 量表编制中内容效度指数的应用. 中南大学学报 (医学版), 2012, 37(2): 152-155.
2.	孙振球, 徐勇勇. 医学统计学. 北京: 人民卫生出版社, 2014.
3.	石炳毅, 陈莉萍. 中国肾移植排斥反应临床诊疗指南 (2016 版). 器官移植, 2016, 5(5): 332-338.
4.	中华医学会糖尿病学分会. 中国 2 型糖尿病防治指南 (2017 年版). 中国实用内科杂志, 2018, 38(4): 34-86.
5.	Tabachnick BG, Fidell LS. Using multivariate statistics (5th edtion). New York: Allyn and Bacon, 2007.
6.	邓晨卉, 刘洋, 张宇, 等. SF-12 量表在西部农村青少年中信度及效度评价. 中国公共卫生, 2016, 32(11): 1513-1516.
7.	王山, 樊文洁, 俞婉琦, 等. SF-36 量表应用于城市化居民生命质量评价的信度和效度分析. 中华流行病学杂志, 2016, 37(3): 344-347.
8.	张莎, 田晋, 刘巧兰, 等. 流动人口 SF-12 生命质量量表信度、效度评价. 中国公共卫生, 2011, 27(2): 226-227.
9.	江梅. 总体相关系数比较的假设检验. 中国卫生统计, 2010, 27(1): 83-84.
10.	Li H, Xie R, Wang Y, et al. A new scale for the evaluation of clinical practice guidelines applicability: development and appraisal. Implement Sci, 2018, 13(1): 61.
11.	Collaboration A. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care, 2003, 12(1): 18-23.
12.	Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ, 2010, 182(10): 1045-1052.
13.	Brouwers MC, Kho ME, Browman GP, Burgers JS, et al. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ, 2010, 182(10): E472-478.
14.	Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ, 2010, 182(18): E839-842.

1. 史静琤, 莫显昆, 孙振球. 量表编制中内容效度指数的应用. 中南大学学报 (医学版), 2012, 37(2): 152-155.
2. 孙振球, 徐勇勇. 医学统计学. 北京: 人民卫生出版社, 2014.
3. 石炳毅, 陈莉萍. 中国肾移植排斥反应临床诊疗指南 (2016 版). 器官移植, 2016, 5(5): 332-338.
4. 中华医学会糖尿病学分会. 中国 2 型糖尿病防治指南 (2017 年版). 中国实用内科杂志, 2018, 38(4): 34-86.
5. Tabachnick BG, Fidell LS. Using multivariate statistics (5th edtion). New York: Allyn and Bacon, 2007.
6. 邓晨卉, 刘洋, 张宇, 等. SF-12 量表在西部农村青少年中信度及效度评价. 中国公共卫生, 2016, 32(11): 1513-1516.
7. 王山, 樊文洁, 俞婉琦, 等. SF-36 量表应用于城市化居民生命质量评价的信度和效度分析. 中华流行病学杂志, 2016, 37(3): 344-347.
8. 张莎, 田晋, 刘巧兰, 等. 流动人口 SF-12 生命质量量表信度、效度评价. 中国公共卫生, 2011, 27(2): 226-227.
9. 江梅. 总体相关系数比较的假设检验. 中国卫生统计, 2010, 27(1): 83-84.
10. Li H, Xie R, Wang Y, et al. A new scale for the evaluation of clinical practice guidelines applicability: development and appraisal. Implement Sci, 2018, 13(1): 61.
11. Collaboration A. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care, 2003, 12(1): 18-23.
12. Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ, 2010, 182(10): 1045-1052.
13. Brouwers MC, Kho ME, Browman GP, Burgers JS, et al. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ, 2010, 182(10): E472-478.
14. Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ, 2010, 182(18): E839-842.

Chinese Journal of Evidence-Based Medicine

Reliability and validity analysis of guideline clinical applicability evaluation tools

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content