Reliability and validity analysis of Guideline Implementation Success Assessment Tool (A-GIST)_Chinese Journal of Evidence-Based Medicine

Authors：

YANG Nan ^1,2,3,4,5 , ZOU Kun ^2,3,4,5 , HE Siyi ^1,2,3,4,5 , ZENG Linan ^2,3,4,5 , LI Hailong ^2,3,4,5 , HUANG Liang ^2,3,4,5 , YI Qiusha ^2,3,4,5 , ZHANG Mingyue ⁶ , HUANG Chao ⁷ ,  WANG Qiang ⁷ ,  ZHANG Lingli ^2,3,4,5,8

1. West China School of Pharmacy, Sichuan University, Chengdu 610041, P. R. China;
2. Department of Pharmacy, West China Second University Hospital, Sichuan University, Chengdu 610041, P. R. China;
3. Evidence-based Pharmacy Center, West China Second University Hospital, Sichuan University, Chengdu 610041, P. R. China;
4. NMPA Key Laboratory for Technical Research on Drug Products in vitro and in vivo Correlation, Chengdu 610041, P. R. China;
5. Key Laboratory of Birth Defects and Related Diseases of Women and Children, Ministry of Education, Chengdu 610041, P. R. China;
6. School of Stomatology, Chinese Medical University, Shenyang 110002, P. R. China;
7. Medical Management Service Guidance Center, National Health Commission of the People’s Republic of China, Beijing 100044, P. R. China;
8. Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, Chengdu 610041, P. R. China;

Corresponding author：

WANG Qiang, Email: jason2019@sina.cn; ZHANG Lingli, Email: zhanglingli@scu.edu.cn

Keywords：

Guideline Implementation Success Assessment Tool; Reliability; Validity

DOI：

10.7507/1672-2531.202308066

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To evaluate the reliability and validity of Guideline Implementation Success Assessment Tool (A-GIST). Methods With the guideline for the prevention and treatment of type 2 diabetes mellitus in China (2020 edition) as the target guideline, health care providers and patients from different hospitals across the country were investigated by questionnaire using A-GIST. Spearman-Brown coefficient and Cronbach's α coefficient were used to evaluate the split-half reliability and internal consistency reliability, while the structural validity, convergent validity and discriminant validity were investigated by confirmatory factor analysis based on structural equation. Results The internal consistency reliability and split half reliability coefficients of the whole tool and each dimension ranged from 0.650 to 0.986. The scale-level content validity index (S-CVI/Ave) of content validity was 0.846. Confirmatory factor analysis showed that, the χ²/df of two sections of the tool were 8.695 and 6.123, respectively. The root mean square residual (RMR), the standard root mean square residual (SRMR) and the root mean square error of approximation (RMSEA) were under or almost under the threshold. Besides, the goodness-of-fit index (GFI) of them were 0.901 and 0.822, the adjusted goodness-of-fit index (AGFI) were 0.836 and 0.787, and the parsimonious normed fit index (PNFI) were 0.545 and 0.788, respectively. Conclusion Guideline Implementation Success Assessment Tool (A-GIST) was proved to be valid and reliable, and it shows that it is necessary to optimize the items under the dimensions of maintenance and evaluation of diagnosis and treatment effect in the future.

Citation： YANG Nan, ZOU Kun, HE Siyi, ZENG Linan, LI Hailong, HUANG Liang, YI Qiusha, ZHANG Mingyue, HUANG Chao, WANG Qiang, ZHANG Lingli. Reliability and validity analysis of Guideline Implementation Success Assessment Tool (A-GIST). Chinese Journal of Evidence-Based Medicine, 2024, 24(8): 904-909. doi: 10.7507/1672-2531.202308066 Copy

1.	Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical practice guidelines we can trust. Washington DC: National Academies Press, 2011.
2.	Pronovost PJ. Enhancing physicians' use of clinical guidelines. JAMA, 2013, 310(23): 2501-2502.
3.	中华医学会杂志社指南与标准研究中心, 中国医学科学院循证评价与指南研究创新单元, 世界卫生组织指南实施与知识转化合作中心. 2021年医学期刊发表中国指南和共识的科学性、透明性和适用性的评级. 中华医学杂志, 2022, 102(30): 2319-2328.
4.	郭强强, 荀杨芹, 刘辉, 等. 临床实践指南的注册. 协和医学杂志, 2023, 14(1): 203-208.
5.	Liu M, Zhang C, Zha Q, et al. A national survey of Chinese medicine doctors and clinical practice guidelines in China. BMC Complement Altern Med, 2017, 17(1): 451.
6.	陈耀龙, 史乾灵, 赵俊强, 等. 从知到行: 跨越指南理论与实践的鸿沟. 协和医学杂志, 2020, 11(6): 746-753.
7.	Shiffman RN, Dixon J, Brandt C, et al. The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak, 2005, 5: 23.
8.	Ierano C, Ayton D, Peel T, et al. Evaluating the implementability of antibiotic surgical prophylaxis guidelines. Infect Dis Health, 2020, 25(1): 11-21.
9.	Maue SK, Segal R, Kimberlin CL, et al. Predicting physician guideline compliance: an assessment of motivators and perceived barriers. Am J Manag Care, 2004, 10(6): 383-391.
10.	Correa VC, Lugo-Agudelo LH, Aguirre-Acevedo DC, et al. Individual, health system, and contextual barriers and facilitators for the implementation of clinical practice guidelines: a systematic metareview. Health Res Policy Syst, 2020, 18(1): 74.
11.	Stewart D, Al Hail M, Al-Shaibi S, et al. A scoping review of theories used to investigate clinician adherence to clinical practice guidelines. Int J Clin Pharm, 2023, 45(1): 52-63.
12.	Deng Q, Liu W. Utilization of clinical practice guideline on antimicrobial in China: an exploratory survey on multilevel determinants. BMC Health Serv Res, 2020, 20(1): 282.
13.	孙振球, 徐勇勇. 医学统计学. 第4版. 北京: 人民卫生出版社, 2020.
14.	李灿, 辛玲. 调查问卷的信度与效度的评价方法研究. 中国卫生统计, 2008, 25(5): 541-544.
15.	史静琤, 莫显昆, 孙振球. 量表编制中内容效度指数的应用. 中南大学学报(医学版), 2012, 37(2): 152-155.
16.	中华医学会糖尿病学分会. 中国2型糖尿病防治指南(2020年版). 中华糖尿病杂志, 2021, 13(4): 315-409.
17.	Wiercioch W, Akl EA, Santesso N, et al. Assessing the process and outcome of the development of practice guidelines and recommendations: PANELVIEW instrument development. CMAJ, 2020, 192(40): E1138-E1145.
18.	Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med, 2010, 7(2): e1000217.
19.	黄超, 李海龙, 张川, 等. 指南临床适用性评价实证研究: 以糖尿病指南临床适用性评价为例. 中国循证医学杂志, 2020, 20(5): 536-542.
20.	Tabachnick BG, Fidell LS. Using multivariate statistics (5th Edition). New York: Allyn and Bacon, 2007.
21.	张淼, 刁莎, 曾力楠, 等. 指南临床适用性评价工具(2. 0版)信效度分析. 中国循证医学杂志, 2023, 23(1): 80-84.
22.	李海龙, 张川, 黄超, 等. 指南临床适用性评价工具信效度分析. 中国循证医学杂志, 2020, 20(4): 378-382.
23.	付会斌, 林小鹏, 孔丹莉, 等. 世界卫生组织生存质量测定简表在高校学科带头人生存质量评价中的信度与效度研究. 山西医药杂志, 2012, 41(9): 867-869.
24.	邱皓政, 林碧芳. 结构方程模型的原理与应用(第二版). 北京: 中国轻工业出版社, 2019.
25.	汪亚萍, 景文展, 吴俣, 等. SF-12量表评价我国接受ART的HIV/AIDS患者生存质量的信效度分析. 中国艾滋病性病, 2022, 28(4): 411-415.
26.	徐迪, 端琼丽, 潘玲, 等. WHOQOL-BREF用于评价HIV感染者生活质量的信、效度研究. 中华疾病控制杂志, 2022, 26(5): 576-582.
27.	Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ, 2010, 182(10): 1045-1052.
28.	AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care, 2003, 12(1): 18-23.
29.	曾力楠, 易秋莎, 张川, 等. 制订指南临床适用性评价工具. 中国循证医学杂志, 2020, 20(4): 466-475.

1. Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical practice guidelines we can trust. Washington DC: National Academies Press, 2011.
2. Pronovost PJ. Enhancing physicians' use of clinical guidelines. JAMA, 2013, 310(23): 2501-2502.
3. 中华医学会杂志社指南与标准研究中心, 中国医学科学院循证评价与指南研究创新单元, 世界卫生组织指南实施与知识转化合作中心. 2021年医学期刊发表中国指南和共识的科学性、透明性和适用性的评级. 中华医学杂志, 2022, 102(30): 2319-2328.
4. 郭强强, 荀杨芹, 刘辉, 等. 临床实践指南的注册. 协和医学杂志, 2023, 14(1): 203-208.
5. Liu M, Zhang C, Zha Q, et al. A national survey of Chinese medicine doctors and clinical practice guidelines in China. BMC Complement Altern Med, 2017, 17(1): 451.
6. 陈耀龙, 史乾灵, 赵俊强, 等. 从知到行: 跨越指南理论与实践的鸿沟. 协和医学杂志, 2020, 11(6): 746-753.
7. Shiffman RN, Dixon J, Brandt C, et al. The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak, 2005, 5: 23.
8. Ierano C, Ayton D, Peel T, et al. Evaluating the implementability of antibiotic surgical prophylaxis guidelines. Infect Dis Health, 2020, 25(1): 11-21.
9. Maue SK, Segal R, Kimberlin CL, et al. Predicting physician guideline compliance: an assessment of motivators and perceived barriers. Am J Manag Care, 2004, 10(6): 383-391.
10. Correa VC, Lugo-Agudelo LH, Aguirre-Acevedo DC, et al. Individual, health system, and contextual barriers and facilitators for the implementation of clinical practice guidelines: a systematic metareview. Health Res Policy Syst, 2020, 18(1): 74.
11. Stewart D, Al Hail M, Al-Shaibi S, et al. A scoping review of theories used to investigate clinician adherence to clinical practice guidelines. Int J Clin Pharm, 2023, 45(1): 52-63.
12. Deng Q, Liu W. Utilization of clinical practice guideline on antimicrobial in China: an exploratory survey on multilevel determinants. BMC Health Serv Res, 2020, 20(1): 282.
13. 孙振球, 徐勇勇. 医学统计学. 第4版. 北京: 人民卫生出版社, 2020.
14. 李灿, 辛玲. 调查问卷的信度与效度的评价方法研究. 中国卫生统计, 2008, 25(5): 541-544.
15. 史静琤, 莫显昆, 孙振球. 量表编制中内容效度指数的应用. 中南大学学报(医学版), 2012, 37(2): 152-155.
16. 中华医学会糖尿病学分会. 中国2型糖尿病防治指南(2020年版). 中华糖尿病杂志, 2021, 13(4): 315-409.
17. Wiercioch W, Akl EA, Santesso N, et al. Assessing the process and outcome of the development of practice guidelines and recommendations: PANELVIEW instrument development. CMAJ, 2020, 192(40): E1138-E1145.
18. Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med, 2010, 7(2): e1000217.
19. 黄超, 李海龙, 张川, 等. 指南临床适用性评价实证研究: 以糖尿病指南临床适用性评价为例. 中国循证医学杂志, 2020, 20(5): 536-542.
20. Tabachnick BG, Fidell LS. Using multivariate statistics (5th Edition). New York: Allyn and Bacon, 2007.
21. 张淼, 刁莎, 曾力楠, 等. 指南临床适用性评价工具(2. 0版)信效度分析. 中国循证医学杂志, 2023, 23(1): 80-84.
22. 李海龙, 张川, 黄超, 等. 指南临床适用性评价工具信效度分析. 中国循证医学杂志, 2020, 20(4): 378-382.
23. 付会斌, 林小鹏, 孔丹莉, 等. 世界卫生组织生存质量测定简表在高校学科带头人生存质量评价中的信度与效度研究. 山西医药杂志, 2012, 41(9): 867-869.
24. 邱皓政, 林碧芳. 结构方程模型的原理与应用(第二版). 北京: 中国轻工业出版社, 2019.
25. 汪亚萍, 景文展, 吴俣, 等. SF-12量表评价我国接受ART的HIV/AIDS患者生存质量的信效度分析. 中国艾滋病性病, 2022, 28(4): 411-415.
26. 徐迪, 端琼丽, 潘玲, 等. WHOQOL-BREF用于评价HIV感染者生活质量的信、效度研究. 中华疾病控制杂志, 2022, 26(5): 576-582.
27. Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ, 2010, 182(10): 1045-1052.
28. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care, 2003, 12(1): 18-23.
29. 曾力楠, 易秋莎, 张川, 等. 制订指南临床适用性评价工具. 中国循证医学杂志, 2020, 20(4): 466-475.

Chinese Journal of Evidence-Based Medicine

Reliability and validity analysis of Guideline Implementation Success Assessment Tool (A-GIST)

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content