Reliability of Objective Structured Clinical Examination: A Meta-Analysis_Chinese Journal of Evidence-Based Medicine

Authors：

 WANGHai-ping ¹ , XIANGLiang-cheng ¹ , ZHANGWei ¹ , YUANJia-ying ¹ , LIMei ² ,  FENGPing ^1,2

1. Department of Statistics, West China School of Public Health, Sichuan University, Chengdu 610041, China;
2. National Institution of Clinical Trial of Drug, West China Hospital, Sichuan University, Chengdu 610041, China;

Corresponding author：

FENGPing, Email: pfyq@yahoo.com

Keywords：

Objective structured clinical examination; Reliability; Coefficient of Cronbach's α; Meta-analysis

DOI：

10.7507/1672-2531.20140145

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To conduct meta-analysis with the reliability of objective structured clinical examination (OSCE), so as to the reliability of OSCE used in the tests for evaluating clinical capacities of medical students. Methods Articles about evaluating clinical capacities of medical students using OSCE and using comprehensive coefficient of Cronbach's α to measure the reliability of OSCE were electronically searched in PubMed, ScienceDirect, CNKI, WanFang data and VIP from Jan. 1998 to May. 2013. Two reviewers screened literature according to the inclusion and exclusion criteria, extracted data, and assessed the methodological quality of the included studies. Then meta-analysis was performed using SSPS 17.0 software. Results Thirty-four studies involving 53 coefficients of Cronbach's α were included, of which, there were 18 articles written in English involving 28 coefficients of Cronbach's α and 16 articles written in Chinese involving 25 coefficients of Cronbach's α. The results of meta-analysis showed that:the total coefficient of Cronbach's α of OSCE was 0.700 (95%CI 0.660 to 0.737). The coefficient of Cronbach's α of internationally-published literature was 0.745 (95%CI 0.696 to 0.790) and that of nationally-published literature was 0.648 (95%CI 0.584 to 0.705), with a significant difference between two groups. Conclusion Currently, the reliability of internationally/nationally-used OSCE is 0.7, which has relatively better reliability when used in the tests for evaluating clinical capacities of medical students at home and aboard. However, the reliability of OSCE at abroad is fairly better than that at home.

Citation： WANGHai-ping, XIANGLiang-cheng, ZHANGWei, YUANJia-ying, LIMei, FENGPing. Reliability of Objective Structured Clinical Examination: A Meta-Analysis. Chinese Journal of Evidence-Based Medicine, 2014, 14(7): 878-883. doi: 10.7507/1672-2531.20140145 Copy

1.	Harden RM, Stevenson M, Downie WW, et al. Assessment of clinical competence using objective structured examination. BMJ, 1975, 22(1):447-451.
2.	Sidhu RS, Grober ED, Musselman LJ, et al. Assessing competency in surgery:Where to begin. Surgery, 2004, 135(1):6-20.
3.	Wanzel KR, Ward M, Reznick RK, et al. Teaching the surgical craft:From selection to certification. Curr Probl Surg, 2002, 39(6):573-659.
4.	曾五一, 黄炳艺.调查问卷的可信度和有效度分析.统计与信息论坛, 2005, 20(6):11-15.
5.	Schultz JH, Nikendei C, Weyrich P, et al. Quality assurance of assessments using the example of the OSCE examination format:experiences of the Medical School of Heidelberg University. Z Evid Fortbild Qual Gesundhwes, 2008, 102(10):668-672.
6.	邹扬, 缪青, 芦开芳, 等.本科和长学制毕业考试中客观结构化临床考试的应用.上海交通大学学报(医学版), 2008, 28(z1):71-75.
7.	Corrie K, Wiles M, Flack J, et al. A novel objective structured clinical examination for the assessment of transfusion practice in anaesthesia. Clin Teach, 2011, 8(2):97-100.
8.	Rodriguez MC, Maeda Y. Meta-analysis of coefficient alpha. Psychol Methods, 2006, 11(3):306-322.
9.	Eberhard L, Hassel A, Bäumer A, et al. Analysis of quality and feasibility of an objective structured clinical examination (OSCE) in preclinical dental education. Eur J Dent Educ, 2011, 15(3):172-178.
10.	Yang YY, LEE FY, Hsu HC, et al. A core competence-based objective structured clinical examination (OSCE) in evaluation of clinical performance of postgraduate year-1 (PGY-1) residents. J Chin Med Assoc, 2011, 74(5):198-204.
11.	Pandya JS, Bhagwat SM, Kini SL, et al. Evaluation of clinical skills for first-year surgical residents using orientation programme and objective structured clinical evaluation as a tool of assessment. J Postgrad Med, 2010, 56(4):297-300.
12.	Yu TC, Wheeler BR, Hill AG. Effectiveness of standardized clerkship teaching across multiple sites. J Surg Res, 2011, 168(1):e17-e23.
13.	Shehmar M, Cruikshank M, Finn C, et al. A validity study of the national UK colposcopy objective structured clinical examination:is it a test fit for purpose? BJOG, 2009, 116(13):1796-1999.
14.	Heckmann JG, Knossalla F, Gollwitzer S, et al. OSCE in the neurology clerkship. Experiences at the neurological department of the university hospital Erlangen. Fortschr Neurol Psychiatr, 2009, 77(1):32-37.
15.	Huang CC, Chan CY, Wu CL, et al. Assessment of Clinical Competence of Medical Students Using the Objective Structured Clinical Examination:First 2 Years' Experience in Taipei Veterans General Hospital. J Chin Med Assoc, 2010, 73(11):589-595.
16.	Pedregal GM, Molina FE, Prados Castillejo JA, et al. Evaluation of clinical competence of family medicine trainer. Aten Primaria, 2004, 34(2):68-72.
17.	Wessel J, Williams R, Finch E, et al. Reliability and validity of an objective structured clinical examination for physical therapy students. J Allied Health, 2003, 32(4):266-269.
18.	Rothman Al, Cusimano M. A comparison of physician examiners', standardized patients', and communication experts' ratings of international medical graduates' English proficiency. Acad Med, 2000, 75(12):1206-1211.
19.	Schwartz RW, Witzke DB, Donnelly MB, et al. Assessing residents' clinical performance:cumulative results of a four-year study with the Objective Structured Clinical Examination. Surgery, 1998, 124(2):307-312.
20.	Sidhu RS, Mcllroy JH, Regehr G, et al. Using a Comprehensive Examination to Assess Multiple Competencies in Surgical Residents:Does the Oral Examination Still Have a Role? J Am Coll Surg, 2001(5):754-758.
21.	Bosse HM, Schultz JH, Nickel M, et al. The effect of using standardized patients or peer role play on ratings of undergraduate communication training:A randomized controlled trial. Patient Educ Couns, 2012, 87(3):300-306.
22.	Yang YY, LEE FY, Hsu HC, et al. Assessment of first-year post-graduate residents:Usefulness of multiple tools. J Chin Med Assoc, 2011, 74(12):531-538.
23.	MacRae H, Regehr G, Leadbetter W, et al. A comprehensive examination for senior surgical residents. Am J Surg, 2000, 179(3):190-193.
24.	Zabar S, Hanley K, Stevens DL, et al. Can interactive skills-based seminars with standardized patients enhance clinicians' prevention skills? Measuring the impact of a CME program. Patient Educ Couns, 2010, 80(2):248-252.
25.	Ault G, Rezinck R, MacRae H, et al. Exporting a technical skills evaluation technology to other sites. Am J Surg, 2001, 182(3):254-256.
26.	陈英, 陈淳, 曾凡, 等. OSCE在本科护生临床实习评价中的应用研究.护理学杂志, 2011, 26(15):5-7.
27.	陈瑶, 厉英, 周玲君, 等.简化的客观结构化临床考核在《急救护理学》技能考核中的应用.解放军护理杂志, 2011, 28(5B):67-70.
28.	郑克勤, 孟庆勇, 刘琼玲, 等.护理学专业本科毕业技能考试改革的探索与实践.广东医学院学报, 2012, 30(5):580-583.
29.	安力彬, 李文涛, 张萍.客观结构化临床考试在护理本科生毕业考核中的应用研究.中国高等医学, 2009, (3):16-17.
30.	黄烨, 王斌全, 商临萍.客观结构化临床考试在护理本科生临床综合能力评价中的应用研究, 护理研究, 2008, 22(3):567-569.
31.	沈娟, 崔焱, 林征, 等.客观结构化临床考试在护理本科毕业生临床能力评价中的应用研究.护理进修杂志, 2009, 24(5):395-397.
32.	路中, 戚丽, 李贵新, 等.内科标准化病人在评价实习医师临床技能中的应用.西北医学教育, 2009, 17(2):308-310.
33.	刘敏, 李颖.应用客观结构化临床考试评价护理本科毕业生的临床能力.中华护理教育, 2011, 8(8):348-349.
34.	吴建军, 刘革新, 陈三妹, 等.客观结构化临床考试在内科护理学中的应用.齐鲁护理杂志, 2010, 16(17):109-111.
35.	吴末生, 殷朝阳, 彭红, 等.从多站考试的成绩分析探讨临床能力评价的有效途径.中国高等医学教育, 2000, (2):6-8.
36.	于晓松, 孙宝志, 时瑾, 等.对医学生临床综合能力评价的初步研究.中国高等医学教育, 2000, (2):9-10.
37.	张艳, 于兰贞, 马国平, 等.客观结构化临床考试在ICU专科护生临床能力评价中的应用研究.护理研究, 2012, 26(3):656-659.
38.	朱滨海, 占伊扬, 孙宁生, 等.客观结构化临床考试在江苏省住院医师培训阶段考核中的应用与分析.中国医院, 2012, 116(12):56-59.
39.	孙玮.国家临床医师资格考试理论考试的实证效度研究.沈阳:中国医科大学, 2010.
40.	Hedges LV, Vevea JL. Fixed-and random-effects models in meta-analysis. Psych Meth, 1998, 3(4):486-504.
41.	罗海波, 胡靖, 王安莉.浅析如何提高OSCE质量.中国中医药现代远程教育, 2007, 5(8):58-59.
42.	许克祥, 齐建强, 刘赟.中医院校开展客观结构化临床考试的思考与探索.黑龙江医药科学, 2012, 35(1):63-63.
43.	鲁威, 张剑戈, 缪青, 等.客观结构化考试的信度及主成分分析.上海交通大学学报(医学版), 2008, 28(12):1577-1580.
44.	Hofer M, Kamper L, Sadio M, et al. Evaluation of an OSCE assessment tool for abdominal ultrasound courses. Ultraschall Med, 2011, 32(2):184-190.
45.	Sloan DA, Donnelly MB, Schwartz RW, et al. The Objective Structured Clinical Examination:The New Gold Standard for Evaluating Postgraduate Clinical Performance. Ann Surg, 1995, 222(6):735-742.
46.	高镭, 李晓松, 万学红, 等.客观结构化临床考试的多元概化模型研究.中国高等医学教育, 2004, 3(1):46-49.

1. Harden RM, Stevenson M, Downie WW, et al. Assessment of clinical competence using objective structured examination. BMJ, 1975, 22(1):447-451.
2. Sidhu RS, Grober ED, Musselman LJ, et al. Assessing competency in surgery:Where to begin. Surgery, 2004, 135(1):6-20.
3. Wanzel KR, Ward M, Reznick RK, et al. Teaching the surgical craft:From selection to certification. Curr Probl Surg, 2002, 39(6):573-659.
4. 曾五一, 黄炳艺.调查问卷的可信度和有效度分析.统计与信息论坛, 2005, 20(6):11-15.
5. Schultz JH, Nikendei C, Weyrich P, et al. Quality assurance of assessments using the example of the OSCE examination format:experiences of the Medical School of Heidelberg University. Z Evid Fortbild Qual Gesundhwes, 2008, 102(10):668-672.
6. 邹扬, 缪青, 芦开芳, 等.本科和长学制毕业考试中客观结构化临床考试的应用.上海交通大学学报(医学版), 2008, 28(z1):71-75.
7. Corrie K, Wiles M, Flack J, et al. A novel objective structured clinical examination for the assessment of transfusion practice in anaesthesia. Clin Teach, 2011, 8(2):97-100.
8. Rodriguez MC, Maeda Y. Meta-analysis of coefficient alpha. Psychol Methods, 2006, 11(3):306-322.
9. Eberhard L, Hassel A, Bäumer A, et al. Analysis of quality and feasibility of an objective structured clinical examination (OSCE) in preclinical dental education. Eur J Dent Educ, 2011, 15(3):172-178.
10. Yang YY, LEE FY, Hsu HC, et al. A core competence-based objective structured clinical examination (OSCE) in evaluation of clinical performance of postgraduate year-1 (PGY-1) residents. J Chin Med Assoc, 2011, 74(5):198-204.
11. Pandya JS, Bhagwat SM, Kini SL, et al. Evaluation of clinical skills for first-year surgical residents using orientation programme and objective structured clinical evaluation as a tool of assessment. J Postgrad Med, 2010, 56(4):297-300.
12. Yu TC, Wheeler BR, Hill AG. Effectiveness of standardized clerkship teaching across multiple sites. J Surg Res, 2011, 168(1):e17-e23.
13. Shehmar M, Cruikshank M, Finn C, et al. A validity study of the national UK colposcopy objective structured clinical examination:is it a test fit for purpose? BJOG, 2009, 116(13):1796-1999.
14. Heckmann JG, Knossalla F, Gollwitzer S, et al. OSCE in the neurology clerkship. Experiences at the neurological department of the university hospital Erlangen. Fortschr Neurol Psychiatr, 2009, 77(1):32-37.
15. Huang CC, Chan CY, Wu CL, et al. Assessment of Clinical Competence of Medical Students Using the Objective Structured Clinical Examination:First 2 Years' Experience in Taipei Veterans General Hospital. J Chin Med Assoc, 2010, 73(11):589-595.
16. Pedregal GM, Molina FE, Prados Castillejo JA, et al. Evaluation of clinical competence of family medicine trainer. Aten Primaria, 2004, 34(2):68-72.
17. Wessel J, Williams R, Finch E, et al. Reliability and validity of an objective structured clinical examination for physical therapy students. J Allied Health, 2003, 32(4):266-269.
18. Rothman Al, Cusimano M. A comparison of physician examiners', standardized patients', and communication experts' ratings of international medical graduates' English proficiency. Acad Med, 2000, 75(12):1206-1211.
19. Schwartz RW, Witzke DB, Donnelly MB, et al. Assessing residents' clinical performance:cumulative results of a four-year study with the Objective Structured Clinical Examination. Surgery, 1998, 124(2):307-312.
20. Sidhu RS, Mcllroy JH, Regehr G, et al. Using a Comprehensive Examination to Assess Multiple Competencies in Surgical Residents:Does the Oral Examination Still Have a Role? J Am Coll Surg, 2001(5):754-758.
21. Bosse HM, Schultz JH, Nickel M, et al. The effect of using standardized patients or peer role play on ratings of undergraduate communication training:A randomized controlled trial. Patient Educ Couns, 2012, 87(3):300-306.
22. Yang YY, LEE FY, Hsu HC, et al. Assessment of first-year post-graduate residents:Usefulness of multiple tools. J Chin Med Assoc, 2011, 74(12):531-538.
23. MacRae H, Regehr G, Leadbetter W, et al. A comprehensive examination for senior surgical residents. Am J Surg, 2000, 179(3):190-193.
24. Zabar S, Hanley K, Stevens DL, et al. Can interactive skills-based seminars with standardized patients enhance clinicians' prevention skills? Measuring the impact of a CME program. Patient Educ Couns, 2010, 80(2):248-252.
25. Ault G, Rezinck R, MacRae H, et al. Exporting a technical skills evaluation technology to other sites. Am J Surg, 2001, 182(3):254-256.
26. 陈英, 陈淳, 曾凡, 等. OSCE在本科护生临床实习评价中的应用研究.护理学杂志, 2011, 26(15):5-7.
27. 陈瑶, 厉英, 周玲君, 等.简化的客观结构化临床考核在《急救护理学》技能考核中的应用.解放军护理杂志, 2011, 28(5B):67-70.
28. 郑克勤, 孟庆勇, 刘琼玲, 等.护理学专业本科毕业技能考试改革的探索与实践.广东医学院学报, 2012, 30(5):580-583.
29. 安力彬, 李文涛, 张萍.客观结构化临床考试在护理本科生毕业考核中的应用研究.中国高等医学, 2009, (3):16-17.
30. 黄烨, 王斌全, 商临萍.客观结构化临床考试在护理本科生临床综合能力评价中的应用研究, 护理研究, 2008, 22(3):567-569.
31. 沈娟, 崔焱, 林征, 等.客观结构化临床考试在护理本科毕业生临床能力评价中的应用研究.护理进修杂志, 2009, 24(5):395-397.
32. 路中, 戚丽, 李贵新, 等.内科标准化病人在评价实习医师临床技能中的应用.西北医学教育, 2009, 17(2):308-310.
33. 刘敏, 李颖.应用客观结构化临床考试评价护理本科毕业生的临床能力.中华护理教育, 2011, 8(8):348-349.
34. 吴建军, 刘革新, 陈三妹, 等.客观结构化临床考试在内科护理学中的应用.齐鲁护理杂志, 2010, 16(17):109-111.
35. 吴末生, 殷朝阳, 彭红, 等.从多站考试的成绩分析探讨临床能力评价的有效途径.中国高等医学教育, 2000, (2):6-8.
36. 于晓松, 孙宝志, 时瑾, 等.对医学生临床综合能力评价的初步研究.中国高等医学教育, 2000, (2):9-10.
37. 张艳, 于兰贞, 马国平, 等.客观结构化临床考试在ICU专科护生临床能力评价中的应用研究.护理研究, 2012, 26(3):656-659.
38. 朱滨海, 占伊扬, 孙宁生, 等.客观结构化临床考试在江苏省住院医师培训阶段考核中的应用与分析.中国医院, 2012, 116(12):56-59.
39. 孙玮.国家临床医师资格考试理论考试的实证效度研究.沈阳:中国医科大学, 2010.
40. Hedges LV, Vevea JL. Fixed-and random-effects models in meta-analysis. Psych Meth, 1998, 3(4):486-504.
41. 罗海波, 胡靖, 王安莉.浅析如何提高OSCE质量.中国中医药现代远程教育, 2007, 5(8):58-59.
42. 许克祥, 齐建强, 刘赟.中医院校开展客观结构化临床考试的思考与探索.黑龙江医药科学, 2012, 35(1):63-63.
43. 鲁威, 张剑戈, 缪青, 等.客观结构化考试的信度及主成分分析.上海交通大学学报(医学版), 2008, 28(12):1577-1580.
44. Hofer M, Kamper L, Sadio M, et al. Evaluation of an OSCE assessment tool for abdominal ultrasound courses. Ultraschall Med, 2011, 32(2):184-190.
45. Sloan DA, Donnelly MB, Schwartz RW, et al. The Objective Structured Clinical Examination:The New Gold Standard for Evaluating Postgraduate Clinical Performance. Ann Surg, 1995, 222(6):735-742.
46. 高镭, 李晓松, 万学红, 等.客观结构化临床考试的多元概化模型研究.中国高等医学教育, 2004, 3(1):46-49.

Previous Article
Distribution of Aminoglycoside-resistant Genes in Pseudomonas aeruginosa in China: A Literature Study
Next Article
Differentiation and Handling of Homogeneity in Network Meta-analysis

Chinese Journal of Evidence-Based Medicine

Reliability of Objective Structured Clinical Examination: A Meta-Analysis

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content