How to set the bar in competency-based medical education: standard setting after an Objective Structured Clinical Examination (OSCE)

BMC Medical Education

Research Article

David Wasserstein¹  Brian Hodges²  Charlotte Ringsted²  Kulamakan Mahan Kulasegaram²  Sarah Wright³  Jaskarndip Chahal⁴  Darrell Ogilvie-Harris⁴  John Theodoropoulos⁵  Tim Dwyer⁵ 

[1] Sunnybrook Health Sciences Centre, 2075 Bayview Ave, M4N 3 M5, Toronto, Canada;The Wilson Centre, 200 Elizabeth St, M5G 2C4, Toronto, Canada;Toronto East General, 825 Coxell Avenue, M4C 3E7, Toronto, Canada;Women’s College Hospital, 76 Grenville St, M5S 1B1, Toronto, Canada;Women’s College Hospital, 76 Grenville St, M5S 1B1, Toronto, Canada;Mt Sinai Hospital, 600 University Avenue, M5G 1X5, Toronto, Canada;

关键词: Objective Structure Clinical Examination;    Senior Resident;    Junior Resident;    Pass Mark;    Progress Testing;   

DOI  :  10.1186/s12909-015-0506-z

 received in 2015-08-14, accepted in 2015-12-08,  发布年份 2016

来源: Springer

【 摘 要 】

BackgroundThe goal of the Objective Structured Clinical Examination (OSCE) in Competency-based Medical Education (CBME) is to establish a minimal level of competence. The purpose of this study was to 1) to determine the credibility and acceptability of the modified Angoff method of standard setting in the setting of CBME, using the Borderline Group (BG) method and the Borderline Regression (BLR) method as a reference standard; 2) to determine if it is feasible to set different standards for junior and senior residents, and 3) to determine the desired characteristics of the judges applying the modified Angoff method.MethodsThe results of a previous OSCE study (21 junior residents, 18 senior residents, and six fellows) were used. Three groups of judges performed the modified Angoff method for both junior and senior residents: 1) sports medicine surgeons, 2) non-sports medicine orthopedic surgeons, and 3) sports fellows. Judges defined a borderline resident as a resident performing at a level between competent and a novice at each station. For each checklist item, the judges answered yes or no for “will the borderline/advanced beginner examinee respond correctly to this item?” The pass mark was calculated by averaging the scores. This pass mark was compared to that created using both the BG and the BLR methods.ResultsA paired t-test showed that all examiner groups expected senior residents to get significantly higher percentage of checklist items correct compared to junior residents (all stations p < 0.001). There were no significant differences due to judge type. For senior residents, there were no significant differences between the cut scores determined by the modified Angoff method and the BG/BLR method. For junior residents, the cut scores determined by the modified Angoff method were lower than the cut scores determined by the BG/BLR Method (all p < 0.01).ConclusionThe results of this study show that the modified Angoff method is an acceptable method of setting different pass marks for senior and junior residents. The use of this method enables both senior and junior residents to sit the same OSCE, preferable in the regular assessment environment of CBME.

【 授权许可】

CC BY   
© Dwyer et al. 2015