Reliability of narrative assessment data on communication skills in a summative OSCE

PATIENT EDUCATION AND COUNSELING

卷:102

Article

Wilby, Kyle John^1,4  Govaerts, Marjan J. B.²  Dolmans, Diana H. J. M.²  Austin, Zubin³  van der Vleuten, Cees² 

[1] Qatar Univ, Coll Pharm, POB 2713, Doha, Qatar

[2] Maastricht Univ, Fac Hlth Med & Life Sci, Sch Hlth Profess Educ, Univ Singel 60, NL-6229 ER Maastricht, Netherlands

[3] Univ Toronto, Leslie & Fac Pharm, 144 Coll St, Toronto, ON M5S 3M2, Canada

[4] Univ Otago, Sch Pharm, POB 56,18 Frederick St, Dunedin 9054, New Zealand

关键词: Assessment;    Communication;    Medical education;    Pharmacy education;   

DOI  :  10.1016/j.pec.2019.01.018

来源: Elsevier

【 摘 要 】

Objective: To quantitatively estimate the reliability of narrative assessment data regarding student communication skills obtained from a summative OSCE and to compare reliability to that of communication scores obtained from direct observation. Methods: Narrative comments and communication scores (scale 1-5) were obtained for 14 graduating pharmacy students across 6 summative OSCE stations with 2 assessors per station who directly observed student performance. Two assessors who had not observed the OSCE reviewed narratives and independently scored communication skills according to the same 5-point scale. Generalizability theory was used to estimate reliability. Correlation was used to evaluate the relationship between scores from each assessment method. Results: A total of 168 narratives and communication scores were obtained. The G-coefficients were 0.571 for scores provided by assessors present during the OSCE and 0.612 for scores from assessors who provided scores based on narratives only. Correlation between the two sets of scores was 0.5. Conclusion: Reliability of communication scores is not dependent on whether assessors directly observe student performance or assess written narratives, yet both conditions appear to measure communication skills somewhat differently. Practice implications: Narratives may be useful for summative decision-making and help overcome the current limitations of using solely quantitative scores. (C) 2019 Elsevier B.V. All rights reserved.

【 授权许可】

Free