Rating and ranking preparedness characteristics important for veterinary workplace clinical training: a novel application of pairwise comparisons and the Elo algorithm

Frontiers in Medicine

Medicine

Sarah Wood¹  Sheena Warman¹  Vishna Devi Nadarajah²  Cornélie Westermann³  Patricia Pawson⁴  John Remnant⁵  Sharmini Julita Paramasivam⁶  Jennifer Routh⁶  Peter Cockcroft⁶  Kamalan Jeevaratnam⁶  Alison Reid⁷ 

[1] Bristol Veterinary School, University of Bristol, Langford, United Kingdom;Division of Human Biology, School of Medicine and IMU Centre for Education, International Medical University, Kuala Lumpur, Malaysia;Faculty of Veterinary Medicine, Utrecht University, Utrecht, Netherlands;School of Biodiversity, One Health and Veterinary Medicine, University of Glasgow, Glasgow, United Kingdom;School of Veterinary Medicine and Science, University of Nottingham, Sutton Bonington, United Kingdom;School of Veterinary Medicine, University of Surrey, Guildford, United Kingdom;School of Veterinary Science, University of Liverpool, Neston, United Kingdom;

关键词: survey;    questionnaire;    Likert;    comparison;    rating;    ranking;    preparedness;    methods;   

DOI  :  10.3389/fmed.2023.1128058

 received in 2022-12-20, accepted in 2023-03-30,  发布年份 2023

来源: Frontiers

【 摘 要 】

Quantitatively eliciting perspectives about a large number of similar entities (such as a list of competences) is a challenge for researchers in health professions education (HPE). Traditional survey methods may include using Likert items. However, a Likert item approach that generates absolute ratings of the entities may suffer from the “ceiling effect,” as ratings cluster at one end of the scale. This impacts on researchers’ ability to detect differences in ratings between the entities themselves and between respondent groups. This paper describes the use of pairwise comparison (this or that?) questions and a novel application of the Elo algorithm to generate relative ratings and rankings of a large number of entities, on a unidimensional scale. A study assessing the relative importance of 91 student “preparedness characteristics” for veterinary workplace clinical training (WCT) is presented as an example of this method in action. The Elo algorithm uses pairwise comparison responses to generate an importance rating for each preparedness characteristic on a scale from zero to one. This is continuous data with measurement variability which, by definition, spans an entire spectrum and is not susceptible to the ceiling effect. The output should allow for the detection of differences in perspectives between groups of survey respondents (such as students and workplace supervisors) which Likert ratings may be insensitive to. Additional advantages of the pairwise comparisons are their low susceptibility to systematic bias and measurement error, they can be quicker and arguably more engaging to complete than Likert items, and they should carry a low cognitive load for respondents. Methods for evaluating the validity and reliability of this survey design are also described. This paper presents a method that holds great potential for a diverse range of applications in HPE research. In the pursuit quantifying perspectives on survey items which are measured on a relative basis and a unidimensional scale (e.g., importance, priority, probability), this method is likely to be a valuable option.

【 授权许可】

Unknown   
Copyright © 2023 Routh, Paramasivam, Cockcroft, Wood, Remnant, Westermann, Reid, Pawson, Warman, Nadarajah and Jeevaratnam.