Assessment of Junior Doctor performance: a validation study

BMC Medical Education

Fiona Lake²  Antonio Celenza³  Sandra E Carr¹ 

[1]Education Centre, Faculty of Medicine, Dentistry and Health Sciences, University of Western Australia, MB 515 35, Stirling Hwy, Crawley 6009, Western Australia, Australia

[2]School of Medicine and Pharmacology, Faculty of Medicine, Dentistry and Health Sciences, University of Western Australia, MB 515 35, Stirling Hwy, Crawley 6009, Western Australia, Australia

[3]School of Primary, Aboriginal and Rural Health, Faculty of Medicine, Dentistry and Health Sciences, University of Western Australia, MB 515 35, Stirling Hwy, Crawley 6009, Western Australia, Australia

关键词: Internship;    Undergraduate medical education;    Educational assessment;   

Others  :  1138666
DOI  :  10.1186/1472-6920-13-129

 received in 2013-04-11, accepted in 2013-09-12,  发布年份 2013

【 摘 要 】

Background

In recent years, Australia has developed a National Junior Doctor Curriculum Framework that sets out the expected standards and describes areas of performance for junior doctors and through this has allowed a national approach to junior doctor assessment to develop. Given the significance of the judgments made, in terms of patient safety, development of junior doctors, and preventing progression of junior doctors moving to the next stage of training, it is essential to develop and validate assessment tools as rigorously as possible. This paper reports on a validation study of the Junior Doctor Assessment Tool as used for PGY1 doctors to evaluate the psychometric properties of the instrument and to explore the effect of length of experience as a PGY1 on assessment scores.

Methods

This validation study of the Australian developed Junior Doctor Assessment Tool as it was used in three public and other associated hospitals in Western Australia for PGY1 across a two year period addressed two core aims, namely: (1) to evaluate the psychometric properties of the instrument; (2) to explore the effect of length of experience as a PGY1 on assessment scores.

Results

The highest mean scores were for professional behaviours, teamwork and interpersonal skills and the lowest were for procedures. Most junior doctors were assessed three or more times and scores were not different in the first rotation compared to subsequent rotations. While statistically significant, there appeared to be little practical influence on scores obtained by the number of times they were assessed. Principal component analysis identified two principal components of junior doctor performance are being assessed rather than the commonly reported three. A Cronbach Alpha of .883 was calculated for the 10 item scale.

Conclusions

Now that the components of the tool have been analysed it will be more meaningful and potentially more influential to consider these factors on the potential educational impact of this assessment process for monitoring junior doctor development and progression.

【 授权许可】

   
2013 Carr et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Epstein RM: Assessment in medical education. N Engl J Med 2007, 356(4):387-396.
• [2]MiIler A, Archer J: Impact of workplace based assessment on doctors’ education and performance: a systematic review. BMJ 2010. doi:10.1136/bmj.c5064: 1-6. [Internet].18th February 2013.
• [3]Batalden P, Leach D, Swing S, Dreyfus H, Dreyfus S: General competencies and accreditation in graduate medical education: an antidote to overspecification in the education of medical specialists. Health Aff 2002, 21(5):103-111.
• [4]Norcini J, Burch V: Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach 2007, 29:855-871.
• [5]Warm EJ, Schauer D, Revis B, Boex JR: Multisource feedback in the ambulatory setting. J Grad Med Educ 2010, 2:269-277.
• [6]van der Vleuten C, Schuwirth L: Assessing professional competence: from methods to programmes. Med Educ 2005, 39:309-317.
• [7]Overeem K, Wollersheim HC, Arah OA, Cruijsberg JK, Grol RPTM, Lombarts K: Evaluation of physicians’ professional performance: an iterative development and validation study of multisource feedback instruments. BMC Health Serv Res 2012, 12(80):2-11.
• [8]cpmec: Guidelines for Junior Doctors using the National Assessment tools Australia: Confederation of Postgraduate Medical Education Councils. [cited 2013 February 17th]. Available from: http://www.cpmec.org.au/index.cfm?Do=View.Page&PageID=198#guidelines webcite
• [9]PMETB: Developing and maintaining an assessment system - a PMETB guide to good practice. United Kingdom; 2007.
• [10]Bingham C, Crampton R: A review of prevocational medical trainee assessment in New South Wales. MJA 2011, 195:410-412.
• [11]pmcwa: Postgraduate Medical Council of Western Australia: Junior Doctor Assessment Form 2012. [cited 2012 17th October]. Available from: http://www.pmcwa.health.wa.gov.au/home/ webcite
• [12]Duvivier RJ, van Dalen J, Muijtjens A, Moulaert V, van der Vleuten C, Scherpbier A: The role of deliberate practice in the acquisition of clinical skills. BMC Med Educ 2011., 11Available from: http://www.biomedcentral.com/1472-6920/11/101 webcite
• [13]Streiner D, Norman G: Health measurement scales: a practical guide to their development and use. 4th Edition ed. Oxford: Oxford University Press; 2008.
• [14]Tabachnick BG, Fidell L: Using multivariate statistics. 5th edition. Boston: Pearson Education Inc; 2007.
• [15]Watmough S, O’Sullivan H, Taylor D: Graduates from a traditional medical curriculum evaluate the effectiveness of their medical curriculum through interviews. BMC Med Educ 2009., 9Available from: http://www.biomedcentral.com/1472-6920/9/64 webcite
• [16]Tallentire VR, Smith SE, Wylde K, Cameron HS: Are medical graduates ready to face the challenges of foundation training? Postgrad Med J 2011, 87:590-595.
• [17]Pella G, Homer M, Robert TS: Assessor training: its effects on criterion‒based assessment in a medical context. Int J Res Method Educ 2008, 31(2):143-154.
• [18]Aram N, Brazil V, Davin L, Greenslade J: Intern underperformance is detected more frequently in emergency medicine rotations. Emerg Med Australas 2013, 25:68-74.