To what extent does the Health Professions Admission Test-Ireland predict performance in early undergraduate tests of communication and clinical skills? – An observational cohort study

BMC Medical Education

Siun O’Flynn⁵  John Newell⁴  Patrick Henn⁵  Fidelma Dunne²  Daniel Regan¹  Maureen E Kelly³ 

[1] School of Psychology, National University of Ireland, Galway (NUI Galway), Galway, Ireland;The Medical School, National University of Ireland, Galway (NUI Galway), Galway, Ireland;Discipline of General Practice, Clinical Science Institute, National University of Ireland, Galway, Ireland;Health Research Board (HRB) Clinical Research Facility, National University of Ireland, Galway (NUI Galway), Galway, Ireland;The Medical School, University College Cork (UCC), Cork, Ireland

关键词: Ability;    Cognitive;    Assessment;    HPAT-Ireland;    Predictive;    Validity;    Student;    Medical;    Selection;   

Others  :  1138916
DOI  :  10.1186/1472-6920-13-68

 received in 2012-12-07, accepted in 2013-04-22,  发布年份 2013

【 摘 要 】

Background

Internationally, tests of general mental ability are used in the selection of medical students. Examples include the Medical College Admission Test, Undergraduate Medicine and Health Sciences Admission Test and the UK Clinical Aptitude Test. The most widely used measure of their efficacy is predictive validity.

A new tool, the Health Professions Admission Test- Ireland (HPAT-Ireland), was introduced in 2009. Traditionally, selection to Irish undergraduate medical schools relied on academic achievement. Since 2009, Irish and EU applicants are selected on a combination of their secondary school academic record (measured predominately by the Leaving Certificate Examination) and HPAT-Ireland score. This is the first study to report on the predictive validity of the HPAT-Ireland for early undergraduate assessments of communication and clinical skills.

Method

Students enrolled at two Irish medical schools in 2009 were followed up for two years. Data collected were gender, HPAT-Ireland total and subsection scores; Leaving Certificate Examination plus HPAT-Ireland combined score, Year 1 Objective Structured Clinical Examination (OSCE) scores (Total score, communication and clinical subtest scores), Year 1 Multiple Choice Questions and Year 2 OSCE and subset scores. We report descriptive statistics, Pearson correlation coefficients and Multiple linear regression models.

Results

Data were available for 312 students. In Year 1 none of the selection criteria were significantly related to student OSCE performance. The Leaving Certificate Examination and Leaving Certificate plus HPAT-Ireland combined scores correlated with MCQ marks.

In Year 2 a series of significant correlations emerged between the HPAT-Ireland and subsections thereof with OSCE Communication Z-scores; OSCE Clinical Z-scores; and Total OSCE Z-scores. However on multiple regression only the relationship between Total OSCE Score and the Total HPAT-Ireland score remained significant; albeit the predictive power was modest.

Conclusion

We found that none of our selection criteria strongly predict clinical and communication skills. The HPAT- Ireland appears to measures ability in domains different to those assessed by the Leaving Certificate Examination. While some significant associations did emerge in Year 2 between HPAT Ireland and total OSCE scores further evaluation is required to establish if this pattern continues during the senior years of the medical course.

【 授权许可】

   
2013 Kelly et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

【 参考文献 】

• [1]Patterson F, Ferguson E: Selection for medical education and training. Edited by Swanwick T. Oxford, UK: Wiley Online Library; 2010.
• [2]Association of American Medical Colleges: Medical College Admission Test. 2012. http://www.aamc.org/mcat webcite (cited 17/11/2012)
• [3]Cambridge Assessment Group: BioMedical Admission Test. 2012. http://www.admissionstests.cambridgeassessment.org.uk/adt/bmat webcite [cited 23/10/2012]
• [4]Australian Council for Educational Research: Undergraduate Medicine and Health Sciences Admission Test. 2012. http://umat.acer.edu.au webcite [cited17/02/2012]
• [5]UKCAT Consortium: The UK Clinical Aptitude Test. 2012. http://www.ukcat.ac.uk webcite (cited 17/10/2012)
• [6]Prideaux D: Assessment for selection for the health care professions and specialty training: Consensus statement and recommendations from the Ottawa 2010.. 2011. Conference Medical Teacher
• [7]Donnon T, Paolucci EO, Violato C: The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med 2007, 82(1):100-106.
• [8]Julian ER: Validity of the medical college admission test for predicting medical school performance. Acad Med 2005, 80(10):910-917.
• [9]Lynch B, Mackenzie R, Dowell J, Cleland J, Prescott G: Does the UKCAT predict Year 1 performance in medical school? Med Educ 2009, 43(12):1203-1209.
• [10]Yates J, James D: The value of the UK clinical aptitude test in predicting pre-clinical performance: a prospective cohort study at Nottingham medical school. BMC Med Educ 2010, 10:55. BioMed Central Full Text
• [11]Yates J, James D: The UK clinical aptitude test and clinical course performance at Nottingham: a prospective cohort study. BMC Med Educ 2013, 13:32. BioMed Central Full Text
• [12]Wright SR, Bradley PM: Has the UK clinical aptitude test improved medical student selection? Med Educ 2010, 44(11):1069-1076.
• [13]Poole P, Shulruf B, Rudland J, Wilkinson T: Comparison of UMAT scores and GPA in prediction of performance in medical school: a national study. Med Educ 2012, 46(2):163-171.
• [14]Wilkinson DZJ, Parker M: Predictive validity of the undergraduate medicine and health sciences admission test for medical students’ academic performance. Med J Aust 2011, 194(7):341-344.
• [15]Emery JLBJ: The predictive validity of the BioMedical admissions test for pre-clinical examination performance. Med Educ 2009, 43(6):557-564.
• [16]McManus ICFE, Wakeford R, Powis D, James D: Predictive validity of the Biomedical admissions test: an evaluation and case study. Med Teach 2011, 33(1):53-57.
• [17]Ferguson E, James D, Madeley L: Factors associated with success in medical school: systematic review of the literature. BMJ 2002, 324(7343):952-957.
• [18]HPAT-Ireland: Health Professions Admission Test - Ireland. 2012. http://www.hpat-ireland.acer.edu.au webcite (cited 17/11/2012)
• [19]Fottrell P: Medical education in Ireland: A new direction. Report of the Working Group on Undergraduate Education and Training (The Fottrell Report). 2006. (cited 30/04/2013) Available from: http://www.dohc.ie/publications/fottrell.html?lang=en webcite
• [20]O Flynn S, Mills A, Fitzgerald T: National Research Group Evaluating Revised Entry Mechanisms to Medicine Interim Report School Leaver Entrants. 2012. (cited 7/03/2013) available from: http://www.ucd.ie/t4cms/HPAT-report-July2012.pdf webcite
• [21]Australian Council for Educational Research. 2012. (cited 26/10/2012]; Available from: http://www.acer.edu.au/ webcite
• [22]Irish Medical Council: Eight Domains of Good Professional Practice. 2010. (cited 20/11/2012); Available from: http://www.medicalcouncil.ie/Education-and-Training/Good-Professional-Practice/Eight-Domains-of-Good-Professional-Practice-as-devised-by-Medical-Council.pdf webcite
• [23]Frank JR: The CanMEDS 2005 physician competency framework: Better standards, better physicians, better care. Ottawa, Ontario, Canada: Royal College of Physicians and Surgeons of Canada; 2005.
• [24]Irish Medical Council: Number and Type of Complaints Received by Preliminary Proceedings Committee in 2010. https://medicalcouncil.ie/News-and-Publications/Publications/Annual-Reports-Statistics-/Statistics/Statistics-2010/Statistics-2010.html webcite (cited 18/02/2012)
• [25]Papadakis MA, Teherani A, Banach MA, Knettler TR, Rattner SL, Stern DT: Disciplinary Action by Medical Boards and Prior Behavior in Medical School. N Engl J Med 2005, 353(25):2673-2682.
• [26]Siu E, Reiter HI: Overview: What’s worked and what hasn’t as a guide towards predictive Admissions tool development. Adv Health Sci Educ 2009, 14(5):759-775.
• [27]Central Applications Office: Selection criteria for Undergraduate Entry to Medicine For EU Applicants 2012. 2012. [cited 02/06/2012]; Available from: http://www.tcd.ie/Admissions/undergraduate/assets/pdf/HPAT_Brochure_2012_lr.pdf webcite
• [28]Harden R, Stevenson M, Downie WW, Wilson G: Assessment of clinical competence using objective structured examination. Br Med J 1975, 1(5955):447-451.
• [29]Hamdy H, Prasad K: BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice. Med Teacher 2006, 28(2):103-116.
• [30]Coates H: Establishing the criterion validity of the Graduate Medical School Admissions Test (GAMSAT). Med Educ 2008, 42:999-1006.
• [31]McManus C, Powis D: Testing medical school selection tests. Med J Australia 2007, 186(3):118-119.
• [32]McManus IC, Smithers E, Partridge P, Keeling A, Fleming PR: A levels and Intelligence as predictors of medical careers in UK Doctors: 20 Year Prospective Study. BMJ: Br Med J 2003, 327(7407):139-142.
• [33]Sackett PR, Yang H: Correction for range restriction: an expanded typology. J Appl Psychol 2000, 85(1):112.
• [34]Halpenny DCK, Halpenny M, Burke J, Torreggiani WC: The Health Professions Admission Test (HPAT) score and leaving certificate results can independently predict academic performance in medical school: do we need both tests? Ir Med J 2010, 103(10):300-302.
• [35]Quinn A, Corrigan MA, Broderick J, McEntee G, Hill AD: A comparison of performances of consultant surgeons, NCHDs and medical students in a modified HPAT examination. Ir Med J 2010, 103(6):172-173. Epub 2010/07/31
• [36]O’Flynn S, Mills A, Fitzgerald T: Entry to medical school-the gender question. What has happened? Ir Med J 2013. in press ref:S5443
• [37]James D, Yates J, Nicholson S: Comparison of A level and UKCAT performance in students applying to UK medical and dental schools in 2006: cohort study. BMJ 2010, 340:c478. Epub 2010/02/18
• [38]Walshe J: Scrap unfair aptitude testing for entry into medicine say doctors. Irish Independent 2009. Ed Tuesday 25-08-09 page 5
• [39]Murray N: More students with lower leaving cert points get medicine. Irish Examiner 2011. Ed. Tuesday, August 23, 2011 (cited 30/04/2013) Available from: http://www.irishexaminer.com/ireland/education/more-students-with-lower-leaving-cert-points-get-medicine-165133 webcite
• [40]Koczwara A, Patterson F, Zibarras L, Kerrin M, Irish B, Wilkinson M: Evaluating cognitive ability, knowledge tests and situational judgement tests for postgraduate selection. Med Educ 2012, 46(4):399-408.