Predictive validity of the UK clinical aptitude test in the final years of medical school: a prospective cohort study

BMC Medical Education

Rhoda MacKenzie²  Jennifer Cleland²  Jonathan Dowell¹  Alistair Mathieson²  Adrian Husbands¹ 

[1] School of Medicine, University of Dundee, The Mackenzie Building, Kirsty Semple Way, Dundee DD2 4BF, UK;Division of Medical and Dental Education, University of Aberdeen, Polwarth Building, Foresterhill, Aberdeen AB25 2ZD, UK

关键词: Aptitude;    Admissions;    Selection;    Assessment;    Psychometric;    Predictive validity;    UKCAT;   

Others  :  866720
DOI  :  10.1186/1472-6920-14-88

 received in 2013-07-31, accepted in 2014-04-11,  发布年份 2014

【 摘 要 】

Background

The UK Clinical Aptitude Test (UKCAT) was designed to address issues identified with traditional methods of selection. This study aims to examine the predictive validity of the UKCAT and compare this to traditional selection methods in the senior years of medical school. This was a follow-up study of two cohorts of students from two medical schools who had previously taken part in a study examining the predictive validity of the UKCAT in first year.

Methods

The sample consisted of 4^th and 5^th Year students who commenced their studies at the University of Aberdeen or University of Dundee medical schools in 2007. Data collected were: demographics (gender and age group), UKCAT scores; Universities and Colleges Admissions Service (UCAS) form scores; admission interview scores; Year 4 and 5 degree examination scores. Pearson’s correlations were used to examine the relationships between admissions variables, examination scores, gender and age group, and to select variables for multiple linear regression analysis to predict examination scores.

Results

Ninety-nine and 89 students at Aberdeen medical school from Years 4 and 5 respectively, and 51 Year 4 students in Dundee, were included in the analysis. Neither UCAS form nor interview scores were statistically significant predictors of examination performance. Conversely, the UKCAT yielded statistically significant validity coefficients between .24 and .36 in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years.

Conclusions

Results suggest the UKCAT appears to predict performance better in the later years of medical school compared to earlier years and provides modest supportive evidence for the UKCAT’s role in student selection within these institutions. Further research is needed to assess the predictive validity of the UKCAT against professional and behavioural outcomes as the cohort commences working life.

【 授权许可】

   
2014 Husbands et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]McManus I, Powis D, Wakeford R, Ferguson E, James D, Richards P: Intellectual aptitude tests and A levels for selecting UK school leaver entrants for medical school. Br Med J 2005, 331(7516):555-559.
• [2]Powis D, James D, Ferguson E: Demographic and socio-economic associations with academic attainment (UCAS tariff scores) in applicants to medical school. Med Educ 2007, 41(3):242-249.
• [3]Siu E, Reiter H: Overview: what’s worked and what hasn’t as a guide towards predictive admissions tool development. Adv Health Sci Educ 2009, 14(5):759-775.
• [4]Ferguson E, James D, Madeley L: Factors associated with success in medical school: systematic review of the literature. Br Med J 2002, 324(7343):952-957.
• [5]Tiffin P, Dowell J, McLachlan J: Widening access to UK medical education for under-represented socioeconomic groups: modelling the impact of the UKCAT in the 2009 cohort. Br Med J 2012, 344:e1805.
• [6]Ferguson E, Sanders A: Predictive validity of personal statements and the role of the five-factor model of personality in relation to medical training. J Occup Organ Psychol 2000, 73:321-344.
• [7]Wright S, Bradley P: Has the UK clinical aptitude test improved medical student selection? Med Educ 2010, 44(11):1069-1076.
• [8]Husbands A, Dowell J: Predictive validity of the Dundee multiple mini-interview. Med Educ 2013, 47:717-725.
• [9]Cleland J, Dowell J, McLachlan J, Nicholson S, Patterson F: Identifying best practice in the selection of medical students. London, United Kingdom: General Medical Council; 2012.
• [10]UKCAT: About the test: what is in the test?. Cambridge, UK: United Kingdom Clinical Aptitude Test; [http://www.ukcat.ac.uk/about-the-test/test-format/ webcite], Accessed March 2013.
• [11]Lynch B, MacKenzie R, Dowell J, Cleland J, Prescott G: Does the UKCAT predict Year 1 performance in medical school? Med Educ 2009, 43(12):1203-1209.
• [12]Donnon T, Paolucci E, Violato C: The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med 2007, 82(1):100-106.
• [13]Coates H: Establishing the criterion validity of the Graduate Medical School Admissions Test (GAMSAT). Med Educ 2008, 42(10):999-1006.
• [14]Callahan C, Hojat M, Veloski J, Erdmann J, Gonnella J: The predictive validity of three versions of the MCAT in relation to performance in medical school, residency, and licensing examinations: a longitudinal study of 36 classes of Jefferson Medical College. Acad Med 2010, 85(6):980-987.
• [15]Yates J, James D: The value of the UK clinical aptitude test in predicting pre-clinical performance: a prospective cohort study at Nottingham Medical School. Bmc Med Educ 2010, 10:15. BioMed Central Full Text
• [16]McManus IC, Dewberry C, Nicholson S, Dowell JS: The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools. Bmc Med 2013, 11:244. BioMed Central Full Text
• [17]Yates J, James D: The UK clinical aptitude test and clinical course performance at Nottingham: a prospective cohort study. Bmc Med Educ 2013, 13:32. BioMed Central Full Text
• [18]Parry J, Mathers J, Stevens A, Parsons A, Lilford R, Spurgeon P, Thomas H: Admissions processes for five year medical courses at English schools: review. Br Med J 2006, 332(7548):1005-1008.
• [19]Eva K, Reiter H, Trinh K, Wasi P, Rosenfeld J, Norman G: Predictive validity of the multiple mini-interview for selecting medical trainees. Med Educ 2009, 43(8):767-775.
• [20]Nunnally JC: Psychometric theory. 2nd edition. New Delhi: Tate McGraw-Hill; 1981.
• [21]Calvert MJ, Ross NM, Freemantle N, Xu Y, Zvauya R, Parle JV: Examination performance of graduate entry medical students compared with mainstream students. J R Soc Med 2009, 102(10):425-430.
• [22]UKCAT: UKCAT: 2008 Annual report. Cambridge, UK: United Kingdom Clinical Aptitude Test; 2009. http://www.ukcat.ac.uk/App_Media/uploads/pdf/Annual%20Report%202008.pdf webcite
• [23]O’Keefe D: Colloquy: Should familywise alpha be adjusted? Against familywise alpha adjustment. Hum Commun Res 2003, 29(3):431-447.
• [24]O’Keefe D: It takes a family - a well defined family - to underwrite family wise corrections. Commun Meth Measures 2007, 1(4):267-273.
• [25]Field AP: Discovering statistics using SPSS: (and sex and drugs and rock ‘n’ roll). 3rd edition. London: SAGE; 2009.
• [26]COHEN J: A power primer. Psychol Bull 1992, 112(1):155-159.
• [27]Department of Labor EaTAU: Testing and assessment: an employer’s guide to good practices. Washington, DC: Department of Labor, Employment and Training Administration (US); 1999.
• [28]Eva K, Reiter H, Rosenfeld J, Norman G: The ability of the multiple mini-interview to predict preclerkship performance in medical school. Acad Med 2004, 79(10):S40-S42.
• [29]Reiter H, Eva K, Rosenfeld J, Norman G: Multiple mini-interviews predict clerkship and licensing examination performance. Med Educ 2007, 41(4):378-384.