The fairness, predictive validity and acceptability of multiple mini interview in an internationally diverse student population- a mixed methods study

BMC Medical Education

Andrew W Murphy²  Fidelma P Dunne²  Thomas Kropmans²  Siun O‘Flynn⁴  John Newell³  Adrian Husbands¹  Jon Dowell¹  Maureen E Kelly² 

[1]School of Medicine, University of Dundee, Dundee, Scotland

[2]The Medical School, National University of Ireland Galway, (NUI Galway), Galway, Ireland

[3]Health Research Board (HRB) Clinical Research Facility, National University of Ireland, Galway (NUI Galway), Galway, Ireland

[4]The Medical School, University College Cork (UCC), Cork, Ireland

关键词: Stakeholder views;    Recommendations;    Language;    Culture;    Mixed methods;    International students;    Selection;    Multiple mini interview;   

Others  :  1089945
DOI  :  10.1186/s12909-014-0267-0

 received in 2014-06-20, accepted in 2014-12-09,  发布年份 2014

【 摘 要 】

Background

International medical students, those attending medical school outside of their country of citizenship, account for a growing proportion of medical undergraduates worldwide. This study aimed to establish the fairness, predictive validity and acceptability of Multiple Mini Interview (MMI) in an internationally diverse student population.

Methods

This was an explanatory sequential, mixed methods study. All students in First Year Medicine, National University of Ireland Galway 2012 were eligible to sit a previously validated 10 station MMI. Quantitative data comprised: demographics, selection tool scores and First Year Assessment scores. Qualitative data comprised separate focus groups with MMI Assessors, EU and Non-EU students.

Results

109 students participated (45% of class). Of this 41.3% (n = 45) were Non-EU and 35.8% (n = 39) did not have English as first language. Age, gender and socioeconomic class did not impact on MMI scores. Non-EU students and those for whom English was not a first language achieved significantly lower scores on MMI than their EU and English speaking counterparts (difference in mean 11.9% and 12.2% respectively, P<0.001). MMI score was associated with English language proficiency (IELTS) (r = 0.5, P<0.01). Correlations emerged between First Year results and IELTS (r = 0.44; p = 0.006; n = 38) and EU school exit exam (r = 0.52; p<0.001; n = 56). MMI predicted EU student OSCE performance (r = 0.27; p = 0.03; n = 64). In the analysis of focus group data two overarching themes emerged: Authenticity and Cultural Awareness. MMI was considered a highly authentic assessment that offered a deeper understanding of the applicant than traditional tools, with an immediate relevance to clinical practice. Cultural specificity of some stations and English language proficiency were seen to disadvantage international students. Recommendations included cultural awareness training for MMI assessors, designing and piloting culturally neutral stations, lengthening station duration and providing high quality advance information to candidates.

Conclusion

MMI is a welcome addition to assessment armamentarium for selection, particularly with regard to stakeholder acceptability. Understanding the mediating and moderating influences for differences in performance of international candidates is essential to ensure that MMI complies with the metrics of good assessment practice and principles of both distributive and procedural justice for all applicants, irrespective of nationality and cultural background.

【 授权许可】

   
2014 Kelly et al.; licensee BioMed Central.

【 参考文献 】

• [1]Hallock JA, McKinley DW, Boulet JR: Migration of doctors for undergraduate medical education. Med Teach 2007, 29(2–3):98-105.
• [2]Medical Schools Council: Number of places available. 2014. available from http://www.medschools.ac.uk/Students/howtoapply/international/Pages/Limitedplaces.aspx (accessed June 16th 2014).
• [3]Medical Deans Australia and New Zealand: Student Statistics Annual Tables. 2013 Available from http://www.medicaldeans.org.au/statistics/annualtables (accessed June 16th 2014).
• [4]Boulet JR, Cooper RA, Seeling SS, Norcini JJ, McKinley DW: US citizens who obtain their medical degrees abroad: an overview 1992–2006. Health Aff 2009, 28(1):226-233.
• [5]OECD: How Many Students Study Abroad and Where Do They Go? In. Education at a Glance 2013: Highlights, OECD Publishing 2013. Available from http://dx.doi.org/10.1787/eag_highlights-2013-12-en (accessed June 16th 2014).
• [6]Cleland J, Dowell J, McLachlan J, Nicholson S, Patterson F: Identifying Best Practice in the Selection of Medical Students. UK. General Medical Council. 2013. Available from http://www.gmc-uk.org/about/research/25036.asp (accessed June 16th 2014).
• [7]Poole P, Shulruf B, Harley B, Monigatti J, Barrow M, Reid P, Bagg W: Shedding light on the decision to retain an interview for medical student selection. NZ Med J 2012, 125(1361):81-88.
• [8]Stevens L, Kelly ME, Hennessy M, Last J, Dunne F, O’Flynn S: Medical Students' views on selection tools for medical school - a mixed methods study. Ir Med J 2014, 107(8):229-231.
• [9]Eva KW, Rosenfeld J, Reiter HI, Norman GR: An admissions OSCE: the multiple mini‐interview. Med Educ 2004, 38(3):314-326.
• [10]Razack S, Faremo S, Drolet F, Snell L, Wiseman J, Pickering J: Multiple mini‐interviews versus traditional interviews: stakeholder acceptability comparison. Med Educ 2009, 43(10):993-1000.
• [11]Pau A, Jeevaratnam K, Chen YS, Fall AA, Khoo C, Nadarajah VD: The Multiple Mini Interview (MMI) for student selection in health professionals training a systematic review. Med Teach 2013, 35(12):1027-1041.
• [12]McManus IC, Wakeford R: PLAB and UK graduate performance on MRCP (UK) and MRCGP examinations: data linkage study. Br Med J 2014, 348:g2621. Published 17 April 2014
• [13]Tiffin PA, Illing J, Kasim AS, McLachlan JC: Annual Review of Competence Progression (ARCP) performance of doctors who passed Professional and Linguistic Assessments Board (PLAB) tests compared with UK medical graduates: national data linkage study. Br Med J 2014, 348:g2622. Published 17 April 2014
• [14]Piele E: Selecting an internationally diverse medical workforce. Br Med J 2014, 348:g2696. Published 17 April 2014
• [15]Humphrey S, Dowson S, Wall D, Diwakar V, Goodyear HM: Multiple mini‐interviews: opinions of candidates and interviewers. Med Educ 2008, 42(2):207-213.
• [16]Dore KL, Kreuger S, Ladhani M, Rolfson D, Kurtz D, Kulasegaram K, Cullimore A, Norman G, Eva K, Reiter HI: The reliability and acceptability of the multiple mini-interview as a selection instrument for postgraduate admissions. Acad Med 2010, 85(10):S60-S63.
• [17]Patterson F, Zibarras L, Carr V, Irish B, Gregory S: Evaluating candidate reactions to selection practices using organisational justice theory. Med Educ 2011, 45(3):289-297.
• [18]Gilliland SW: Effects of procedural and distributive justice on reactions to a selection system. J Appl Psychol 1994, 79(5):691-701.
• [19]Prideaux D, Roberts C, Eva K, Centeno A, McCrorie P, McManus C, Patterson F, Powis D, Tekian A, Wilkinson D: Assessment for selection for the health care professions and specialty training: consensus statement and recommendations from the Ottawa 2010 conference. Med Teach 2011, 33(3):215-223.
• [20]Kelly ME, Dowell J, Husbands A, Kropmans T, Jackson A, Dunne F, O’ Flynn S, Newell J, Murphy AW: Can Multiple Mini Interviews work in an Irish context? A feasibility study. Ir Med J 2014, 107(7):210-212.
• [21]Milburn A: “Unleashing Aspiration: The Final Report of the Panel on Fair Access to the Professions.”. Cabinet Office, London UK; 2009.
• [22]Freeman RE: Strategic Management: A Stakeholder Approach. Cambridge University Press, New York; 2010.
• [23]Australian Council for Educational Research (ACER): HPAT-IRELAND- Health Professions Admission Test - Ireland. 2014. Available from: http://www.hpat-ireland.acer.edu.au/ (accessed June 16th 2014).
• [24]Central Applications Office (CAO): Selection criteria for Undergraduate Entry to Medicine EU applicants. 2014 available from http://www2.cao.ie/downloads/documents/UGMedEntry2015.pdf (accessed June 16th 2014).
• [25]Central Applications Office (CAO): Guideline entry requirements for EU/EFTA Applicants (other than Irish Leaving Certificate) 2014 available from http://www2.cao.ie/downloads/documents/Guidelines-EU-EFTA.pdf (accessed June 16th 2014).
• [26]International English Language Testing System (IETLS): What is IELTS? 2014. Available from http://www.ielts.org/test_takers_information/what_is_ielts.aspx. (accessed June 16th 2014).
• [27]Tashakkori A, Teddlie C: Mixed Methods in Social and Behavioural Research. 2nd edition. Sage Publications, Inc, California; 2010.
• [28]Schifferdecker KE, Reed VA: Using mixed methods research in medical education: basic guidelines for researchers. Med Educ 2009, 43(7):637-644.
• [29]Maudsley G: Mixing it but not mixed-up: mixed methods research in medical education (a critical narrative review). Med Teach 2011, 33(2):e92-e104.
• [30]Creswell JW, Plano Clark VL: Designing and Conducting Mixed Methods Research. 2nd edition. Sage Publications Inc., California; 2011.
• [31]Medical Council: Eight Domains of Good Professional Practice as devised by Medical Council. 2011. Available from https://www.medicalcouncil.ie/News-and-Publications/Publications/Information-for-Doctors/Professional-Competence,-Guidelines-for-Doctors.pdf (accessed June 16th 2014).
• [32]P.A.S.W. Statistics 18, Release version 18.0.0. SPSS 2010 Chicago www.ibm.com/SPSS.
• [33]Cohen J: A power primer. Psychol Bull 1992, 112:155-159.
• [34]Mack N, Woodsong C, Mac Queen K, Guest G, Namey E: Qualitative research methods: a data Collector’s field guide. Fam Health Int 2005, Module 4:51-83. available from http://www.fhi360.org/resource/qualitative-research-methods-data-collectors-field-guide (accessed Mar 27th 2014)
• [35]Patton MQ: Qualitative Research and Evaluation Methods. 3rd edition. Sage publications, Thousand Oaks; 2002.
• [36]Pope, Mays: Qualitative Research in Health Care. 3rd edition. British Medical Journal BMJ Books Blackwell Publishing, Oxford UK; 2006.
• [37]QSR International: NVivo Qualitative Data Analysis Software. QSR International Pty Ltd, ᅟ; 2012.
• [38]Corbin J, Strauss A: Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Sage Publications, Thousand Oaks, CA; 2008.
• [39]Tufford L, Newman P: Bracketing in qualitative research. Qual Soc Work 2012, 11(1):80-96.
• [40]Census: This is Ireland (Part 2): 2011 Available from http://www.cso.ie/en/census/census2011reports/census2011thisisirelandpart2/ (accessed June 16/ 2014).
• [41]Loh Ee Chin: Clinical Consultant and Academic.Personal Communication. Dec 2014.
• [42]British Medical Association: The Demography of Medical Schools: A Discussion Paper. British Medical Association, London; 2004.
• [43]Higher Education Authority: National plan for equity of access to higher education, Higher Education Authority 2008–2013 National Office of Equity to Higher Education, Dublin July 2008. Available from http://www.hea.ie/files/files/file/New_pdf/National_Access_Plan_2008-2013_(English).pdf (accessed June 16 2014).
• [44]Husbands A, Dowell J: Predictive validity of the Dundee multiple mini‐interview. Med Educ 2013, 47(7):717-725.
• [45]Boulet JR, Swanson DB, Cooper RA, Norcini JJ, McKinley DW: A comparison of the characteristics and examination performances of US and non-US citizen international medical graduates who sought educational commission for foreign medical graduates certification: 1995–2004. Acad Med 2006, 81(10):S116-S119.
• [46]Esmail A, Roberts C: Academic performance of ethnic minority candidates and discrimination in the MRCGP examinations between 2010 and 2012: analysis of data. Br Med J 2013, 347:f5662.
• [47]Stafford JR, Bowman R, Ewing T, Hanna J, Lopez-De Fede A: Building Culture Bridges. National Educational Service, Bloomington, IN; 1997.
• [48]McManus IC, Elder AT, Dacre J: Investigating possible ethnicity and sex bias in clinical examiners: an analysis of data from the MRCP (UK) PACES and nPACES examinations. BMC Med Educ 2013, 13(1):103. BioMed Central Full Text
• [49]McManus IC, Woolf K, Dacre J: The educational background and qualifications of UK medical students from ethnic minorities. BMC Med Educ 2008, 8(1):21. BioMed Central Full Text
• [50]Says FE, Ayuob N, Fahmy AR, Fayez FE, Hasanian M, El Deek B: Experience of establishment of multiple mini structure interview as part of student admission policy at faculty of medicine, King Abdulaziz University, 2011–2012. Med Teach 2013, 35(s1):S74-S77.
• [51]WFME World Federation for Medical Education Basic Medical Education WFME Global Standards for Quality Improvement The 2012 Revision. 2012. available from http://www.wfme.org/standards/bme (accessed June 16/ 2014).