Past-behavioural versus situational questions in a postgraduate admissions multiple mini-interview: a reliability and acceptability comparison

BMC Medical Education

Gominda Ponnamperuma²  Yasuyuki Suzuki¹  Takuya Saiki¹  Junji Machi⁴  Shigeki Fujitani³  Hidetaka Kitazono³  Hiroshi Yoshimura⁴ 

[1]Medical Education Development Centre, Faculty of Medicine, Gifu University, Gifu City, Gifu Prefecture, Japan

[2]Faculty of Medicine, University of Colombo, Colombo, Western Province, Sri Lanka

[3]Educational Committee, Tokyo Bay Urayasu-Ichikawa Medical Centre, Urayasu City, Chiba Prefecture, Japan

[4]Department of Surgery, University of Hawaii, John A. Burns School of Medicine, Honolulu, State of Hawaii, USA

关键词: Multivariate generalisability analysis;    Acceptability;    Reliability;    Situational questions;    Past-behavioural questions;    Station interview format;    Multiple mini-interview;    Postgraduate training;    Selection;   

Others  :  1206354
DOI  :  10.1186/s12909-015-0361-y

 received in 2014-09-21, accepted in 2015-03-30,  发布年份 2015

【 摘 要 】

Background

The Multiple Mini-Interview (MMI) mostly uses ‘Situational’ Questions (SQs) as an interview format within a station, rather than ‘Past-Behavioural’ Questions (PBQs), which are most frequently adopted in traditional single-station personal interviews (SSPIs) for non-medical and medical selection. This study investigated reliability and acceptability of the postgraduate admissions MMI with PBQ and SQ interview formats within MMI stations.

Methods

Twenty-six Japanese medical graduates, first completed the two-year national obligatory initial postgraduate clinical training programme and then applied to three specialty training programmes - internal medicine, general surgery, and emergency medicine - in a Japanese teaching hospital, where they underwent the Accreditation Council for Graduate Medical Education (ACGME)-competency-based MMI. This MMI contained five stations, with two examiners per station. In each station, a PBQ, and then an SQ were asked consecutively. PBQ and SQ interview formats were not separated into two different stations, or the order of questioning of PBQs and SQs in individual stations was not changed due to lack of space and experienced examiners. Reliability was analysed for the scores of these two MMI question types. Candidates and examiners were surveyed on this experience.

Results

The PBQ and SQ formats had generalisability coefficients of 0.822 and 0.821, respectively. With one examiner per station, seven stations could produce a reliability of more than 0.80 in both PBQ and SQ formats. More than 60% of both candidates and examiners felt positive about the overall candidates’ ability. All participants liked the fairness of this MMI when compared with the previously experienced SSPI. SQs were perceived more favourable by candidates; in contrast, PBQs were perceived more relevant by examiners.

Conclusions

Both PBQs and SQs are equally reliable and acceptable as station interview formats in the postgraduate admissions MMI. However, the use of the two formats within the same station, and with a fixed order, is not the best to maximise its utility as an admission test. Future studies are required to evaluate how best the SQs and PBQs should be combined as station interview formats to enhance reliability, feasibility, acceptability and predictive validity of the MMI.

【 授权许可】

   
2015 Yoshimura et al.; licensee BioMed Central.

【 参考文献 】

• [1]Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: the multiple mini-interview. Med Educ. 2004; 38(3):314-26.
• [2]Lemay JF, Lockyer JM, Collin VT, Brownell AK. Assessment of non-cognitive traits through the admissions multiple mini-interview. Med Educ. 2007; 41(6):573-9.
• [3]Roberts C, Walton M, Rothnie I, Crossley J, Lyon P, Kumar K. Factors affecting the utility of the multiple mini-interview in selecting candidates for graduate-entry medical school. Med Educ. 2008; 42(4):396-404.
• [4]Uijtdehaage S, Doyle L, Parker N. Enhancing the reliability of the multiple mini-interview for selecting prospective health care leaders. Acad Med. 2011; 86(8):1032-9.
• [5]Goodyear HM, Jyothish D, Diwakar V, Wall D. Reliability of a regional junior doctor recruitment process. Med Teach. 2007; 29(5):501-3.
• [6]Hofmeister M, Lockyer J, Crutcher R. The multiple mini-interview for selection of international medical graduates into family medicine residency education. Med Educ. 2009; 43(6):573-9.
• [7]Dore KL, Kreuger S, Ladhani M, Rolfson D, Kurtz D, Kulasegaram K et al.. The reliability and acceptability of the multiple mini-interview as a selection instrument for postgraduate admissions. Acad Med. 2010; 85(10):60-3.
• [8]Fraga JD, Oluwasanjo A, Wasser T, Donato A, Alweis R: Reliability and acceptability of a five-station multiple mini-interview model for residency program recruitment. J Community Hosp Intern Med Perspect 2013, 3(3-4): 10.3402/jchimp. v3i3-4.21362. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3879511/pdf/JCHIMP-3-21362.pdf [Accessed 9 Sept. 2014].
• [9]Campagna-Vaillancourt M, Manoukian J, Razack S, Nguyen LHP. Acceptability and reliability of multiple mini interviews for admission to otolaryngology residency. Laryngoscope. 2014; 124(1):91-6.
• [10]Ahmed A, Qayed KI, Abdulrahman M, Tavares W, Rosenfeld J. The multiple mini-interview for selecting medical residents: first experience in the Middle East region. Med Teach. 2014; 36(8):703-9.
• [11]Brownell K, Lockyer J, Collin T, Lemay J. Introduction of the multiple mini interview into the admissions process at the University of Calgary: acceptability and feasibility. Med Teach. 2007; 29(4):394-6.
• [12]Dowell J, Lynch B, Till H, Kumwenda B, Husbands A. The multiple mini-interview in the U.K. context: 3 years of experience at Dundee. Med Teach. 2012; 34(4):297-304.
• [13]Hofmeister M, Lockyer J, Crutcher R. The acceptability of the multiple mini interview for resident selection. Fam Med. 2008; 40(10):734-40.
• [14]Humphrey S, Dowson S, Wall D, Diwakar V, Goodyear HM. Multiple mini-interviews: opinions of candidates and interviewers. Med Educ. 2008; 42(2):207-13.
• [15]Hopson LR, Burkhardt JC, Stansfield RB, Vohra T, Turner-Lawrence D, Losman ED. The multiple mini-interview for emergency medicine resident selection. J Emerg Med. 2014; 46(4):537-43.
• [16]Eva KW. On the generality of specificity. Med Educ. 2003; 37(7):587-8.
• [17]Kriter CD, Yin P, Solow C, Brennan RL. Investigating the reliability of the medical school admissions interview. Adv Health Sci Educ Theory Pract. 2004; 9(2):147-59.
• [18]Prideaux D, Roberts C, Eva K, Centeno A, Mccrorie P, Mcmanus C et al.. Assessment for selection for the health care professions and specialty training: consensus statement and recommendations from the Ottawa 2010 conference. Med Teach. 2011; 33(3):215-23.
• [19]Pau A, Jeevaratnam K, Chen YS, Fall AA, Khoo C, Nadarajah VD. The Multiple mini-interview (MMI) for student selection in health professions training–a systematic review. Med Teach. 2013; 35(12):1027-41.
• [20]Cleland J, Dowell J, McLachlan J, Nicholson S, Patterson F: Research Report. Identifying best practice in the selection of medical students (literature review and interview survey). 2012. http://www.sgptg.org/app/download/7964849/Identifying_best_practice_in_the_selection_of_medical_students.pdf_51119804.pdf [Accessed 8 Sept. 2014].
• [21]Latham GP, Saari LM, Pursell ED, Campion MA. The situational interview. J Appl Psychol. 1980; 65(4):422-7.
• [22]Campion MA, Palmer DK, Campion JE. A review of structure in the selection interview. Pers Psychol. 1997; 50(3):655-702.
• [23]Guidelines and ethical considerations for assessment center operations. Int J Sel Ass. 2009; 17(3):243-53.
• [24]Patterson F, Ferguson E. Testing non-cognitive attributes in selection centres: how to avoid being reliably wrong. Med Educ. 2012; 46(3):240-2.
• [25]Patterson F, Ferguson E, Knight AL. Selection into medical education and training. In: Understanding Medical Education. Evidence, Theory, and Practice 2nd edition. Swanwick T, editor. Wiley-Brackwell, Chichester, UK; 2014: p.403-20.
• [26]Finlayson HC, Townson AF. Resident selection for a physical medicine and rehabilitation program: feasibility and reliability of the multiple mini-interview. Am J Phys Med Rehabil. 2011; 90(4):330-5.
• [27]Levashina J, Hartwell CJ, Morgeson FP, Campion MA. The structured employment interview: narrative and quantitative review of the research literature. Pers Psychol. 2014; 67(1):241-93.
• [28]Janz T. Behavior description interviewing: new, accurate, cost-effective. Allynand Bacon, Inc, Boston, MA; 1986.
• [29]Kopriva P: The residency interview: making the most of it. American Medical Association. http://www.ama-assn.org/ama/pub/about-ama/our-people/member-groups-sections/minority-affairs-section/transitioning-residency/the-residency-interview-making-most-it.page. 2014. [Accessed 21 Sept. 2014].
• [30]Educational Commission for Foreign Medical Graduates. Ask the experts: mastering the residency interview. http://www.ecfmg.org/echo/mastering-residency-interview.html 2012. [Accessed 21 Sept. 2014].
• [31]Easdown JL, Castro PL, Shinkle EP, Small L, Algren J: The behavioral interview, a method to evaluate ACGME competencies in resident selection: a pilot project. Journal of Education in Perioperative Medicine. 2005;7(1):1–10.
• [32]Thaxton RE, Kacpowicz RJ, Rayfield J. “Are they who they say they are?” New behavioral-based interview style [abstract]. Acad Emerg Med. 2010; 17:S5-11.
• [33]Strand EA, Moore E, Laube DW. Can a structured, behavior-based interview predict future resident success? Am J Obstet Gynecol. 2011; 204(446):e1-13.
• [34]Prager JD, Myer IV CM, Hayes K, Myer III CM, Pensak ML: Improving methods of resident selection. 2010. http://www.triomeetingposters.org/wp-content/uploads/2010/04/A020.pdf [Accessed 9 Sept. 2014].
• [35]Lee WT, Esclamado RM, Puscas L. Selecting among otolaryngology residency applicants to train as tomorrow’s leaders. JAMA Otolaryngology–Head & Neck Surgery. 2013; 139(8):770-1.
• [36]Accreditation Council for Graduate Medical Education: 2013 common program requirements. 2013. https://www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramRequirements/CPRs2013.pdf [Accessed 10 Sept. 2014].
• [37]Kozu T. Medical education in Japan. Acad Med. 2006; 81(12):1069-75.
• [38]Bangerter A, Corvalan P, Cavin C. Storytelling in the selection interview? How applicants respond to past behaviour questions. J Bus Psychol. published online;16 March 2014. doi:10.1007/s10869-014-9350-0.
• [39]Poole A, Catano VM, Cunningham DP. Predicting performance in Canadian dental schools: the new Canadian Dental Association (CDA) structured interview, a new personality assessment, and the Canadian Dental Aptitude Test (DAT). J Dent Educ. 2007; 71(5):664-76.
• [40]Eva KW, Macala C. Multiple mini-interview test characteristics: ‘tis better to ask candidates to recall than to imagine. Med Educ. 2014; 48(6):604-13.
• [41]Roberts C, Clark T, Burgess A, Frommer M, Grant M, Mossman K. The validity of a behavioural multiple-mini-interview within an assessment centre for selection into specialty training. BMC Med Educ. 2014; 14:115. BioMed Central Full Text