Is a picture worth a thousand words: an analysis of the difficulty and discrimination parameters of illustrated vs. text-alone vignettes in histology multiple choice questions

BMC Medical Education

Richard Arnett¹  Robin O’Sullivan³  Jane Holland² 

[1] Quality Enhancement Office, RCSI, 123 St Stephens Green, Dublin 2, Ireland;Department of Anatomy RCSI, 123 St Stephens Green, Dublin 2, Ireland;Department of Anatomy, RCSI Bahrain, Adliya, Kingdom of Bahrain

关键词: Cognitive load;    Multiple choice questions;    Images;    Histology;    Assessment;   

Others  :  1233404
DOI  :  10.1186/s12909-015-0452-9

 received in 2015-06-12, accepted in 2015-09-22,  发布年份 2015

【 摘 要 】

Background

Advances in cognitive load theory have led to greater understanding of how we process verbal and visual material during learning, but the evidence base with regard to the use of images within written assessments is still sparse. This study examines whether the inclusion of images within the stimulus format of multiple choice questions (MCQs) has a predictable or consistent influence on psychometric item properties, such as difficulty or discrimination.

Methods

Item analysis data from three consecutive years of histology multiple choice examinations were included in this study. All items were reviewed and categorised according to whether their stem, or stimulus format, was purely textual or included an associated image.

Results

A total of 195 MCQs were identified for inclusion and analysed using classical test theory; 95 used text alone and 100 included an image within the question stem. The number of students per examination ranged from 277 to 347, with a total of 60,850 student-question interactions. We initially examined whether the inclusion of an image within the item stem altered the item difficulty using Mann–Whitney U. The median item difficulty for images with purely textual stems was 0.77, while that for items incorporating an appropriate image was 0.80; this difference was not significant (0.77 vs. 0.80; p = 0.862, Mann–Whitney-U = 4818.5). Mean values showed that the Item Discrimination Index appeared unaffected by the inclusion of an image within the stem, and Item point biserial correlation also showed no difference in means between these two groups (Independent samples t-test; 2-tailed).

Conclusion

We demonstrate that the addition of illustrations within undergraduate histology Multiple Choice Question stems has no overall influence on item difficulty, or measures of item discrimination. We conclude that the use of images in this context is statistically uncritical, and suggest that their inclusion within item stems should be based upon the principles of constructive alignment. However, further research with respect to the effect of images within item stems on cognitive processing, particularly with regard to image complexity or type, would enable the development of more informed guidelines for their use.

【 授权许可】

   
2015 Holland et al.

【 图 表 】

Fig. 1.

Fig. 2.

Fig. 3.

Fig. 4.

【 参考文献 】

• [1]Benninger B, Matsler N, Delamarter T. Classic versus millennial medical lab anatomy. Clin Anat. 2014; 27:988-993.
• [2]Berends IE, Van Lieshout EC. The effect of illustrations in arithmetic problem-solving: Effects of increased cognitive load. Learn Instr. 2009; 19:345-353.
• [3]Beullens J, Struyf E, Van Damme B. Do extended matching multiple‐choice questions measure clinical reasoning? Med Educ. 2005; 39:410-417.
• [4]Biggs J. Aligning teaching and assessing to course objectives. Teach Learn Higher Educ: New Trends Innov. 2003; 2:13-17.
• [5]Bloodgood RA, Ogilvie RW. Trends in histology laboratory teaching in United States medical schools. Anat Rec B New Anat. 2006; 289:169-175.
• [6]Buzzard A, Bajsdaranayake R, Harvey C. How to produce visual material for multiple choice examinations. Med Teach. 1987; 9:451-456.
• [7]Carney R, Levin J. Pictorial illustrations still improve students' learning from text. Educ Psychol Rev. 2002; 14:5-26.
• [8]Case SM, Swanson DB. Constructing written test questions for the basic and clinical sciences. National Board of Medical Examiners, Philadelphia; 2002.
• [9]Chen R, Grierson LE, Norman GR. Evaluating the impact of high‐and low‐fidelity instruction in the development of auscultation skills. Med Educ. 2015; 49:276-285.
• [10]Coderre S, Woloschuk W, McLaughlin K. Twelve tips for blueprinting. Med Teach. 2009; 31:322-4.
• [11]Coderre SP, Harasym P, Mandin H, Fick G. The impact of two multiple-choice question formats on the problem-solving strategies used by novices and experts. BMC Med Educ. 2004; 4:23. BioMed Central Full Text
• [12]Cook MP. Visual representations in science education: the influence of prior knowledge and cognitive load theory on instructional design principles. Sci Educ. 2006; 90:1073-1091.
• [13]Crisp V, Sweiry E. Can a picture ruin a thousand words? The effects of visual resources in exam questions. Educ Res. 2006; 48:139-154.
• [14]de Champlain AF. A primer on classical test theory and item response theory for assessments in medical education. Med Educ. 2010; 44:109-117.
• [15]Engelhardt PV. An Introduction to Classical Test Theory as Applied to Conceptual Multiple-choice Tests. C Henderson and KA Harper, editors. Getting Started in PER. College Park: American Association of Physics Teachers; Reviews in PER, 2. 2009.
• [16]Gunderman RB, Wilson PK. Exploring the human interior: the roles of cadaver dissection and radiologic imaging in teaching anatomy. Acad Med. 2005; 80:745-749.
• [17]Hays RB, Hamlin G, Crane L. Twelve tips for increasing the defensibility of assessment decisions. Med Teach. 2014; 37(5):433-6.
• [18]Heidger PM, Dee F, Consoer D, Leaven T, Duncan J, Kreiter C. Integrated approach to teaching and testing in histology with real and virtual imaging. Anat Rec. 2002; 269:107-112.
• [19]Hunt DR. Illustrated multiple choice examinations. Med Educ. 1978; 12:417-420.
• [20]Korf H-W, Wicht H, Snipes RL, Timmermans J-P, Paulsen F, Rune G, Baumgart-vogt E. The dissection course–necessary and indispensable for teaching anatomy to medical students. Ann Anat. 2008; 190:16-22.
• [21]Langer MM, Swanson DB. Practical considerations in equating progress tests. Med Teach. 2010; 32:509-512.
• [22]Larsen DP, Butler AC, Roediger HL. Test-enhanced learning in medical education. Med Educ. 2008; 42:959-66.
• [23]Levie WH, Lentz R. Effects of text illustrations: a review of research. ECTJ. 1982; 30:195-232.
• [24]Mathai S, Ramadas J. Visuals and visualisation of human body systems. Int J Sci Educ. 2009; 31:439-458.
• [25]Mayer RE. Applying the science of learning to medical education. Med Educ. 2010; 44:543-549.
• [26]Mayer RE, Moreno R. Animation as an aid to multimedia learning. Educ Psychol Rev. 2002; 14:87-99.
• [27]Mayer RE, Moreno R. Nine ways to reduce cognitive load in multimedia learning. Educ Psychol. 2003; 38:43-52.
• [28]Nguyen N, Nelson AJ, Wilson TD. Computer visualizations: factors that influence spatial anatomy comprehension. Anat Sci Educ. 2012; 5:98-108.
• [29]O'Brien KE, Cannarozzi ML, Torre DM, Mechaber AJ, Durning SJ. Training and assessment of CXR/basic radiology interpretation skills: results from the 2005 CDIM survey. Teach Learn Med. 2008; 20:157-162.
• [30]Osterlind SJ, Everson HT. Differential item functioning. Vol 161. Sage Publications; 2009. https://uk. sagepub.com/en-gb/eur/differential-item-functioning/book230959#osterlind webcite
• [31]Paulsen FP, Eichhorn M, Bräuer L. Virtual microscopy—The future of teaching histology in the medical curriculum? Ann Anat. 2010; 192:378-382.
• [32]Ruiz JG, Cook DA, Levinson AJ. Computer animations in medical education: a critical literature review. Med Educ. 2009; 43:838-46.
• [33]Schuwirth L, Vleuten CVD, Donkers H. A closer look at cueing effects in multiple‐choice questions. Med Educ. 1996; 30:44-49.
• [34]Schuwirth LW, van der Vleuten CP. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ. 2004; 38:974-9.
• [35]Schuwirth LW, van der Vleuten CP. General overview of the theories used in assessment: AMEE Guide No. 57. Med Teach. 2011; 33:783-797.
• [36]Schuwirth LW, Verheggen M, van der Vleuten C, Boshuizen H, Dinant G. Do short cases elicit different thinking processes than factual knowledge questions do? Med Educ. 2001; 35:348-356.
• [37]Scoville SA, Buskirk TD. Traditional and virtual microscopy compared experimentally in a classroom setting. Clin Anat. 2007; 20:565-570.
• [38]Sugand K, Abrahams P, Khurana A. The anatomy of anatomy: a review for its modernization. Anat Sci Educ. 2010; 3:83-93.
• [39]Swanson DB, Clauser BE, Case SM, Nungester RJ, Featherman C. Analysis of Differential Item Functioning (DIF) using hierarchical logistic regression models. J Educ Behav Stat. 2002; 27:53-75.
• [40]Swanson DB, Holtzman KZ, Allbee K. Measurement characteristics of content-parallel single-best-answer and extended-matching questions in relation to number and source of options. Acad Med. 2008; 83:S21-4.
• [41]Szabo M, Dwyer FM, Demelo H. Visual testing—Visual literacy’s second dimension. ECTJ. 1981; 29:177-187.
• [42]Tarrant M, Ware J. A comparison of the psychometric properties of three-and four-option multiple-choice questions in nursing assessments. Nurse Educ Today. 2010; 30:539-543.
• [43]van der Vleuten CP, Schuwirth LW. Assessing professional competence: from methods to programmes. Med Educ. 2005; 39:309-17.
• [44]Vorstenbosch MA, Bouter ST, Hurk MM, Kooloos JG, Bolhuis SM, Laan RF. Exploring the validity of assessment in anatomy: Do images influence cognitive processes used in answering extended matching questions? Anat Sci Educ. 2014; 7:107-116.
• [45]Vorstenbosch MA, Klaassen TP, Kooloos JG, Bolhuis SM, Laan RF. Do images influence assessment in anatomy? Exploring the effect of images on item difficulty and item discrimination. Anat Sci Educ. 2013; 6:29-41.
• [46]WFME. WFME Global Standards for Quality Improvement in Medical Education - European Specifications. Denmark; 2007.
• [47]Wood T, Cole G, Lee C. Developing multiple choice questions for the RCPSC certification examinations. The Royal College of Physicians and Surgeons of Canada, Office of Education; 2004. http://www. schulich.uwo.ca/medicine/undergraduate/docs/about_us/teaching_aids/rcpsc_mcq_guidelines.pdf webcite
• [48]World Health Organization. eLearning for undergraduate health professional education - a systematic review informing a radical transformation of health workforce development. World Health Organization, Imperial College London; 2015. http://whoeducationguidelines. org/content/elearning-report webcite