The assessment of a structured online formative assessment program: a randomised controlled trial

BMC Medical Education

Peter Devitt²  Edward Palmer¹ 

[1] School of Medicine, University of Adelaide, Adelaide, Australia;School of Education, University of Adelaide, Adelaide, Australia

关键词: Engagement;    Summative assessment;    Online learning;    Formative assessment;   

Others  :  1135668
DOI  :  10.1186/1472-6920-14-8

 received in 2012-11-28, accepted in 2013-12-30,  发布年份 2014

【 摘 要 】

Background

Online formative assessment continues to be an important area of research and methods which actively engage the learner and provide useful learning outcomes are of particular interest. This study reports on the outcomes of a two year study of medical students using formative assessment tools.

Method

The study was conducted over two consecutive years using two different strategies for engaging students. The Year 1 strategy involved voluntary use of the formative assessment tool by 129 students. In Year 2, a second cohort of 130 students was encouraged to complete the formative assessment by incorporating summative assessment elements into it. Outcomes from pre and post testing students around the formative assessment intervention were used as measures of learning. To compare improvement scores between the two years a two-way Analysis of Variance (ANOVA) model was fitted to the data.

Results

The ANOVA model showed that there was a significant difference in improvement scores between students in the two years (mean improvement percentage 19% vs. 38.5%, p < 0.0001). Students were more likely to complete formative assessment items if they had a summative component. In Year 2, the time spent using the formative assessment tool had no impact on student improvement, nor did the number of assessment items completed.

Conclusion

The online medium is a valuable learning resource, capable of providing timely formative feedback and stimulating student-centered learning. However the production of quality content is a time-consuming task and careful consideration must be given to the strategies employed to ensure its efficacy. Course designers should consider the potential positive impact summative components to formative assessment may have on student engagement and outcomes.

【 授权许可】

   
2014 Palmer and Devitt; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

【 参考文献 】

• [1]Harlen W: On the relationship between assessment for formative and summative purposes. In Assessment and learning. edn. Edited by London J, Gardner J. London, UK: Sage Pubns Ltd; 2006.
• [2]Carless D, Salter D, Yang M, Lam J: Developing sustainable feedback practices. Stud High Educ 2011, (4):395-407.
• [3]Clynes MP, Raftery SEC: Feedback: an essential element of student learning in clinical practice. Nurse Educ Pract 2008, 8(6):405-411.
• [4]University of Adelaide: SELT Aggregates: Health Sciences. University of Adelaide; 2011. [http://www.adelaide.edu.au/planning/selt/aggregates/2011/health/2011FY_Agg_HLTHSCI_(Standard%20Course%20Evaluation%20All%20Student%20Levels).pdf webcite] Retrieved January 2014
• [5]Joyce CM, Piterman L, Wesselingh SL: The widening gap between clinical, teaching and research work. Med J Aust 2009, 191(3):169-172.
• [6]Kumar K, Roberts C, Thistlethwaite J: Entering and navigating academic medicine: academic clinician-educators’ experiences. Med Educ 2011, 45(5):497-503.
• [7]Alexandraki I, Mooradian A: Academic advancement of clinician educators: why is it so difficult? Int J Clin Pract 2011, 65(11):1118-1125.
• [8]Bernard R, Abrami P, Lou Y, Borokhovski E, Wade A, Wozney L, Wallet P, Fiset M, Huang B: How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Rev Educ Res 2004, 74(3):379-439.
• [9]Al-Jewair T, Azarpazhooh A, Suri S, Shah P: Computer-assisted learning in orthodontic education: a systematic review and meta-analysis. J Dent Educ 2009, 73(6):730-739.
• [10]Cohen P, Dacanay LS: Computer-based instruction and health professions education: a meta-analysis of outcomes. Eval Health Prof 1992, 15(3):259-281.
• [11]Liao YC: Effects of computer-assisted instruction on studentsí achievement in Taiwan: A meta-analysis. Comput Educ 2007, 48(2):216-233.
• [12]Kadiyala M, Crynes B: Where's the proof? A review of literature on effectiveness of information technology in education. In Frontiers in Education Conference: 2002; Tempe. AZ, USA: IEEE; 2002:33-37.
• [13]Kulik C, Kulik J: Effectiveness of computer-based instruction: an updated analysis. Comput Hum Behav 1991, 7(1–2):75-94.
• [14]Khalili A, Shashaani L: The effectiveness of computer applications: a meta-analysis. J Res Comput Educ 1994, 27:48-61.
• [15]Means B, Toyama Y, Murphy R, Bakia M, Jones K: Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. 2010.
• [16]Palmer EJ, Devitt PG: Limitations of student-driven formative assessment in a clinical clerkship. A randomised controlled trial. BMC Med Educ 2008, 8:29. BioMed Central Full Text
• [17]Velan G, Jones P, McNeil HP, Kumar R: Integrated online formative assessments in the biomedical sciences for medical students: benefits for learning. BMC Med Educ 2008, 8(1):52. BioMed Central Full Text
• [18]Palmer EJ, Duggan P, Devitt PG, Russell R: The modified essay question: its exit from the exit examination? Med Teach 2010, 32(7):300-307.
• [19]Charlin B, Roy L, Brailovsky C, Goulet F, van der Vleuten C: The Script Concordance test: a tool to assess the reflective clinician. Teach Learn Med 2000, 12(4):189-195.
• [20]Uden L, Beaumont C: Technology and problem-based learning: Information Science Publishing. 2006.
• [21]Sadler DR: Perils in the meticulous specification of goals and assessment criteria. Assessment in Education 2007, 14(3):387-392.
• [22]Leach L, Neutze G, Zepke N: Assessment and empowerment: Some critical questions. Assessment & Evaluation in Higher Education 2001, 26(4):293-305.
• [23]Race P, Learning, Centre TSNG: A briefing on self, peer & group assessment: Learning and Teaching Support Network. 2001.
• [24]Regan JA: Motivating students towards self-directed learning. Nurse Educ Today 2003, 23(8):593-599.
• [25]Seale JK, Chapman J, Davey C: The influence of assessments on students’ motivation to learn in a therapy degree course. Med Educ 2000, 34(8):614-621.
• [26]Deci EL, Vallerand RJ, Pelletier LG, Ryan RM: Motivation and education: The self-determination perspective. Educ Psychol 1991, 26(3–4):325-346.
• [27]Sadler DR: Formative assessment and the design of instructional systems. Instr Sci 1989, 18(2):119-144.
• [28]Black P, Wiliam D: Assessment and classroom learning. Assessment in Education 1998, 5(1):7-74.
• [29]Greenhalgh T: Computer assisted learning in undergraduate medical education. BMJ 2001, 322(7277):40-44.
• [30]Letterie GS: Medical education as a science: the quality of evidence for computer-assisted instruction. Am J Obstet Gynecol 2003, 188(3):849-853.