A Multidimensional IRT Approach for Dynamically Monitoring Ability Growth in Computerized Practice Environments

Frontiers in Psychology

article

Jung Yeon Park¹  Frederik Cornillie¹  Han L. J. van der Maas²  Wim Van Den Noortgate¹ 

[1] Faculty of Psychology and Educational Sciences;Department of Psychology, University of Amsterdam

关键词: multidimensional IRT;    Elo rating system;    adaptive practice;    speed-accuracy trade-off;    e-learning;   

DOI  :  10.3389/fpsyg.2019.00620

学科分类：社会科学、人文和艺术（综合）

来源: Frontiers

【 摘 要 】

Adaptive learning systems have received an increasing attention as they enable to provide personalized instructions tailored to the behaviors and needs of individual learners. In order to reach this goal, it is desired to have an assessment system, monitoring each learner's ability change in real time. The Elo Rating System (ERS), a popular scoring algorithm for paired competitions, has recently been considered as a fast and flexible method that can assess learning progress in online learning environments. However, it has been argued that a standard ERS may be problematic due to the multidimensional nature of the abilities embedded in learning materials. In order to handle this issue, we propose a system that incorporates a multidimensional item response theory model (MIRT) in the ERS. The basic idea is that instead of updating a single ability parameter from the Rasch model, our method allows a simultaneous update of multiple ability parameters based on a compensatory MIRT model, resulting in a multidimensional extension of the ERS (“M-ERS”). To evaluate the approach, three simulation studies were conducted. Results suggest that the ERS that incorrectly assumes unidimensionality has a seriously lower prediction accuracy compared to the M-ERS. Accounting for both speed and accuracy in M-ERS is shown to perform better than using accuracy data only. An application further illustrates the method using real-life data from a popular educational platform for exercising math skills.

【 授权许可】

CC BY