Integrating a Statistical Topic Model and a Diagnostic Classification Model for Analyzing Items in a Mixed Format Assessment

Frontiers in Psychology

Jonathan Templin¹  Seohyun Kim²  Allan S. Cohen³  H.-J. Choi³  Yasemin Copur-Gencturk⁴ 

[1] Department of Psychological and Quantitative Foundations, University of Iowa, Iowa City, IA, United States;Department of Psychology, University of Virginia, Charlottesville, VA, United States;Georgia Center for Assessment, Department of Educational Psychology, University of Georgia, Athens, GA, United States;Rossier School of Education, University of Southern California, Los Angeles, CA, United States;

关键词: text analysis;    mixed format test;    diagnostic classification model;    structural topic model;    statistical topic models;   

DOI  :  10.3389/fpsyg.2020.579199

来源: Frontiers

【 摘 要 】

Selected response items and constructed response (CR) items are often found in the same test. Conventional psychometric models for these two types of items typically focus on using the scores for correctness of the responses. Recent research suggests, however, that more information may be available from the CR items than just scores for correctness. In this study, we describe an approach in which a statistical topic model along with a diagnostic classification model (DCM) was applied to a mixed item format formative test of English and Language Arts. The DCM was used to estimate students’ mastery status of reading skills. These mastery statuses were then included in a topic model as covariates to predict students’ use of each of the latent topics in their written answers to a CR item. This approach enabled investigation of the effects of mastery status of reading skills on writing patterns. Results indicated that one of the skills, Integration of Knowledge and Ideas, helped detect and explain students’ writing patterns with respect to students’ use of individual topics.

【 授权许可】

CC BY