A power study of a compromised item detection procedure based on item response theory under different scenarios of subjects’ latent trait

Compromised item detection procedure;Ability with linear growth;Ability with periodical variation;Power;Lag

Cao, Canxi ; Anderson, Carolyn J., ; ,Chang, Hua-Hua ; Zhang ; Jinming

关键词: Compromised item detection procedure;    Ability with linear growth;    Ability with periodical variation;    Power;    Lag;   

Others  :  https://www.ideals.illinois.edu/bitstream/handle/2142/99525/CAO-THESIS-2017.pdf?sequence=1&isAllowed=y

美国|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

This thesis explores whether or not the changing of students’ latent trait influences the power and the lag performance of a detection procedure based on Item Response Theory for successfully identifying a compromised item in computerized adaptive testing. A simulation study was conducted under three scenarios. The first scenario is the regular scenario, where the students’ latent trait follows the standard normal distribution. In the other two scenarios, the mean of true ability of student population can change in different pattern but not due to the item compromising. Therefore, this simulation mimics two more difficult scenarios, where one shows the ability with linear growth, and the other one has the ability with periodical variation.The simulation experiment yielded five main findings. (1) The mean and median of the distribution of the ability with linear growth scenario were larger than that under the other two scenarios, and the dispersion level of the distribution of the ability with periodical variation scenario is wider than that under the other two conditions; (2) The critical value c_0.01 is always higher than c_0.05, and the value of the moving sample size hardly affects the critical value c_α when moving sample size is greater than 20; (3) Nearly all of the items in the item pool are monitored under all three scenarios conducted in the simulation; (4) The detection procedure always holds a high quality of power (almost stays at 1 all the time); that is, it would not be affected by the changing of students’ latent traits in terms of the power index; (5) The critical values would produce a little bit longer lag under the setting of ability with linear growth scenario than the regular scenario, and there is no difference between the regular scenario and the ability with periodical variation scenario; 6)There is no significant difference of the value of power between α=0.01 and α=0.05, and also for lag.