Explore the moving sample size used in a procedure for detecting compromised items based on item response theory

Item Response Theory (IRT)-based detection procedure;Moving sample size;Power;Lag

Jie, Yongjing ; Chang, Hua-Hua, ; ,Anderson, Carolyn J. ; Zhang ; Jinming

关键词: Item Response Theory (IRT)-based detection procedure;    Moving sample size;    Power;    Lag;   

Others  :  https://www.ideals.illinois.edu/bitstream/handle/2142/99519/JIE-THESIS-2017.pdf?sequence=1&isAllowed=y

美国|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

A computerized adaptive testing (CAT) system can provide tests for examinees continuously, so items in the item bank face the risk of being leaked. The IRT-based detection procedure developed by Zhang and Li (2016) is further studied in this thesis, which is designed to identify compromised items via a series of hypothesis tests. The moving sample size m in the Zhang and Li procedure is very important because it can affect the lag and the power of the procedure in identifying compromised items. In this thesis, the author will explore the relationship between the moving sample size and the lag, and the relationship between the moving sample size and the power. The simulation results indicate that at the significance level of 0.05, moving sample size m has an exponential relationship with the lag, and has a quadratic relationship with the power. At the significance level of 0.01, moving sample size m has a logarithmic relationship with the lag, and has a cubic relationship with the power. The author attempts to find an appropriate value of moving sample size m to achieve a large value of the power and a small value of the lag. The simulation results show that at both significance levels 0.05 and 0.01, setting the moving sample size m to 200 can accomplish the mission, while setting moving sample size m to 10 will decrease the performance of the procedure in the case that the significance level is 0.05.