An investigation of item "bounce" in a special grammar test

item bounce;language testing

Xu, Jingru ; Davidson ; Frederick G.

关键词: item bounce;    language testing;   

Others  :  https://www.ideals.illinois.edu/bitstream/handle/2142/16724/1_Xu_Jingru.pdf?sequence=2&isAllowed=y

美国|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

In this thesis, the focus is the analysis of bounced language test items - rejected test questions which, if retained, may result in strange effects compared with normal items.Typically, test items are expected to show the real language trait that the test is designed to measure, whereas “item bounce” is an example of error measurement.In this research -- a special English grammar test -- the researcher specifically constructed some items intended to bounce and tried to find whether these items lead to statistical errors as expected.When such errors were detected, the research investigated those errors through item-total correlation, distracter analysis, DIF analysis between native-speakers group and non-native speakers group, and item characteristic curves. However, the research went one step further: it actively engaged the test-takers as what Gene Roddenberry called: ‘intelligent life forms’.Test-takers were encouraged to comment critically on test items, and in particular, to provide insight about the intentionally bounced items.In order to achieve this goal, two types of questions were designed in the test, the multiple-response multiple-choice questions and the open-ended evaluative questions.The latter allowed test-takers to comment on the former.Those evaluation responses helped the researcher to take a closer look at the reasons behind those multiple choices; thus, they gave a comprehensive explanation for the reasons that caused item bounce.In conclusion, the special design of the test combined with the statistical measurement analysis suggests new measurement tools that would improve the language test design and benefit the traditional language testing development. Hopefully, this research could also give suggestions for language test designers on what they could do and what they should not try while listing some potential risks that may occur during the test design and test administration.