Ability evaluation by binary tests: Problems, challenges & recent advances

2016 Joint IMEKO TC1-TC7-TC13 Symposium: Metrology Across the Sciences: Wishful Thinking?

Bashkansky, E.^1 ; Turetsky, V.^2

Industrial Engineering and Management Department, ORT Braude College of Engineering, P.O. Box 78, Karmiel

2161002, Israel^1

Department of Applied Mathematics, ORT Braude College of Engineering, P.O. Box 78, Karmiel

2161002, Israel^2

关键词: Ability evaluation;    Binary tests;    Computational tools;    Item response models;    Proficiency testing;    Scale invariant;    Theory and practice;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/772/1/012012/pdf
DOI  :  10.1088/1742-6596/772/1/012012

来源: IOP

【 摘 要 】

Binary tests designed to measure abilities of objects under test (OUTs) are widely used in different fields of measurement theory and practice. The number of test items in such tests is usually very limited. The response to each test item provides only one bit of information per OUT. The problem of correct ability assessment is even more complicated, when the levels of difficulty of the test items are unknown beforehand. This fact makes the search for effective ways of planning and processing the results of such tests highly relevant. In recent years, there has been some progress in this direction, generated by both the development of computational tools and the emergence of new ideas. The latter are associated with the use of so-called "scale invariant item response models". Together with maximum likelihood estimation (MLE) approach, they helped to solve some problems of engineering and proficiency testing. However, several issues related to the assessment of uncertainties, replications scheduling, the use of placebo, as well as evaluation of multidimensional abilities still present a challenge for researchers. The authors attempt to outline the ways to solve the above problems.