Proceedings of The International Conference on Research in Teaching and Education
Comparison of Different Test Termination Rules in Terms of Measurement Precision in Computerized Adaptive Testing
Arzu Uçar and Ebru Balta
Computerized Adaptive Tests (CAT) are gaining much more attention than ever by the institutions especially the ones attracting students worldwide due to the nature of CAT not allowing the same items to be presented to different individuals taking the test. The aim of this study is to measure the effect of different termination rules on measurement precision and test length in computer adaptive testing. The research was implemented as a Monte Carlo simulation study. The data generation of the computerized adaptive test was carried out using “catR” package. In comparing termination rules, starting rules (b=0 and -1<b<1), ability estimation methods (Maksimum Likelihood Estimation, Bayesian Method and Expected a Posteriori), stopping rules (fixed length (15, 20), standard error (SE<.30, SE<.50). 25 replications was performed for each condition in the generated data. Different conditions are in place in each termination rule and a total of 48 conditions are compared. RMSE, bias and fidelity values were calculated for the measurement precision was obtained and compared for each of the CAT implemantation. Not a significant change was observed in the evaluation of the effect of the starting rules. The preference of Expected a Posteriori method for the ability estimation is observed to cause a drop in values for RMSE and bias values.
Keywords: bias; fidelity; rmse; item response theory; measurement and evaluation.