A Study of the Use of Collateral Statistical Information in Attempting to Reduce TOEFL® IRT Item Parameter Estimation Sample Sizes

Author(s):
Tang, K. Linda; Eignor, Daniel R.
Publication Year:
2001
Report Number:
RR-01-11, TOEFL-TR-17
Source:
Document Type:
Subject/Key Words:
IRT item parameter estimation marginal maximum likelihood Bayesian priors collateral information online pretesting

Abstract

The purpose of this study was to investigate whether classical item statistics could be used as collateral information in the IRT calibration of pretest items for the computer-based TOEFL® using BILOG and thus lead to a reduction in examinee sample sizes. The development and maintenance of item pools to support computer-based testing (CBT) programs have placed much greater demands on the item pretesting process than was the case with paper-and-pencil testing, and this study attempted to show whether, using this procedure, more items than might be expected could be pretested, given a fixed overall examinee volume. Data from three TOEFL pretest item pools used in implementing CBT were used to simulate conditions under which the effects of reduced sample sizes, augmented by statistical collateral information from classical item statistics, could be investigated. Results indicated that classical statistics alone do not provide a sufficient level of collateral information to allow a reduction in pretest sample sizes. Reasons for these results are offered and suggestions for further research are provided.

Read More