skip to main content skip to footer

A Study of the Use of Collateral Statistical Information in Attempting to Reduce TOEFL IRT Item Parameter Estimation Sample Sizes TOEFL IRT CBT

Author(s):
Tang, K. Linda; Eignor, Daniel R.
Publication Year:
2001
Report Number:
RR-01-11, TOEFL-TR-17
Source:
ETS Research Report
Document Type:
Report
Page Count:
13
Subject/Key Words:
Test of English as a Foreign Language (TOEFL), Collateral Information, Marginal Maximum Likelihood Estimation, Item Parameter Estimation, Item Response Theory (IRT), Pretesting, Computer-Based Testing (CBT)

Abstract

The purpose of this study was to investigate whether classical item statistics could be used as collateral information in the IRT calibration of pretest items for the computer-based Test of English as a Foreign Language« using BILOG, and thus lead to a reduction in examinee sample sizes. The development and maintenance of item pools to support computer-based testing (CBT) programs have placed much greater demands on the item pretesting process than was the case with paper-and-pencil testing, and this study attempted to show whether, using this procedure, more items than might be expected could be pretested, given a fixed overall examinee volume. Data from three TOEFL pretest item pools used in implementing CBT were used to simulate conditions under which the effects of reduced sample sizes, augmented by statistical collateral information from classical item statistics, could be investigated. Results indicated that classical statistics alone do not provide a sufficient level of collateral information to allow a reduction in pretest sample sizes. Reasons for these results are offered and suggestions for further research are provided.

Read More