skip to main content skip to footer

An Investigation of IRT-Based Assembly of the TOEFL Test AIS IRT TOEFL

Author(s):
Chyn, Susan; Tang, K. Linda; Way, Walter D.
Publication Year:
1994
Report Number:
RR-94-38
Source:
ETS Research Report
Document Type:
Report
Page Count:
47
Subject/Key Words:
Automated Item Selection (AIS), Item Pools, Item Response Theory (IRT), Parallel Test Forms, Test Construction, Test Items, Test of English as a Foreign Language (TOEFL)

Abstract

The current study, carried out jointly by Test Development and Statistical Analysis staff at Educational Testing Service (ETS), investigated the feasibility of the automated item selection (AIS) procedure for the TOEFL® test (Test of English as a Foreign Language™). IRT-based statistical specifications were developed. Two TOEFL test forms were assembled using AIS, and statistical and content-related properties were evaluated. The results of this study show that the statistical consistency (parallelism) of the tests assembled using AIS would appear to be superior to the consistency of tests assembled using traditional test- assembly procedures. The results of the study also provided strong evidence that AIS-assembled TOEFL tests can successfully meet the IRT-based specifications. Test Development staff observed visible gains in efficiency in item selection for Section 1 and Section 2 and the potential for time gains in Section 3. The implications of AIS for pool management were explored.

Read More