An Investigation of IRT-Based Assembly of the TOEFL® Test

Chyn, Susan; Tang, K. Linda; Way, Walter D.
Publication Year:
Report Number:
RR-94-38, TOEFL-TR-09
Document Type:
Subject/Key Words:
Automated item selection (AIS) item pools item response theory parallel test forms test construction test items


The current study, carried out jointly by Test Development and Statistical Analysis staff at Educational Testing Service (ETS), investigated the feasibility of the automated item selection (AIS) procedure for the TOEFL® test (Test of English as a Foreign Language™). IRT-based statistical specifications were developed. Two TOEFL test forms were assembled using AIS, and statistical and content-related properties were evaluated. The results of this study show that the statistical consistency (parallelism) of the tests assembled using AIS would appear to be superior to the consistency of tests assembled using traditional test- assembly procedures. The results of the study also provided strong evidence that AIS-assembled TOEFL tests can successfully meet the IRT-based specifications. Test Development staff observed visible gains in efficiency in item selection for Section 1 and Section 2 and the potential for time gains in Section 3. The implications of AIS for pool management were explored.

Read More


Find a Publication

Advanced Search

Closing the Achievement Gap

Closing the Achievement GapLearn more about ETS's commitment to closing the achievement gap through rigorous research, thought-provoking forums and more.