How Reliable is the TOEFL® Test?

Author(s):
Wainer, Howard; Lukhele, Robert
Publication Year:
1997
Report Number:
RR-97-08
TOEFL-TR-12
Source:
Document Type:
Subject/Key Words:
Item response models item response theory test reliability testlets

Abstract

We estimated the reliability of scores on four forms of the TOEFL® test (Test of English as a Foreign Language™) using a hybrid IRT model. Very little difference between their overall reliability was found when the testlet items were assumed to be independent and their dependence was modeled. A larger difference in reliability was found when test sections were analyzed individually. Then we found as much as a 40% overestimate in reading comprehension testlets, with the longer testlets of the newest form of the TOEFL test showing the most local dependence. The listening comprehension testlets exhibited much less local dependence. We also found that the test was unidimensional enough for the use of univariate IRT to be efficacious, and that the reading comprehension testlets showed essentially no differential functioning by sex.