skip to main content skip to footer

How Reliable is the TOEFL Test? IRT TOEFL

Wainer, Howard; Lukhele, Robert
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
TOEFL Policy Council, Item Response Models, Item Response Theory (IRT), Test Reliability, Test of English as a Foreign Language (TOEFL), Testlets


We estimated the reliability of scores on four forms of the TOEFL® test (Test of English as a Foreign Language™) using a hybrid IRT model. Very little difference between their overall reliability was found when the testlet items were assumed to be independent and their dependence was modeled. A larger difference in reliability was found when test sections were analyzed individually. Then we found as much as a 40% overestimate in reading comprehension testlets, with the longer testlets of the newest form of the TOEFL test showing the most local dependence. The listening comprehension testlets exhibited much less local dependence. We also found that the test was unidimensional enough for the use of univariate IRT to be efficacious, and that the reading comprehension testlets showed essentially no differential functioning by sex.

Read More