How Reliable is the TOEFL® Test?

Wainer, Howard; Lukhele, Robert
Publication Year:
Report Number:
Document Type:
Subject/Key Words:
Item response models item response theory test reliability testlets


We estimated the reliability of scores on four forms of the TOEFL® test (Test of English as a Foreign Language™) using a hybrid IRT model. Very little difference between their overall reliability was found when the testlet items were assumed to be independent and their dependence was modeled. A larger difference in reliability was found when test sections were analyzed individually. Then we found as much as a 40% overestimate in reading comprehension testlets, with the longer testlets of the newest form of the TOEFL test showing the most local dependence. The listening comprehension testlets exhibited much less local dependence. We also found that the test was unidimensional enough for the use of univariate IRT to be efficacious, and that the reading comprehension testlets showed essentially no differential functioning by sex.


Find a Publication

Advanced Search

Closing the Achievement Gap

Closing the Achievement GapLearn more about ETS's commitment to closing the achievement gap through rigorous research, thought-provoking forums and more.