angle-up angle-right angle-down angle-left close user menu open menu closed search globe bars phone store

Investigating the Value of TOEFL iBT Section Scores

Sawaki, Yasuyo; Sinharay, Sandip
Publication Year:
Report Number:
RR-13-35, TOEFLiBT-21
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Dimensionality Factor Analysis Generalizability Theory Reliability Score Dependability Score Profile Subscore Analysis Test of English as a Foreign Language (TOEFL) TOEFL iBT


This study investigates the value of reporting the reading, listening, speaking, and writing section scores for the TOEFL iBT test, focusing on 4 related aspects of the psychometric quality of the TOEFL iBT section scores: reliability of the section scores, dimensionality of the test, presence of distinct score profiles, and the section scores’ generalizability for norm-referenced decisions as well as the dependability of criterion-referenced decisions for international student admission. Four operational TOEFL iBT test forms were analyzed for all examinees as well as for 3 native language (L1) groups (Arabic, Korean, and Spanish). Haberman’s (2008) subscore analysis suggested that the speaking section score had added value due to its relative distinctness from the other modalities. Consistent with the subscore analysis results, a series of exploratory factor analyses (EFAs) indicated the possibility of the presence of 2 correlated factors—a reading/listening/writing factor and a speaking factor. In contrast, the CFAs conducted separately for the 3 L1 groups as well as a multiple-group confirmatory factor analyses (CFAs) identified a correlated 4-factor model with reading, listening, speaking, and writing factors as the best representation of the structure of the entire test for all examinees as well as for the 3 L1 groups. Reliability of the observed section scores for norm-referenced score interpretations and the dependability of classification decisions made based on different cut scores were generally satisfactory while they were also found to be relatively low in some circumstances. Based on the mixed results concerning the value-added information the TOEFL iBT section scores provide, recommendations for future research directions and some key issues of consideration for high-stakes decision making based on the section scores were summarized.

Read More