skip to main content skip to footer

The Consistency of TOEIC Speaking Scores Across Ratings and Tasks TOEIC

Schmidgall, Jonathan
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Generalizability Theory, Rater Consistency, Rater Reliability, Validation, Test of English for International Communication (TOEIC), TOEIC Speaking Test, Human Raters, Scale Scores


This report briefly reviews the design and scoring procedure for the TOEIC Speaking test and summarizes existing evidence about the consistency of TOEIC Speaking test scores. It then describes several analyses conducted using generalizability theory to provide additional information about the consistency of scores across different aspects of the scoring procedure. The results of these analyses provide more robust information about consistency of TOEIC Speaking scores with respect to important facets of the assessment procedure, such as tasks and ratings. Specifically, the results provide evidence to support claims about the consistency of ratings across different levels of the scoring procedure: tasks, claims, and scale scores.

Read More