angle-up angle-right angle-down angle-left close user menu open menu closed search globe bars phone store

The Consistency of TOEIC Speaking Scores Across Ratings and Tasks

Schmidgall, Jonathan
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Generalizability Theory Human Raters Rater Consistency Rater Reliability Scale Scores Test of English for International Communication (TOEIC) TOEIC Speaking Test Validation


This report briefly reviews the design and scoring procedure for the TOEIC Speaking test and summarizes existing evidence about the consistency of TOEIC Speaking test scores. It then describes several analyses conducted using generalizability theory to provide additional information about the consistency of scores across different aspects of the scoring procedure. The results of these analyses provide more robust information about consistency of TOEIC Speaking scores with respect to important facets of the assessment procedure, such as tasks and ratings. Specifically, the results provide evidence to support claims about the consistency of ratings across different levels of the scoring procedure: tasks, claims, and scale scores.

Read More