skip to main content skip to footer

The Consistency of TOEIC Speaking Scores Across Ratings and Tasks TOEIC

Author(s):
Schmidgall, Jonathan
Publication Year:
2017
Report Number:
RR-17-46
Source:
ETS Research Report
Document Type:
Report
Page Count:
10
Subject/Key Words:
Generalizability Theory, Rater Consistency, Rater Reliability, Validation, Test of English for International Communication (TOEIC), TOEIC Speaking Test, Human Raters, Scale Scores

Abstract

This report briefly reviews the design and scoring procedure for the TOEIC Speaking test and summarizes existing evidence about the consistency of TOEIC Speaking test scores. It then describes several analyses conducted using generalizability theory to provide additional information about the consistency of scores across different aspects of the scoring procedure. The results of these analyses provide more robust information about consistency of TOEIC Speaking scores with respect to important facets of the assessment procedure, such as tasks and ratings. Specifically, the results provide evidence to support claims about the consistency of ratings across different levels of the scoring procedure: tasks, claims, and scale scores.

Read More