skip to main content skip to footer

A Trend-Scoring Study for the TOEFL iBT Speaking and Writing Sections TOEFL iBT

Author(s):
Li, Yanmei; Brown, Terran
Publication Year:
2013
Report Number:
RM-13-05
Source:
ETS Research Memorandum
Document Type:
Report
Page Count:
28
Subject/Key Words:
Trend Scoring Method, Constructed Responses, Rater Effect, Linear Equating, Test of English as a Foreign Language (TOEFL), Internet Based Testing (iBT)

Abstract

A trend-scoring study was conducted to evaluate whether raters maintain the same scoring standards across administrations for the constructed responses (CR) in the TOEFL iBT® Speaking and Writing sections. Random samples of TOEFL iBT Speaking and Writing responses were selected from a test administration and rescored by raters when the same test was administered a second time. The trend CR score distributions were compared with their original score distributions to check score consistency. Results suggest that item and raw-score means, standard deviations, and frequency distributions are similar between the original administration and readministration of the test. Additionally, results from a linear equating indicate that any differences found in the raw score distributions can be attributed to random variation. This study demonstrated that the trend-scoring procedure could be implemented in the Speaking and Writing scoring for the TOEFL iBT program. It is recommended that the TOEFL iBT program use this procedure as a tool to periodically monitor Speaking and Writing scoring.

Read More