skip to main content skip to footer

Automated Subscores for TOEFL iBT Independent Essays AES TOEFL

Author(s):
Attali, Yigal
Publication Year:
2011
Report Number:
RR-11-39
Source:
ETS Research Report
Document Type:
Report
Page Count:
16
Subject/Key Words:
Automated Essay Scoring (AES), Electronic Essay Rater (E-rater), Subscores, Test of English as a Foreign Language (TOEFL), Automated Scoring and Natural Language Processing

Abstract

The e-rater automated essay scoring system is used operationally in the scoring of TOEFL iBT independent essays. Previous research has found support for a 3-factor structure of the e-rater features. This 3-factor structure has an attractive hierarchical linguistic interpretation with a word choice factor, a grammatical convention within a sentence factor, and a fluency factor. The purpose of this study was to explore the feasibility of developing automated subscores for the TOEFL iBT independent task based on this 3-factor structure. First, using a multiple-group confirmatory factor analysis, the 3-factor structure was found to be quite stable across major language groups. Next, the subscores based on these 3 factors were found to have added value in the context of repeater examinees by comparing the ability to predict a subscore on one test from the subscore on the other test or from the total e-rater score on the other test. The results of this study could be used to develop and report subscores for the performance of examinees on TOEFL iBT independent prompts. Reporting subscores could have instructional and remedial value, both at the individual and institutional level.

Read More