Analysis of Discourse Features and Verification of Scoring Levels for Independent and Integrated Prototype Written Tasks for the New TOEFL® Test

Author(s):
Cumming, Alister; Kantor, Robert; Baba, Kyoko; Eouanzoui, Keanre; Erdosy, Usman; James, Mark
Publication Year:
2006
Report Number:
RR-05-13, TOEFL-MS-30
Source:
Document Type:
Subject/Key Words:
Discourse analysis ESL composition assessment ESL proficiency independent tasks integrated tasks second-language writing text characteristics

Abstract

We assessed whether and how the discourse written for prototype integrated tasks (involving writing in response to print or audio source texts) field tested for the new TOEFL® test differs from the discourse written for independent essays (i.e., the TOEFL essay). We selected 216 compositions written for 6 tasks by 36 examinees in a field test—representing Score Levels 3, 4, and 5 on the TOEFL essay—then coded the texts for lexical and syntactic complexity, grammatical accuracy, argument structure, orientations to evidence, and verbatim uses of source text. Analyses with nonparametric MANOVAs, following a 3-by-3 (task type by English proficiency level) within-subjects factorial design, showed that the discourse produced for the integrated writing tasks differed significantly at the lexical, syntactic, rhetorical, and pragmatic levels from the discourse produced in the independent essay on most of these variables. In certain analyses, these differences were also obtained across the 3 ESL proficiency levels.

Read More