Topical Trends in a Corpus of Persuasive Writing

Author(s):
Heilman, Michael; Madnani, Nitin
Publication Year:
2012
Report Number:
RR-12-19
Source:
ETS Research Report
Document Type:
Report
Page Count:
23
Subject/Key Words:
Automated Scoring Computational Linguistics Time Series Trends Writing Assessment

Abstract

Many writing assessments use generic prompts about social issues. However, we currently lack an understanding of how test takers respond to such prompts. In the absence of such an understanding, automated scoring systems may not be as reliable as they could be and may worsen over time. To move toward a deeper understanding of responses to generic issue prompts, we analyzed topical trends in test takers’ responses and correlated these trends with those found in the news. We found evidence that many trends are similar across essays and the news. However, we also observed some interesting differences. Based on these analyses, we make recommendations in this paper for developers of writing assessments and automated scoring systems.

Read More