skip to main content skip to footer

Method and System for Detecting Off-Topic Essays Without Topic-Specific Training

Higgins, Derrick; Burstein, Jill
Patent Issued:
May 04, 2010
Patent Number:
ETS Patent
Document Type:
Family ID:
Subject/Key Words:
Patent, Active Patent, Automated Response Evaluation, Automated Essay Scoring (AES), Automated Scoring and Natural Language Processing, Essay Topics, Writing Errors


Methods and systems for detecting off-topic essays are described that do not require training using human-scored essays. The methods can detect different types of off-topic essays, such as unexpected topic essays and bad faith essays. Unexpected topic essays are essays that address an incorrect topic. Bad faith essays address no topic. The methods can use content vector analysis to determine the similarity between the essay and one or more prompts. If the essay prompt with which an essay is associated is among the most similar to the essay, the essay is on-topic. Otherwise, the essay is considered to be an unexpected topic essay. Similarly, if the essay is sufficiently dissimilar to all essay prompts, the essay is considered to be a bad faith essay.

Read More