skip to main content skip to footer

Automated Scoring of Short-Answer Open-Ended GRE Subject Test Items GRE NLP

Author(s):
Attali, Yigal; Powers, Donald E.; Freedman, Marshall; Harrison, Marissa; Obetz, Susan A.
Publication Year:
2008
Report Number:
RR-08-20
Source:
ETS Research Report
Document Type:
Report
Page Count:
22
Subject/Key Words:
Graduate Record Examination (GRE), Subject Tests (GRE), Constructed Response Items, Natural Language Processing (NLP), Automated Scoring, Automated Scoring and Natural Language Processing, c-rater

Abstract

This report describes the development, administration, and scoring of open-ended variants of GRE Subject Test items in biology and psychology. These questions were administered in a Web-based experiment to registered examinees of the respective Subject Tests. The questions required a short answer of 1-3 sentences, and responses were automatically scored by natural language processing methods, using the c-rater scoring engine, immediately after participants submitted their responses. Participants received immediate feedback on the correctness of their answers, and an opportunity to revise their answers. Subsequent human scoring of the responses allowed an evaluation of the quality of automated scoring. This report focuses on the success of the automated scoring process. A separate report describes the feedback and revision results.

Read More