angle-up angle-right angle-down angle-left close user menu open menu closed search globe bars phone store

Electronic Sources as Input to GRE Reading Comprehension Item Development: SourceFinder Prototype Evaluation

Author(s):
Passonneau, Rebecca; Hemat, Lisa; Plante, Janice; Sheehan, Kathleen M.
Publication Year:
2002
Report Number:
GREB-99-18P, RR-02-12
Source:
ETS Research Report
Document Type:
Report
Page Count:
60
Subject/Key Words:
Automated Scoring and Natural Language Processing Graduate Record Examination (GRE) Natural Language Processing (NLP) Reading Comprehension SourceFinder Test Development

Abstract

This evaluation study compares the performance of a prototype tool called SourceFinder against the performance of highly trained human test developers. SourceFinder - a specialized search engine developed to locate source material for Graduate Record Examinations (GRE) reading comprehension passages - employs a variety of shallow linguistic features to model the search criteria employed by expert test developers, to automate the source selection process, and to reduce source-processing time. The current evaluation provides detailed information about the aspects of source variation that are not well modeled by the current prototype. Approaches for enhancing performance in identified areas are discussed. The present study also provides a more explicit description of the source selection task, and a rich data set for developing a less subjective, more explicit definition of the types of documents preferred by test developers.

Read More