Psychometric Evaluation of the New GRE Writing Assessment

Author(s):
Schaeffer, Gary A.; Briel, Jacqueline B.; Fowles, Mary E.
Publication Year:
2001
Report Number:
RR-01-08
GREB-96-11P
Source:
Document Type:
Subject/Key Words:
Writing constructs higher education writing assessment relative difficulty of essay prompts order effects of prompts subgroup performance

Abstract

This study analyzed examinee responses to two essay prompts being considered for the Graduate Record Examinations (GRE) Writing Assessment: "Present your Perspective on an Issue" and "Analyze an Argument." Forty prompts (20 issue and 20 argument) were administered to over 2,300 students at 26 participating U.S. colleges and universities. Each student wrote two essays in response to either two issue prompts, two argument prompts, or one of each. Results show that the issue and argument writing tasks appear to assess relatively similar constructs, supporting the decision to include both types of prompts in the operational GRE Writing Assessment and to report a single, combined score. The results also support the random administration of prompts without equating adjustments, because within each task type, most of the prompts were comparable in difficulty and no important subgroup interactions with prompt classifications were detected. Finally, from a fairness perspective, the results show there were advantages to administering the issue prompt first and the argument prompt second. The GRE Program used the psychometric information provided by this study to make final design, delivery, and scoring decisions before introducing the operational assessment in the fall of 1999.

Read More