GRE Automated-Editing-Task Phase II Report: Item Analyses, Revisions, Validity Study, and Taxonomy Development

Author(s):
Breland, Hunter; Kukich, Karen; Hemat, Lisa
Publication Year:
2001
Report Number:
RR-01-12
GREB-98-05P
Source:
Document Type:
Subject/Key Words:
Writing assessment automation essays diagnosis

Abstract

Two automated editing tasks developed in a Phase I study were subjected to item analyses, revised, and then used in a computer-based test administration at a local college. The data collected in the administration were compared with questionnaire data obtained from students to examine the construct validity of the tasks. In a second approach to construct validation, a taxonomy of writing skills was developed and compared to the skills assessed by the editing tasks. Data analyses indicate that total editing score correlates more strongly with self-reported English grades than with self-reported mathematics grades, and that total editing score correlates positively with student self-assessments of their writing skill, recent grades on writing assignments, and college grade point average. A review of the task elements against the taxonomy indicates that the editing tasks assess important writing skills not assessed by free-response essays. For this and other reasons, it was concluded that automated editing tasks would serve as a useful complement to free-response writing assessments.

Read More