GRE Automated-Editing-Task Phase II Report: Item Analyses, Revisions, Validity Study, and Taxonomy Development
- Author(s):
- Breland, Hunter; Kukich, Karen; Hemat, Lisa
- Publication Year:
- 2001
- Report Number:
- RR-01-12
- GREB-98-05P
- Source:
- Document Type:
- Subject/Key Words:
- Writing assessment automation essays diagnosis
Abstract
Two automated editing tasks developed in a Phase I study were subjected to item analyses, revised, and then used in a computer-based test administration at a local college. The data collected in the administration were compared with questionnaire data obtained from students to examine the construct validity of the tasks. In a second approach to construct validation, a taxonomy of writing skills was developed and compared to the skills assessed by the editing tasks. Data analyses indicate that total editing score correlates more strongly with self-reported English grades than with self-reported mathematics grades, and that total editing score correlates positively with student self-assessments of their writing skill, recent grades on writing assignments, and college grade point average. A review of the task elements against the taxonomy indicates that the editing tasks assess important writing skills not assessed by free-response essays. For this and other reasons, it was concluded that automated editing tasks would serve as a useful complement to free-response writing assessments.