skip to main content skip to footer

Does the Time Between Scoring Sessions Impact Scoring Accuracy? An Evaluation of Constructed‐Response Essay Responses on the GRE General Test GRE

Author(s):
Finn, Bridgid; Wendler, Cathy; Ricker-Pedley, Kathryn L.; Arslan, Burcu
Publication Year:
2018
Report Number:
RR-18-31
Source:
ETS Research Report
Document Type:
Report
Page Count:
12
Subject/Key Words:
Graduate Record Examinations (GRE), General Test (GRE), Forgetting Curves, Time, Constructed-Response Scoring, Human Raters, Rater Accuracy, Essay Tests, Calibration, Rater Cognition, Test Score Validity, Intervals

Abstract

This report investigates whether the time between scoring sessions has an influence on operational and nonoperational scoring accuracy. The study evaluates raters' scoring accuracy on constructed‐response essay responses for the GRE General Test. Binomial linear mixed‐effect models are presented that evaluate how the effect of various predictors, such as time spent scoring each response, days in a scoring gap, and number of consecutive days of scoring, relate to scoring accuracy. Results suggest that for operational scoring, the number of days in a scoring gap has a negative influence on performance. The findings, as well as other results from the models, are discussed in the context of cognitive influences on knowledge and skill retention.

Read More