skip to main content skip to footer

Potential Impact of Context Effects on the Scoring and Equating of the Multistage GRE Revised General Test GRE IRT

Author(s):
Davey, Tim; Lee, Yi-Hsuan
Publication Year:
2011
Report Number:
RR-11-26
Source:
ETS Research Report
Document Type:
Report
Page Count:
44
Subject/Key Words:
Graduate Record Examination (GRE), Context Effects, Position Effects, Randomization, Test Revision, Item Response Theory (IRT)

Abstract

Both theoretical and practical considerations have led the revision of the Graduate Record Examinations (GRE) revised General Test, here called the rGRE, to adopt a multistage adaptive design that will be continuously or nearly continuously administered and that can provide immediate score reporting. These circumstances sharply constrain the available options for equating or linking together the thousands of unique test forms that will need to be administered each year. The only practical method is to pre-equate each form through item response theory (IRT) methods. Unfortunately, doing so means that item position and context effects are both vitally important and potentially difficult to control. This study examines item position effects in a GRE context and attempts to assess the extent to which they may affect the rGRE. It was found that position effects are in fact present in GRE data and that they can be particularly troublesome under multistage testing. Pretest strategies are able to mitigate effects to some degree, but complete control will require careful attention be paid to the time limits and associated test configuration details in the design of the rGRE.

Read More