angle-up angle-right angle-down angle-left close user menu open menu closed search globe bars phone store

Variations in Mean Response Times for Questions on the Computer-Adaptive GRE General Test: Implications for Fair Assessment

Author(s):
Bridgeman, Brent; Cline, Frederick
Publication Year:
2000
Report Number:
GREB-96-20P, RR-00-07
Source:
ETS Research Report
Document Type:
Report
Page Count:
29
Subject/Key Words:
Computer Adaptive Testing Graduate Record Examination Graduate Record Examinations Board GRE Computer Adaptive Test Speededness (Tests) Test Fairness Validity

Abstract

In a computer adaptive test (CAT), different examinees receive different sets of questions. Questions at the same overall difficulty level and meeting roughly the same content specifications could vary substantially in the amount of time needed to answer them. If the CAT is speeded (that is, if substantial numbers of students either do not finish or must guess randomly at the end to finish), individuals who happened to get a disproportionate number of questions that took a long time to answer could be disadvantaged. The purpose of this study was to determine whether--in computer-adaptive testing situations-- the administration of a disproportionate number of questions with long, expected response times unfairly disadvantages examinees. Data from 5,957 examinees who took the computer-delivered Graduate Record Examination (GRE) quantitative measure and 14,745 examinees who took the computer-delivered GRE analytical measure were used to investigate variation in response time in light of other factors, such as mean test score and the position of the question on the test. For both measures, substantial variation in response times was found, even for items with the same specifications and same difficulty level. But despite these differences, there was no indication that the scores of students who were administered items with long expected response times were disadvantaged.

Read More