skip to main content skip to footer

Reliability of Speeded Number-right Multiple-choice Tests

Author(s):
Attali, Yigal
Publication Year:
2004
Report Number:
RR-04-15
Source:
ETS Research Report
Document Type:
Report
Page Count:
15
Subject/Key Words:
Reliability, Test Speededness, Multiple-Choice Tests

Abstract

Contrary to common belief, reliability estimates of number-right multiple-choice tests are not inflated by speededness. Because examinees guess on questions when they run out of time, the responses to these questions show less consistency with the responses of other questions, and the reliability of the test will be decreased. The surprising implication is that adding questions to a multiple-choice test may lower its reliability when the test is speeded. This paper develops the mathematical derivations and shows the effects of speededness on reliability in simulations.

Read More