skip to main content skip to footer

On the Reliability of Testlet-Based Tests IRT

Author(s):
Sireci, Stephen G.; Thissen, David; Wainer, Howard
Publication Year:
1991
Report Number:
RR-91-22
Source:
ETS Research Report
Document Type:
Report
Page Count:
15
Subject/Key Words:
Item Response Theory (IRT), Statistical Analysis, Test Reliability, Test Theory, Testlets

Abstract

If a test is constructed of testlets, one must take into account the within-testlet structure in the calculation of test statistics. Failing to do so may yield serious biases in the estimation of such statistics as reliability. In this paper we demonstrate how to calculate the reliability of a testlet-based test. We show that traditional reliabilities calculated on two reading comprehension tests constructed of four testlets are substantial overestimates. (15pp.)

Read More