skip to main content skip to footer

An Adaptive Algebra Test: A Testlet-based,Hierarchically-Structured Test With Validity-based Scoring CAT

Author(s):
Wainer, Howard; Lewis, Charles; Kaplan, Bruce A.; Braswell, James S.
Publication Year:
1990
Report Number:
RR-90-21
Source:
ETS Research Report
Document Type:
Report
Page Count:
29
Subject/Key Words:
Adaptive Testing, Algebra Tests, Computer Assisted Testing, Test Validity

Abstract

An earlier report (Wainer & Lewis, 1990) described the initial development of a testlet-based algebra test. In this account the details of this excursion into the use of hierarchial testlets and validity-based scoring are provided. A pretest of two 15-item hierarchial testlets was carried out in which examinees' performance on a four-item subset of each testlet was used to predict performance on the entire testlet. Four models for constructing hierarchies were considered. These presentation hierarchies were compared with one another and with an optimally chosen set of four linearly administered items. The comparison was carried out using both the root mean square error and the conditional posterior variance as the criterion. It was found on cross validation that although an adaptive test is everywhere superior to a fixed format test, this superiority is crucially dependent upon the quality of the items. When items vary considerably in quality, a fixed format test, which uses the best items, can do almost as well as an adaptive test of equal length. (29pp.)

Read More