skip to main content skip to footer

A Consideration for Variable Length Adaptive Tests CAT

Author(s):
Wingersky, Marilyn S.
Publication Year:
1989
Report Number:
RR-89-40
Source:
ETS Research Report
Document Type:
Report
Page Count:
32
Subject/Key Words:
Adaptive Testing, Computer Assisted Testing, Scoring, Test Length

Abstract

In a variable-length adaptive test with a stopping rule that relied on the asymptotic standard error of measurement of the examinee's estimated true score, Stocking (1987) discovered that it was sufficient to know the examinee's true score and the number of items administered to predict with some accuracy whether an examinee's true score was over or under estimated. She theorized that this result might be due to the standard error being correct only asymptotically. Sympson (1985) recommended two Bayesian stopping rules that do not rely on any asymptotic properties. This paper replicates the Stocking study using one of the variable-length adaptive testing procedures recommended by Sympson to see whether that procedure gives the same result as found in the Stocking study. The Sympson procedure uses a stopping rule that relies on the posterior standard deviation of the number-right true score on a criterion test. In both the Stocking study and the Sympson procedure, knowing the examinee's true score and the number of items administered is sufficient for predicting whether the estimated true score is over or under estimated. This is due to the fact that the magnitude of both the asymptotic standard error of measurement of the estimated scores and the posterior standard deviation of the true score varies scores and the posterior standard deviation of the true score varies as the estimated true score varies. (32pp.)

Read More