skip to main content skip to footer

Revising Answers to Items in Computerized Adaptive Tests: A Comparison of Three Models CAT

Author(s):
Stocking, Martha L.
Publication Year:
1996
Report Number:
RR-96-12
Source:
ETS Research Report
Document Type:
Report
Page Count:
39
Subject/Key Words:
Adaptive Testing, Computer Assisted Testing, Models, Response Style (Tests), Test Taking Behavior

Abstract

The interest in the application of large-scale computerized adaptive testing has served to focus attention on issues that arise when theoretical advances are made operational. Some of these issues stem less from changes in testing conditions and more from changes in testing paradigms. One such issue is that of the order in which questions are answered within a test or separately timed test section. In linear testing, this order of responses is entirely under the control of the test taker, who can omit questions, look ahead at questions, and return and revise answers to previous questions. The attempt to permit the same, or even reasonably restricted, control in adaptive testing can unintentionally result in transferring to the test taker control over which items are chosen for administration, threatening both test fairness and accuracy. This paper explores, using simulations, three models of permitting restricted test taker control over revising previous answers in the context of adaptive testing. Even under a worst- case model of test taker revising behavior, two of the models of permitting item revisions work well in preserving test fairness and accuracy and one model studied may also preserve some cognitive processing styles developed by test takers for a linear testing environment.

Read More