skip to main content skip to footer

Three Practical Issues for Modern Adaptive Testing Item Pools

Author(s):
Stocking, Martha L.
Publication Year:
1994
Report Number:
RR-94-05
Source:
ETS Research Report
Document Type:
Report
Page Count:
40
Subject/Key Words:
Adaptive Testing, Computer Assisted Testing, Item Pools, Test Construction, Test Items, Test Security

Abstract

(40pp.) As adaptive testing moves towards operational implementation in large scale testing programs, where it is important that adaptive tests be as parallel as possible to existing linear tests, a number of practical issues arise. This paper concerns three such issues. First, optimum item pool size is difficult to determine in advance of pool construction. Retrospective results are analyzed for five operational pools; these analyses indicate that item pools of the size and quality of six to eight linear tests are adequate to support adaptive tests of roughly half the length of a parallel linear test. Second, item pools may not support sufficiently low exposure rates for items or a sufficiently small amount of test overlap to maintain test security when testing is conducted on a continuous basis. Various simulations suggest that multiple pools which can be chosen randomly before adaptive testing begins provide a satisfactory solution. Finally, over time it will be necessary to refresh or replace operational item pools. Issues that must be considered in refreshing or replacing item pools are discussed, and guidance is given for establishing benchmark values from initial pools as standards that must be met for adaptive tests from new pools to be considered parallel to adaptive tests from existing pools.

Read More