On Examinee Choice in Educational Testing

Author(s):
Wainer, Howard; Thissen, David
Publication Year:
1994
Report Number:
RR-94-31
GREB-91-17P
Source:
Document Type:
Subject/Key Words:
Constructed response items difficulty level equated scores item choice test validity

Abstract

This report examines some of the history and effects of examinee choice in educational testing. Questions considered include: 1) have the item choices offered been of equal difficulty? 2) what can we do to improve equating of items? 3) under what conditions and how can we equate choice items? 4) what assumptions are necessary for equating? 5) how can we test the assumptions? and 6) what can we learn from choice behavior? The article examines two strategies for fair testing when choice is allowed: 1) aiding students in making wiser choices; and 2) diminishing the unintended consequences of a poor choice through statistical equating of the choice items. Conclusions include: 1) portfolio assessments, since they can contain as many forms as there are examinees, cannot be statistically equated; 2) building examinee choice into a test is possible but requires extra work, including gathering data and evaluating responses from an unselected sample of fully motivated examinees; 3) allowing examinee choice does not achieve the goal of eliciting the examinees best performance, and therefore choice cannot fairly be allowed if the items can't be equated; and 4) "choice is anathema to standardized testing unless those aspects that characterize the choice are irrelevant to what is being tested."

Read More