skip to main content skip to footer

Equating a Mixed-Format Test With and Without Constructed-Response Anchor Items

Author(s):
Liao, Chi-Wen; Wang, Wei; Cao, Yi
Publication Year:
2019
Report Number:
RM-19-08
Source:
ETS Research Memorandum
Document Type:
Report
Page Count:
34
Subject/Key Words:
Test Equating, Constructed-Response Items, Scoring Shift, Mixed-Format Test, Rescoring, Trend Scoring Method, Equating Bias, Multiple Choice Items, Anchor Items

Abstract

In this study, we investigated the effectiveness of different anchor designs for equating a mixed-format test that contains multiple-choice (MC) and constructed-response (CR) items. In the test, the MC score makes up most of the total test score. The criterion equating employed an MC/CR anchor with rescoring of CR anchor responses of the reference-form test takers. Using the MC/CR anchor without rescoring produced large equating error. Equating through an MC-only anchor produced smaller conditional equating biases than the MC/CR anchor without rescoring of the CR responses in the part of the score range where the majority of test takers scored, yet it still yielded an equating error greater than half a raw score point. The equating that employed only 1 scorer in the CR anchor rescoring produced similar results to the equating that employed 2 scorers; however, we observed a trend that the new-form conversion tended to be too high for low-scoring test takers and too low for high-scoring test takers.

Read More