skip to main content skip to footer

Statistical Procedures to Evaluate Quality of Scale Anchoring

Author(s):
Haberman, Shelby J.; Sinharay, Sandip; Lee, Yi-Hsuan
Publication Year:
2011
Report Number:
RR-11-02
Source:
ETS Research Report
Document Type:
Report
Page Count:
20
Subject/Key Words:
Feedback, Writing Development, Meta-Analysis, Commenting, Error Analysis

Abstract

Providing information to test takers and test score users about the abilities of test takers at different score levels has been a persistent problem in educational and psychological measurement (Carroll, 1993). Scale anchoring (Beaton & Allen, 1992), a technique that describes what students at different points on a score scale know and can do, is a tool to provide such information. Scale anchoring for a test involves substantial amount of work, both by the statistical analysts and test developers involved with the test. In addition, scale anchoring involves considerable use of subjective judgment, so its conclusions may be questionable. This paper describes statistical procedures that can be used to determine if scale anchoring is likely to be successful for a test. If these procedures indicate that scale anchoring is unlikely to be successful, then there is little reason to perform a detailed scale anchoring study. The procedures are applied to several data sets from a teacher licensing test.

Read More