skip to main content skip to footer

Standard Setting Panelist Cognition: A Framework and Implications for Practice

Author(s):
Katz, Irvin R.
Publication Year:
2019
Report Number:
RM-19-12
Source:
ETS Research Memorandum
Document Type:
Report
Page Count:
23
Subject/Key Words:
Angoff Method, Standard Setting, Cognition, Educational Assessment, Cognitive Biases, Cut Scores, Probability Models, Analytical Reasoning, Cognitive Psychology, Cognitive Models, Decision Making, Judgment

Abstract

Standard setting plays a significant role in the validity argument for assessments. Standard setting practices at Educational Testing Service (ETS) have evolved over time, motivated by practice and empirical research from the standard setting literature. However, few efforts in the literature have been directed toward understanding the drivers of standard setting decisions. By reviewing empirical research that might inform how experts make standard setting judgments— including cognitive psychology research rarely considered in the development of standard setting practice—this paper seeks both to inform current ETS practices and suggest research studies.

Read More