skip to main content skip to footer

Standard Setting Panelist Cognition: A Framework and Implications for Practice

Katz, Irvin R.
Publication Year:
Report Number:
ETS Research Memorandum
Document Type:
Page Count:
Subject/Key Words:
Angoff Method, Standard Setting, Cognition, Educational Assessment, Cognitive Biases, Cut Scores, Probability Models, Analytical Reasoning, Cognitive Psychology, Cognitive Models, Decision Making, Judgment


Standard setting plays a significant role in the validity argument for assessments. Standard setting practices at Educational Testing Service (ETS) have evolved over time, motivated by practice and empirical research from the standard setting literature. However, few efforts in the literature have been directed toward understanding the drivers of standard setting decisions. By reviewing empirical research that might inform how experts make standard setting judgments— including cognitive psychology research rarely considered in the development of standard setting practice—this paper seeks both to inform current ETS practices and suggest research studies.

Read More