skip to main content skip to footer

Human Judgments Considered as Fallible Measures

Author(s):
Lord, Frederic M.
Publication Year:
1962
Report Number:
RM-62-08
Source:
ETS Research Memorandum
Document Type:
Report
Page Count:
22
Subject/Key Words:
Error of Measurement, Personnel Selection, True Scores, Value Judgment

Abstract

Human judgment which contains an error is called fallible. The difference between a quantitative judgment and its error will be referred to as the true value. An outstanding characteristic of mental test scores, like human judgment, is their fallibility. So, mental test theory is, to a considerable extent, an extension of the classical theory of errors, with particular emphasis on the case where many different objects and many different true values are considered simultaneously. For this reason, mental test theory may help to throw some light on the effects of using fallible human judgments in place of the corresponding unknown true values. This paper considers the simplest case, in which the errors of measurement are drawn from frequency distributions with zero mean and experimentally estimable variance, and are statistically independent of each other and of the true values. When a set of fallible human judgments are substituted for the corresponding true values, certain results occur. In some cases, the fact that a variable is fallible changes the appropriate statistical analysis. Illustrations are given. Situations in which optimization is important are also discussed: simple selection, multiple cutting scores, differences between judgments, and linear programming problems. Paper presented at symposium on "Human Judgments and Optimality," Meeting of the American Psychological Association, September 4, 1962.

Read More