Errors of Measurement, Theory, and Public Policy

Author(s):
Kane, Michael
Publication Year:
2010
Report Number:
PIC-ANG12
Source:
Document Type:
Subject/Key Words:
Constructs errors of measurement random errors systematic errors validity generalizability theory classical test theory controlling errors of measurement standardized testing error tolerance ratios social consequences validity

Abstract

Errors of measurement arise because our observations are affected by many sources of variability, but our conceptual frameworks necessarily ignore much of this variability. Sources of variability that are not included in our models and descriptions of phenomena are treated as error or noise. A good theory of error supports the development of precise measurements, clearly defined constructs and sound public policy. Narrowly defined constructs that do not generalize much beyond the observed performances do not involve many sources of error, but constructs that generalize observed scores over a broad range of conditions of observation (e.g., context, time, test tasks) necessarily involve many potential sources of error. We can have narrow constructs with small errors or more broadly defined constructs with larger errors. Some errors that are negligible for individuals can have a substantial impact on estimates of group performance, and therefore, can have serious consequences. The 12th annual William H. Angoff Memorial Lecture was given at ETS in Princeton, NJ, on November 18, 2008. While supplies last, print copies can be ordered from the Policy Information Center for $15.00.

Read More