skip to main content skip to footer

Measurement Error and Bias in Value-Added Models VAM

Author(s):
Kane, Michael
Publication Year:
2017
Report Number:
RR-17-25
Source:
ETS Research Report
Document Type:
Report
Page Count:
14
Subject/Key Words:
Error of Measurement, Score Bias, Value Added Models (VAM), Residual Variance, Validity, Generalizability, Teacher Effectiveness, School Effectiveness, Student Growth

Abstract

By aggregating residual gain scores (the differences between each student’s current score and a predicted score based on prior performance) for a school or a teacher, value-added models (VAMs) can be used to generate estimates of school or teacher effects. It is known that random errors in the prior scores will introduce bias into predictions of the current scores, and thereby, into the estimated residual gain scores and VAM scores. The analyses in this paper examine the origins of this bias and its potential impact and indicate that the bias is an increasing linear function of the student’s prior achievement and can be quite large (e.g., half a true-score standard deviation) for very low-scoring and high-scoring students. To the extent that students with relatively low or high prior scores are clustered in particular classes and schools, the student-level bias will tend to generate bias in VAM estimates of teacher and school effects. Adjusting for this bias is possible, but it requires estimates of generalizability (or reliability) coefficients that are more accurate and precise than those that are generally available for standardized achievement tests.

Read More