skip to main content skip to footer

Why Matching, Analysis of Covariance, and Multiple Regression Analysis Generally Fail to Control for Preexisting Differences

O'Connor, Edward F., Jr.
Publication Year:
Report Number:
ETS Research Memorandum
Document Type:
Page Count:
Subject/Key Words:
Office of Education, Analysis of Covariance (ANCOVA), Multiple Regression Analysis, Statistical Analysis, Statistical Bias, Testing Problems


Examples are used to demonstrate algebraically and graphically that analysis of covariance and multiple regression analysis are subject to the same regression fallacy as matching and that the fundamental problem is not errors of measurement but the tendency of members of naturally occurring groups to regress toward their own means rather than toward a common mean. If two naturally occurring populations have bivariate normal distributions for the pretest and posttest scores with the same regression slope, matching, analysis of covariance, and multiple regression analysis will produce the same estimated treatment effect. Furthermore, if the populations have different pretest means, the estimated treatment effect will be the sum of the actual treatment effect and the regression artifact. This regression artifact is the result of regression toward different means.

Read More