Why Matching, Analysis of Covariance, and Multiple Regression Analysis Generally Fail to Control for Preexisting Differences

Author(s):

O'Connor, Edward F., Jr.

Publication Year:

1973

Report Number:

RM-73-19

Source:

ETS Research Memorandum

Document Type:

Report

Page Count:

24

Subject/Key Words:

Office of Education, Analysis of Covariance (ANCOVA), Multiple Regression Analysis, Statistical Analysis, Statistical Bias, Testing Problems

Abstract

Examples are used to demonstrate algebraically and graphically that analysis of covariance and multiple regression analysis are subject to the same regression fallacy as matching and that the fundamental problem is not errors of measurement but the tendency of members of naturally occurring groups to regress toward their own means rather than toward a common mean. If two naturally occurring populations have bivariate normal distributions for the pretest and posttest scores with the same regression slope, matching, analysis of covariance, and multiple regression analysis will produce the same estimated treatment effect. Furthermore, if the populations have different pretest means, the estimated treatment effect will be the sum of the actual treatment effect and the regression artifact. This regression artifact is the result of regression toward different means.

Read More

• Request Copy (specify title and report number, if any)