skip to main content skip to footer

"Thin" Versus "Thick" in the Mantel-Haenszel Procedure for Detecting DIF DIF IRT

Author(s):
Allen, Nancy L.; Donoghue, John R.
Publication Year:
1992
Report Number:
RR-92-76
Source:
ETS Research Report
Document Type:
Report
Page Count:
45
Subject/Key Words:
Differential Item Functioning (DIF), Item Response Theory (IRT), Mantel-Haenszel Technique, Models, Monte Carlo Methods

Abstract

(45pp.) This Monte Carlo study examined strategies for forming the matching variable for the Mantel-Haenszel DIF procedure; "thin" matching on total test score was compared to forms of "thick" matching, pooling levels of the matching variable. Data were generated using a three-parameter item response theory model with common guessing parameter. Number of subjects and test length were manipulated, as were the difficulty, discrimination, and presence/ absence of DIF in the studied item. Outcome measures were the transformed log-odds DIF measure statistic, its standard error, and the MH chi-square statistic. For short tests (5 or 10 items), thin matching yielded very poor results, with a tendency to falsely identify items as possessing DIF against the reference group. The best methods of thick matching yielded outcome measure values closer to the expected value for non-DIF items, and a larger value than thin matching when the studied item possessed DIF. Intermediate length tests yielded similar results for thin matching and the best methods of thick matching. The method of thick matching that performed best depended upon the measure used to detect DIF. Both difficulty and discrimination of the studied item were found to have a strong effect on the value of the DIF measure statistic.

Read More