skip to main content skip to footer

An Initial Evaluation of the Use of Bivariate Matching in DIF Analyses for Formula Scored Tests DIF SAT

Baron, Patricia A.; McHale, Frederick J.; Pomplun, Mark
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Differential Item Functioning (DIF), Formula Scores, Item Analysis, Mantel-Haenszel Technique, Scholastic Aptitude Test (SAT), Scoring


This study evaluated the use of bivariate matching as a solution to the problem of studying DIF with formula scored tests. This problem specifically involves including in or excluding from a formula scored matching criterion formula scored items to-be- studied for a DIF analysis. Using SAT Verbal data with large and small samples both Male-Female and Black-White group comparisons were investigated. MH D-DIF values and DIF category classifications based on bivariate matching were compared with MH D-DIF values and categories based on rights scored and formula scored matching criteria. When large samples were used, MH D-DIF values based on the bivariate matching criterion were ordered very similarly to MH D-DIF values based on the other criteria. The DIF category classifications were almost identical. However, with small samples the MH D-DIF values based on the bivariate matching criterion displayed only moderate correlations with MH D-DIF values from the other criteria. In addition, the DIF category classifications based on the bivariate matching criterion showed fewer high DIF items than those based on the rights or formula scored matching criteria. As a secondary result, this study documented the differences between formula and rights scored criteria in DIF analyses of formula scored tests. These results showed that the substi-tution of a rights scored criterion in a DIF analysis of a formula scored test resulted in MH D-DIF values that were ordered very similarly as those based on the proper formula scored criterion. The MH D-DIF values based on the rights score criterion were, however, different in magnitude from those based on the formula scored criterion. The differences were related to item difficulty and were greater for comparisons with larger ability differences. (73pp.)

Read More