skip to main content skip to footer

Examining the Relationship Between Differential Item Functioning and Item Difficulty DIF SAT

Kulick, Edward; Hu, P. Guillian
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
College Board, Differential Item Functioning (DIF), Difficulty Level, Item Analysis, Minority Groups, Scholastic Aptitude Test (SAT), Test Items


This study examined the relationship of differential item functioning (DIF) to item difficulty on the Scholastic Aptitude Test (SAT). The data comprise verbal and mathematical item statistics from nine recent administrations of the SAT. In general, item difficulty is related to DIF. The nature of that relationship appears to be independent of the choice of DIF index (either the Mantel-Haenszel or the standardization approach) as well as of test form. However, the relationship was dependent on the particular group comparison and on both the test sections and item type being analyzed. The relationship was strong for each of the racial and ethnic group contrasts—in which Black, Hispanic, and Asian American examinees were compared in turn with White examinees--but was weak for the female and male examinee contrast. The relationship also appeared stronger on the verbal sections than on the mathematical sections. The relationship is such that more difficult items tended to exhibit positive DIF (DIF favored the focal group over the White reference group). On the verbal sections, only the reading comprehension item type (with the smallest observed range in item difficulty) failed to exhibit a strong relationship. Another index, the standardized difference in percentage omit (DIFPOM), correlated very highly (negatively) with DIF. Differential omission refers to a relative difference in omit rates between groups matched in ability. In fact, DIFPOM was consistently a better predictor of DIF in most models than was item difficulty. The relationship between DIF and DIFPOM held up across all four comparisons, including gender. It was also present in the mathematical sections with nearly the same magnitude exhibited in the verbal sections. (36pp.)

Read More