skip to main content skip to footer

Clusters as the Unit of Analysis in Differential Item Functioning DIF SAT

Bennett, Randy Elliot; Rock, Donald A.; Novatkoski, Inge
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Black Students, Blind Students, Differential Item Functioning (DIF), Item Analysis, Scholastic Aptitude Test (SAT), Test Bias, Test Theory


This study developed, applied, and evaluated a theory-based method of detecting the underlying causes of differential difficulty. The method was intended to improve on traditional approaches that too often produce uninterpretable results. Key elements of the method were the analysis of item clusters and the incorporation of theoretical predictions about cluster performance. The method was applied in two subgroups taking SAT-M and involved (1) reviewing literature syntheses to identify factors that might cause differential item functioning, (2) forming item categories based on those factors, (3) identifying categories that functioned differentially, (4) assessing the functioning of the items composing deviant categories, and (5) relating item and category functioning. In both subgroups, the cluster and traditional item level analysis methods agreed on the overall extent of differential functioning (substantial in the first group, virtually absent in the second). Additionally, the pattern of differential functioning detected was interpretable. The method would seem to be applied most productively when a small number of hypotheses can be derived from a reasonably strong research base, overlap among cluster structures can be avoided, and results can be supplemented with experimental studies or protocol analyses. (53pp.)

Read More