skip to main content skip to footer

Equity Assessment for Polytomously Scored Items: A Taxonomy of Procedures for Assessing Differential Item Functioning DIF

Author(s):
Dorans, Neil J.; Potenza, Maria T.
Publication Year:
1994
Report Number:
RR-94-49
Source:
ETS Research Report
Document Type:
Report
Page Count:
35
Subject/Key Words:
Constructed-Response Tests, Differential Item Functioning (DIF), Performance Assessment, Scoring, Test Fairness

Abstract

Educational reform efforts have led to increased use of alternatives to the traditional binary-scored multiple choice item. Many of the stimuli employed by these alterative assessments yield complex responses that require complex scoring rules. Some of these new item types can be polytomously scored. Differential item functioning (DIF) assessment is a form of equity assessment. DIF assessment attempts to identify items for which subpopulations of examinees exhibit performance differentials that are inconsistent with the performance differentials typically seen for those subpopulations on collections of items that purport to measure a common construct. Any DIF technique can be evaluated in terms of how well it meets certain statistical and practical criteria. These DIF assessment criteria need to be attended to before we can conclude that the items associated with alternative forms of assessment can be adequately tested for DIF. DIF methodology is well-defined for traditional, binary-scored multiple choice items. This paper provides a classification scheme of DIF procedures for binary-scored items that is applicable to new DIF procedures for polytomously scored items. In the process, a formal development of a polytomous version of a binary DIF technique is presented. Finally, several polytomous DIF techniques are evaluated in terms of statistical and practical criteria. (35pp.)

Read More