Exploration of Causes of Bias in Test Items

Scheuneman, Janice Dowd
Publication Year:
Report Number:
ED268157 Journal of Educational Measurement, 1987, 24(2), 97-118.
Document Type:
Subject/Key Words:
African American test takers item analysis performance factors test bias test construction Whites test takers


A number of hypotheses were developed concerning elements of test items that might affect the performance of Black and White test takers differently. The elements investigated were characteristics that would be common to several items that otherwise measured different concepts or content. Seven general hypotheses were developed and tested in the form of 16 specific hypotheses, most with five or six items. For purposes of this study, items were developed in pairs, with a hypothesized element present in one item and missing or modified in the other. The item pairs were assembled into two separate sections and administered as part of the Graduate Record Examinations General Test. Pairs of test sections were developed for each of the three areas tested in the General Test: verbal, quantitative, and analytical ability. These sections were spiraled for administration to yield randomly equivalent groups of examinees. Ten of the sixteen hypotheses showed interactions between group membership and item version indicating a differential effect of the manipulation on the performance of Black and White examinees. Although item manipulations did have a differential effect on the performance of Black and White examinees, the complexity of the effects suggested that other uncontrolled factors affecting performance were also operating.

Read More


Find a Publication

Advanced Search

Closing the Achievement Gap

Closing the Achievement GapLearn more about ETS's commitment to closing the achievement gap through rigorous research, thought-provoking forums and more.