skip to main content skip to footer

Statistical Approaches to the Study of Item Difficulty

Author(s):
Olson, John F.; Scheuneman, Janice Dowd; Grima, Angela
Publication Year:
1989
Report Number:
RR-89-21
Source:
ETS Research Report
Document Type:
Report
Page Count:
33
Subject/Key Words:
Difficulty Level, Item Analysis, Statistical Analysis, Test Analysis

Abstract

Traditionally, item difficulty has been defined in terms of the performance of examinees. For test development purposes, a more useful concept would be some kind of intrinsic item difficulty, defined in terms of the item's content, context, or characteristics and the task demands set by the item. To the extent that we can come to understand more fully the intrinsic difficulty of items, we can also begin to understand better the functioning of test items and to bring that functioning increasingly under control. An important step in developing the knowledge base required to acquire an understanding of those item properties that affect difficulty is appropriate analyses of existing test data. In this investigation, the measurement literature was surveyed for statistical approaches which might be fruitfully applied to the study of item difficulty. Two broad methodological approaches were identified: exploratory and confirmatory approaches. Exploratory methods were those that attempt to categorize or cluster items that appear to measure similar abilities and that function in a similar manner in order to determine their common characteristics as well as those that differentiate them from other items not in the cluster. Confirmatory methods would be applied to test hypotheses developed from exploratory results or from psychological theory. Described in the final section of the paper are the results of analyses using real test data that assessed the usefulness of two of the exploratory methods. (33pp.)

Read More