skip to main content skip to footer

Analyzing the Option Effects of Difficult TOEFL Items With Low Biserials: Methods Developed for Use by Test Assemblers TOEFL

Author(s):
Hicks, Marilyn M.
Publication Year:
1988
Report Number:
RR-88-15
Source:
ETS Research Report
Document Type:
Report
Page Count:
41
Subject/Key Words:
Item Analysis, Test Construction, Test Difficulty, Test of English as a Foreign Language (TOEFL)

Abstract

Several exploratory analyses of the fifths data generated by TOEFL item analyses were developed in order to evaluate the effects of options on the discriminability of difficult items, and to identify difficult items with low, unreliable biserials which have been rejected by Test Development but for which acceptable a-parameters are probably estimable. If item writers could account for the factors underlying the interaction between ability level and option responses, it might be possible to modify options accordingly, thereby improving the measurement effectiveness of the item. Departing from the usual reliance on a single index, the approaches in these analyses included, among other things, an evaluation of the biplot generated from a correspondence analysis of the matrix of fifths information, and an analysis of the total option response configuration. Many examples of these analyses were provided. A significant limitation of the r-biserial for very difficult items which restricts the ability of test assemblers to construct tests with effective measurement properties at high score levels was illustrated. The index developed in this study to identify such items is regarded as an interim strategy until a conventional measure of item discrimination which is optimal over the entire scale of difficulty is developed, a current critical need. (41pp.)

Read More