skip to main content skip to footer

Effects of Directions Regarding Guessing on Item Statistics of a Multiple-Choice Vocabulary Test

Miller, Peter M.; Swineford, Frances
Publication Year:
Report Number:
ETS Research Bulletin
Document Type:
Page Count:
Subject/Key Words:
Difficulty Level, Guessing (Tests), Item Analysis, Multiple Choice Tests, Test Instructions, Test Items, Vocabulary Tests


A 100-item vocabulary test containing 80 words of normal difficulty, ten extremely difficult words, and ten nonexistent "words" was administered to a group of 801 Annapolis first-year students. There were three sets of directions, one third of the population being told nothing about guessing, one third being warned against guessing, and one third being encouraged to guess. Directions not to guess were found to have the most direct effect on the number of questions attempted; students who were told nothing about guessing acted very similarly to students who were encouraged to guess. The tendency to guess proved to have an almost negligible positive relation to ability in the area of the test. The reliability of the test was lowered by the presence of the special items. Individual item difficulties were affected by the directions, deltas being higher where there was least guessing. Biserial correlations of the ordinary items were adversely affected, but to a slight extent only, by the presence of the special items. The very difficult items were found to react in exactly the same way as the nonsense items.

Read More