skip to main content skip to footer

Distractor Method Study

Riegel, Eleanor
Publication Year:
Report Number:
ETS Research Bulletin
Document Type:
Page Count:
Subject/Key Words:
College Entrance Examinations, Distractors (Tests), Item Analysis, Mathematics Tests, Multiple Choice Tests, Scholastic Aptitude Test (SAT), Test Construction


A study was done on the January 1950 Scholastic Aptitude Test (SAT) experimentals in order to study the best type of distractors to use in mathematics aptitude items. Specifically, the study evaluated the relative merits of: 1) distractors based on wrong solution methods; 2) distractors arbitrarily selected; and 3) distractors obtained by combining two of the most obvious from the first and two of the most attractive from the second. Twelve experimental sections of 20 items each were used. First, four distinct sets of 20 items were picked. Two sets were tried items, the other two, untried. Each of the 12 forms was analyzed with the regular SAT math score as a criterion. The r, delta, and p were calculated, for each item, and the mean raw scores and the standard deviations of raw score distributions were calculated. Table I shows what type of distractor appeared in each form, the base N for each, and the number trying the last item. Table II shows the results of the t test of significance applied to differences between item and test statistics. The items with arbitrary distractors are easier and as reliable as those with distractors based on wrong solution methods or those with mixed distractors. The mean raw score and the standard deviation of a test do not change significantly when arbitrary distractors are substituted for distractors based on methods. Arbitrary distractors can be used exclusively in math aptitude items. (SGK).

Read More