skip to main content skip to footer

Group Approach to the Analysis of Individual Differences in the Randomness of Guessing Behavior on Multiple-Choice Tests and the Development of Scoring Methods to Take Such Difference into Account SAT

Author(s):
Price, Dorothy Bird
Publication Year:
1964
Report Number:
RB-64-59, RDR-64-65, No. 10
Source:
ETS Research Bulletin
Document Type:
Report
Page Count:
123
Subject/Key Words:
College Board, Guessing (Tests), Multiple Choice Tests, Scholastic Aptitude Test (SAT), Scoring Formulas, Test Theory

Abstract

The major purpose of this investigation was to see if groups of examinees, with similar ratios of right to wrong answers, differ sufficiently with respect to an analysis of the mean number of effective distractors (E) (a measure of the degree to which the number of item options were narrowed before guessing) that the E results for each group may be inferred to the individuals within that group. The second purpose was to develop and study characteristics of scoring methods that take into account individual differences in narrowing behavior. The data were sampled from two recent administrations of the Scholastic Aptitude Test--two verbal and two mathematical sections totaling 12,140 examinees. Statistically significant variance analyses for all sections supported the conclusion that groups differentiated on the ratio of right to wrong answers (Ratio Groups), came from populations with different mean E values and that narrowing behavior increases as the right to wrong ratio increases. From Ratio Group variance homogeneity tests it was concluded that as the ratio increases, item differences in susceptibility to narrowing increase. Among other findings, it was concluded that the relationship between item difficulty and number of effective distractors is very low.

Read More