skip to main content skip to footer

An Investigation of the Effect of Difficulty and Chance Success on Item-Test Correlation and on Test Reliability (Abstract of Dissertation)

Author(s):
Plumlee, Lynnette B.
Publication Year:
1950
Report Number:
RB-50-29
Source:
ETS Research Bulletin
Document Type:
Report
Page Count:
14
Subject/Key Words:
Chance Level Scores, Difficulty Level, Guessing (Tests), Item Analysis, Multiple Choice Tests, Test Items, Test Reliability

Abstract

The purpose of this thesis is to consider the effects of chance success on item difficulty, item-test correlation, and test reliability. Equations were derived for expressing the indicated statistics for a multiple choice test as a function of the corresponding statistics of the some answer-only test. The methodology in coming up with the equations was similar to Carroll's method. A series of parallel mathematics achievement tests was established. The same items were used in multiple-choice and answer only form. The tests were given to a population sample of about 560 males of college age. The four multiple-choice reliability values that were obtained (one for each test) tended to fall between the obtained answer-only reliabilities and the multiple-choice reliabilities predicted for equation from the obtained answer-only reliabilities. Equations were developed which determine the expected effect of chance success relative to item difficulty on item-test correlation and on test reliability when answer options are supplied. The effect on item difficulty in supplying answer choices for mathematics test items corresponded to the expected effect. The effect on item-test correlation and test reliability was in the direction expected but tended not to be as large as expected. (SGK).

Read More