skip to main content skip to footer

Estimating Reliability for Tests Composed of Item Sets SAT

Lawrence, Ida M.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Difficulty Level, Item Analysis, Item Formats, Multiple Choice Tests, Scholastic Aptitude Test (SAT), Scholastic Assessment Test, Test Content, Test Items, Test Length, Test Reliability


To what extent, if any, are estimates of reliability for a multiple choice test affected by the presence of large item sets where each set shares common reading material? The purpose of this research was to assess the effect of local item dependence on estimates of reliability for verbal portions of the old and new SAT, where the new test contains larger item sets associated with reading passages. Estimates based on a single administration of the test (estimates based on internal consistency and estimates based on covariances among parts) were compared to estimates based on two administrations of the test. When adjusted for a fixed standard deviation, estimates based on covariances among parts tended to be similar to estimates based on parallel forms. Both types of estimates were lower than the internal consistency estimates. (27pp.)

Read More