skip to main content skip to footer

Examination of the Added Value of the Scores on Two Sections of an Admissions Test

Author(s):
Sinharay, Sandip; Haberman, Shelby J.; Melendez, Carlos R.
Publication Year:
2012
Report Number:
RM-12-21
Source:
ETS Research Memorandum
Document Type:
Report
Page Count:
22
Subject/Key Words:
KR-20, Proportional Reduction in Mean-Squared Error, Reliability

Abstract

Haberman suggested a method based on classical test theory to examine whether the subscores (or section scores) of a test provide any added value over the total score. Haberman also suggested augmented section scores that provide more accurate diagnostic information than the section scores themselves. We used the method of Haberman to examine whether it is justified to report two separate scores for the two sections of an admissions test that is administered to examinees in several countries. We also examined whether augmented section scores have added value for the test. We analyzed data from several administrations of the test in two countries and examined if there is a country effect in the results of the method of Haberman. The results indicate that it is justified to report one section score for examinees from one country, even without augmentation. However, it is not justified to report the same section score for examinees from the other country, even after augmentation. In addition, it is not justified to report the other section score for either country, even after augmentation. The results indicate that there is a country effect in the results of the method of Haberman for these data; that is, the application of the method of Haberman leads to substantially different conclusions for the two countries.

Read More