skip to main content skip to footer

How Does the Knowledge of Subgroup Membership of Examinees Affect the Prediction of True Subscores? CTT

Author(s):
Haberman, Shelby J.; Sinharay, Sandip
Publication Year:
2011
Report Number:
RR-11-43
Source:
ETS Research Report
Document Type:
Report
Page Count:
31
Subject/Key Words:
Augmented Subscore, Classical Test Theory (CTT), Mean-Squared Error, Proportional Reduction in Mean-Squared Error, Reliability

Abstract

Subscores are reported for several operational assessments. Haberman (2008) suggested a method based on classical test theory to determine if the true subscore is predicted better by the corresponding subscore or the total score. Researchers are often interested in learning how different subgroups perform on subtests. Stricker (1993) and Livingston and Rupp (2004) found that the mean difference between the subgroups was not the same for the different subscores. We suggest new methods to investigate whether the quality of prediction of the true subscore improves if the investigator knows about the subgroup membership. We applied our suggested method to data from 4 operational testing programs. We found that the quality of prediction of the true subscore does not improve if the investigator knows the subgroup membership. We also found that whether the subscores have value added does not depend on the subgroups.

Read More