skip to main content skip to footer

A Generalization of Adjustment, With an Application to the Scaling of Essay Scores

Rosenbaum, Paul R.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Advanced Placement Examinations, Biology Tests, Equated Scores, Essay Tests, Observational Studies, Psychometrics, Scaling, Statistical Analysis


In a subclassified finite population, the sample mean and the estimator based on direct adjustment are the two extreme representatives of a family of empirically weighted estimators derived from different logit models for selection probabilities or propensity scores. Other members of this family often have several advantages: (a) unlike the sample mean, they permit adjustment for subclasses, (b) unlike the directly adjusted estimator, they can often be calculated when there are no respondents in some subclasses, and (c) the weights assigned to the subclasses are less variable than the weights associated with the directly adjusted estimator. In conjunction with subclasses formed from an estimated balancing score, the empirically weighted estimator is applied to scale the scores of self-selected essays on the 1982 College Board® Advanced Placement Program® Examination in Biology.

Read More