skip to main content skip to footer

Approaches to Nonignorable Nonresponse With Application to Selection Bias

Allen, Nancy L.; Holland, Paul W.; Thayer, Dorothy T.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Bias, Essay Tests, Nonresponse, Optional Items, Score Distribution


(43pp.) The main purpose of this paper is to illuminate different approaches to nonignorable nonresponse. Previously, literature in educational settings has emphasized bias due to using techniques that are appropriate in the case of ignorable nonresponse when nonignorable nonresponse has actually occurred. The literature in biometrics, econometrics, and statistics contains work that is more specifically applicable in this setting. Methods introduced by Pearson, Rubin, Heckman, and Tukey are described and contrasted for two different cases. The two cases include: (1) the simple case where there is one variable of interest and the sampling rule depends on that variable, and (2) the more complicated case where there is one variable of interest and a covariate. In this second case, as in the first, the sampling rule depends on the variable of interest. Finally, the methods, as applied in case 1, are used in an example. The goal of the example is to estimate the total distribution of scores for an essay, when the essay is answered by only some of the test takers. Data from an actual test administration where examinees were given an optional essay question were used.

Read More