skip to main content skip to footer

A Missing Data Approach to Estimating Distributions of Scores for Optional Test Sections

Allen, Nancy L.; Holland, Paul W.; Thayer, Dorothy T.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Missing Data, Nonresponse, Optional Items, Score Distribution, Scoring


(48pp.) Many testing programs include a section of optional questions, in addition to mandatory parts of a test. For instance, some testing programs have examinees select one of several essays or performance tasks as part of the test. These optional parts of a test are not often truly parallel to one another, making direct equating impossible. Groups of examinees selecting each optional test section are not equivalent to one another, so statistical moderation is impossible unless explicit assumptions are made. The purpose of this paper is to provide a general method based on missing- data methods for nonignorable nonresponse (Little & Rubin, 1987) to estimate distributions of scores on an optional part of a test. The distributions for each selected test part can then be linked to one another to develop comparable scales. The method is compared with other methods used to put optional test sections on comparable scales. If issues about the validity of assumption are resolved, this method could eventually provide a basis for the combination of scores based on optional portions of a test with those based on common sections of a test. This paper is the second in a series of three papers. In the first, "Approaches to Nonignorable Nonresponse with Application to Selection Bias" (Allen, Holland, & Thayer, in press, a), we apply missing-data methods from settings external to education to this setting. A method from the econometric literature (Heckman, 1976) is contrasted to methods proposed by Little and Rubin (1987) and Tukey (Holland, 1986). In this paper, the perspective of measurement specialists who proposed methods to transform scores for optional questions to comparable scales is contrasted with the approach based on the work of Little and Rubin. In the final paper, "Estimating Scores for an Optional Test Section Using Information from a Common Section" (Allen, Holland, & Thayer, in press, b), the methods of Rubin and Tukey are extended to include common scores for the mandatory part of the test. Data from an example are actually transformed to comparable scales.

Read More