skip to main content skip to footer

The Optional Essay Problem and the Hypothesis of Equal Difficulty AP

Author(s):
Allen, Nancy L.; Holland, Paul W.; Thayer, Dorothy T.
Publication Year:
1993
Report Number:
RR-93-40
Source:
ETS Research Report
Document Type:
Report
Page Count:
44
Subject/Key Words:
Advanced Placement Program (AP), Difficulty Level, Equated Scores, Essay Tests, Item Choice, Scoring

Abstract

(44pp.) We examine a special case of examinee choice, the Optional Essay Problem, from the point of view of test equating. The Optional Essay Problem involves equating essay scores when the examinees are required to select an optional essay topic from a list of topics in addi- tion to taking a mandatory test required of all examinees. We derive conditions that must be satisfied if the null hypothesis of "equal difficulty" of the essays is true. (We call this Livingston's Null Hypothesis.) If this hypothesis holds, then there is no need to equate the scores on the optional essays. Our conditions take the form of inequalities about unobservable quantities that may be displayed graphically. We illustrate them using a real example from the Advanced Placement Examinations. Then we analyze Livingston's (1988) proposal for adjusting essay scores in the Optional Essay Problem and explain it from the perspective of test equating. We use our explanation to generalize his proposal to two new proposals that are explicit about the assumptions they make concerning the unobserved data. (We argue that every method for adjusting the essay scores in the Optional Essay Problem must make assumptions about unobserved data.) We illustrate the adjustment methods with an example from the Advanced Placement Examina- tions. Finally, we use the results for adjusting optional essay scores to propose comparable procedures for directly adjusting linear composite scores that include both a mandatory and an optional test score.

Read More