skip to main content skip to footer

Single Score Versus Multiple Score Reading of the American History Advanced Placement Examination

Coffman, William E.; Kurfman, Dana G.
Publication Year:
Report Number:
RB-66-22, RDR-65-66, No. 14
ETS Research Bulletin
Document Type:
Page Count:
Subject/Key Words:
College Board, Advanced Placement, Advanced Placement Examinations, American History Tests, College Entrance Examinations, Essay Tests, Holistic Evaluation, Interrater Reliability, Scoring


Four readers assigned to read a single essay question of the Advanced Placement Examination in American History in 1962 participated in two experiments designed to assess the differential effects of reading by an analytical method by which three different scores were assigned to a paper and by a holistic method by which a single score was assigned. Because of limited time and the small number of readers, the two experiments were conducted in an interlocking design which permitted the findings of one experiment to sharpen the interpretation of the effects observed in the other. Analyses of variance procedures revealed significant effects attributable to differences in standards among the four readers, to differences in standards from day to day, and to fluctuations in standards of individual readers from time to time during the reading period. There was no definite evidence of differences in reading reliability from method to method.

Read More