skip to main content skip to footer

A Score Checking Machine

Fan, Chung-Teh; Lord, Frederic M.; Tucker, Ledyard R
Publication Year:
Report Number:
ETS Research Memorandum
Document Type:
Page Count:
Subject/Key Words:
Quality Assurance, Test Reliability, Test Scoring Machines, Testing Problems


A system for reducing the number of large errors in machine-scoring of multiple choice answer sheets is described. The idea was conceived that if two scoring machines could be made of differing sensitivity, it would be unlikely for them to make the same errors. This proposition would be feasible provided that the less sensitive machine would still score a large proportion of well marked answer sheets correctly. Answer sheets for which the two machines gave different scores would then be carefully hand scored. Economy of operation could then be obtained by reducing the amount of hand scoring from 100% to about 10 to 15%. Two scoring machines were altered to be less sensitive to pencil marks by having two of the five scoring blades removed and the remaining blades spread to cover the same space. Then experiments were performed on three samples of answer sheets using both a highly-sensitive and a lowly-sensitive scoring machine. It is concluded that "a method of scoring using the two types of machines is feasible." The operational steps recommended for such a scoring are 1) scan the answer sheets for obvious mismarkings and hand-score those sheets; 2) score the answer sheets on the highly-sensitive machine first and then on the lowly-sensitive machine; 3) sort out the answer sheets for which the two scores disagree and hand-score; and 4) check that for none of the large errors the lowly-sensitive machine scored higher than the highly- sensitive machine; inspect the adjustment and condition of both machines if such papers are found.

Read More