skip to main content skip to footer

Automatic Scoring of Paper-and-Pencil Figural Responses CAT

Martinez, Michael E.; Ferris, John J.; Kraft, William D.; Manning, Winton H.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Computer Assisted Testing, Figural Response Items, Figural Responses, Optical Scanning, Scoring


Large-scale testing is dominated by the multiple-choice question format. Widespread use of the format is due, in part, to the ease with which multiple-choice items can be scored automatically. This paper examines automatic scoring procedures for an alternative item type: figural response. Figural response items call for the completion or modification of figural material, including illustrations, diagrams, and graphs. Twenty-five science items were written in cooperation with the National Assessment of Educational Progress (NAEP) and printed with an ink that was invisible to scanning equipment. The items were answered with pencils, response sheets were scanned, and the resulting data were processed by computer-based scoring algorithms. The paper describes the technology that leads to successful pilot scoring of seven items for ten subjects. Implications of this technology for the future of large-scale testing are discussed. (24pp.)

Read More