skip to main content skip to footer

Keylist Items for the Measurement of Verbal Aptitude

Ward, William C.; Dworkin, Dan; Carlson, Sybil B.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Aptitude Tests, Keylist Tests, Test Construction, Test Items, Verbal Ability


Two verbal item types employed in standardized aptitude tests were administered in a conventional multiple-choice format and in the keylist format, in which the examinee is required to think of an answer, look it up in a long ordered list, and write its number. The keylist format provides a machine-scorable surrogate for a truly free-response test. Its potential attractions include the increased acceptability of items given in a production rather than a recognition format, resistance to coaching based on "gaming" strategies for eliminating multiple-choice alternatives, and elimination of the need in item writing to produce distractors for an item. Analyses indicated that the format has little or no systematic effect on the construct validity of tests employing item types used in standardized tests of verbal aptitude. One of the purposes of the study was to determine the degree to which experienced test developers could agree on the set of keys that should be supplied for each keylist item. Agreement among reviewers was far from the near-perfect consensus that would be required for use of this format, perhaps because the two item types investigated, Antonyms and Analogies, represent tests dealing with word meanings taken largely out of context. (42pp.)

Read More