skip to main content skip to footer

Dimensions of Response Consistency in Paired Comparisons

Sadacca, Robert
Publication Year:
Report Number:
ETS Research Bulletin
Document Type:
Page Count:
Subject/Key Words:
National Science Foundation (NSF), Office of Naval Research, Attitude Measures, Individual Differences, Paired Comparisons, Preference Data, Response Style (Tests), Similarity Ratings, Test Reliability


The present study explores the structure of individual differences in self-inconsistency using as the basic measure of inconsistency the number of circular triads made in choosing between stimulus pairs presented in paired comparison schedules. Twelve paired comparison schedules were constructed--six called for judgments of similarity, six for judgments of preference. Three different kinds of stimuli were used in the schedules--colors, verbal material, and designs or figures. It was hypothesized that factor analyses of circular triad scores obtained for the 12 schedules would yield a general inconsistency factor and, possibly, additional factors reflecting the types of judgment and the kinds of stimuli involved. Three principal conclusions were drawn: 1) For subjects of the type used in this experiment, inconsistency in judging paired comparison schedules is to a considerable degree, at least, a general trait. 2) The motivation of the subjects appears to have an important influence on the amount and pattern of inconsistency and must be taken into consideration in administering paired comparison schedules and in evaluating resultant circular triad scores. 3) Longer schedules, using carefully selected stimuli, should be developed to increase the sensitivity and reliability of any inconsistency measures to be used operationally or in further research.

Read More