skip to main content skip to footer

Examination of the Test-Retest Reliability of a Forced-Choice Personality Measure FFM CAT IRT

Author(s):
Seybert, Jacob M.; Becker, Dovid
Publication Year:
2019
Report Number:
RR-19-37
Source:
ETS Research Report
Document Type:
Report
Page Count:
17
Subject/Key Words:
Personality Measures, Five-Factor Model (FFM), Test Reliability, Forced Choice Technique, Computerized Adaptive Testing (CAT), Likert Scales, Item Response Theory (IRT), Construct Validity, Test Retest Reliability

Abstract

Forced-choice (FC) measures are becoming increasingly common in the assessment of personality for high-stakes testing purposes in both educational and organizational settings. Despite this, there has been relatively little research into the reliability of scores obtained from these measures, particularly when administered as a computerized adaptive test (CAT). This study examined the test–retest reliability of an FC personality CAT, comparing its reliability to the reliability of personality measures using a Likert-type rating scale. Using a relatively large sample (N = 743), participants completed multiple personality measures across two time points. The test–retest reliability estimates for the personality dimensions had a mean of .63 with a mean reliability of .73 when formed into Big Five personality trait composites. The FC personality reliabilities were lower than those of the Likert-type scales, though the findings are within the range of those found in meta-analytic studies.

Read More