skip to main content skip to footer

An Analysis of Factors Affecting the Difficulty of Dialogue Items in TOEFL Listening Comprehension TOEFL

DeVincenzi, Felicia; Nissan, Susan; Tang, K. Linda
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Difficulty Level, English (Second Language), Factor Analysis, Item Pools, Listening Comprehension, Test Items, Test of English as a Foreign Language (TOEFL)


One of the item types in the Listening Comprehension Section of the TOEFL® test is the Dialogue. Because the Dialogue item pool needs to have an appropriate balance of items at a range of difficulty levels, test developers have examined items at various difficulty levels in an attempt to identify their features. In this study, the authors created a classification system for certain item features, classified a sample of the current Dialogue item pool, and conducted data analyses in an attempt to characterize the features of easy and difficult Dialogue items. The results of the analyses indicate that, of the features studied, five were significant: the presence of infrequent oral vocabulary, the sentence pattern of the utterances in the stimulus, the presence of negatives in the stimulus, the necessity of making an inference to answer the item, and the roles of the speakers in the stimulus.

Read More