The Prediction of TOEFL® Listening Comprehension Item Difficulty for Minitalk Passages: Implications for Construct Validity

Author(s):
Freedle, Roy O.; Kostin, Irene W.
Publication Year:
1996
Report Number:
RR-96-29
TOEFL-RR-56
Source:
Document Type:
Subject/Key Words:
Construct validity difficulty level listening comprehension prediction

Abstract

The purpose of the current study was to predict the difficulty of a large sample (n = 337) of the TOEFL® test (Test of English as a Foreign Language™) listening comprehension items that dealt with the minitalk (listening) passages. Four item types were examined: Main Idea items (consisting of two subtypes: explicit and implicit gist), Supporting Idea items (also called explicit detail items), and two types of Inference items (one subtype called pure inference items and a second subtype called inference-application items--both of these subtypes are also called implicit detail items). A related purpose was to examine whether particular types of predictors (i.e., text and text- associated variables) play a significant role in predicting item difficulty. We maintain that evidence favoring construct validity in part requires significant contributions from these text and text- associated predictor variables. This paper also explores the hypothesis that multiple-choice listening comprehension tests are sensitive to many sentential and discourse variables found to influence comprehension processes in the experimental language comprehension literature. We demonstrate the following: (1) listening as well as reading item difficulty can be significantly predicted by variables similar to those reported in the experimental literature on language comprehension, (2) some similarities and differences between listening and reading comprehension abilities can be studied using the methodology employed in the current study, and (3) the TOEFL listening (minitalk) items examined here appear to be construct valid in the restricted sense that we have demonstrated that variables that code for features of the whole passage and selected aspects of the passage significantly influenced item difficulty; this pattern of results implies, we believe, that examinees were significantly influenced by relevant listening information presented in the passage and were actively attempting to comprehend the listening materials.