angle-up angle-right angle-down angle-left close user menu open menu closed search globe bars phone store

Effects of Printed Option Sets on Listening Item Performance Among Young English-as-a-Foreign-Language Learners

Author(s):
Getman, Edward; Cho, Yeonsuk; Luce, Christine
Publication Year:
2016
Report Number:
RM-16-16
Source:
ETS Research Memorandum
Document Type:
Report
Page Count:
23
Subject/Key Words:
Children English Language Learners (ELL) Generalized Linear Mixed Model Listening Tests Printed Materials Test of English as a Foreign Language (TOEFL) TOEFL Primary

Abstract

The TOEFL Primary tests from Educational Testing Service (ETS) are intended to assess the English language proficiency of young students 8 years of age and older who are learning English as a foreign language (EFL). Separate tests assess listening, reading, and speaking skills. The TOEFL Primary Listening test includes selected-response items in which the option sets are both printed in a test book and read aloud. Concern was raised by test administrators that presenting the options in print might unduly impact young test takers, especially those whose reading ability in English is low, as it may require the ability to process both visual and auditory information concurrently. The current study investigated the effects of the presence of printed option sets on TOEFL Primary Listening test performance. In a sample of 747 test takers from Mongolia, Colombia, and Brazil, we examined whether any difference in performance on listening items existed when the option sets were presented both aurally and visually versus being presented aurally only. The participants took one of four experimental test forms consisting of (a) a reading section, (b) a listening section with options sets presented in both aural and visual modes, and (c) a second listening section with option sets presented in an aural mode only. The listening test items were counterbalanced across test forms to control for order effects. Participants also completed a brief survey on their perceptions about the relative difficulty of listening items under the two conditions. A repeated-measure general linear model (GLM) was employed to compare performance of listening items under both conditions. Results indicated that the presence of printed text options did not make items more difficult for test takers. In addition, test takers perceived that the presence of text options made the listening items easier, contrary to the concern that motivated the current study.

Read More