Multiple-Choice Cloze Items and the TOEFL® Test

Author(s):
Hale, Gordon A.; Stansfield, Charles W.; Rock, Donald A.; Hicks, Marilyn M.; Oller, John W., Jr.; Butler, Frances A.
Publication Year:
1988
Report Number:
RR-88-02, TOEFL-RR-26
Source:
Language Testing, 1989, 6(1), 47-76.
Document Type:
Subject/Key Words:
Cloze procedure English as a second language (ESL) factor analysis test construction test reliability

Abstract

This study examined the relation of TOEFL® performance to a widely used variant of the cloze procedure--the multiple-choice (MC) cloze method. A main objective was to determine if categories of MC cloze items could be identified that related differentially to the various parts of the TOEFL test. Examinees taking an operational TOEFL test at domestic test centers were given the three basic sections of the test along with a fourth section containing the MC cloze items. Factor analyses suggested that, from a practical standpoint, TOEFL performance can be adequately described by just two factors, which relate to (a) Listening Comprehension, and (b) all other parts of the test--Structure, Written Expression, Vocabulary, and Reading Comprehension. Examination of the MC cloze test showed that the total MC cloze score was relatively reliable and that it was possible to estimate item response theory parameters for the MC cloze items with reasonable accuracy. However, the correlations among scores for the four MC cloze item categories were approximately as high as their reliabilities, thus providing no strong empirical evidence that the item types within the MC cloze test reflected distinct skills.

Read More