skip to main content skip to footer

Topic and Topic Type Comparability on the Test of Written English TOEFL TWE

Author(s):
Golub-Smith, Marna L.; Reese, Clyde M.; Steinhaus, Karin S.
Publication Year:
1993
Report Number:
RR-93-10
Source:
ETS Research Report
Document Type:
Report
Page Count:
36
Subject/Key Words:
English Tests, Essay Tests, Test of English as a Foreign Language (TOEFL), Test of Written English (TWE)

Abstract

The Test of Written English™ (TWE®), administered with certain designated TOEFL® examinations, consists of a single essay prompt to which examinees have 30 minutes to respond. It was introduced in 1986 to provide TOEFL score users with a direct measure of examinees' writing ability. Preliminary studies had indicated that the two different kinds of prompts: prose compare, contrast, and take a position, and describe or interpret a chart or graph, elicited comparable writing performance. However, questions were subsequently raised with respect to continued comparability of different TWE prompts administered under operational conditions. The present study was designed to elicit essays for prompts that differed in both subject matter (Topic) and in the level of explicitness with which the essay task was presented (Topic Type). Eight different prompts were spiraled worldwide at the October 1989 TOEFL administration, with each prompt eliciting approximately 10,000 essays. The results of the analyses indicated that there were small differences among the prompts. The most notable differences were obtained among the scores for topics using the explicit comparison. Across all the prompts, the chart-graph with the explicit comparison statement produced the highest mean scores. Because it was the first study of its kind to focus on the comparability of prompts in a major testing program, the authors had difficulty making definitive statements regarding the meaningfulness of the obtained differences. While many of the differences in means observed in this study were so small as to be of no practical significance, differenc- es observed across prompts in the numbers of examinees at each score level were not. Such differences may warrant further consideration by the TOEFL program.

Read More