An Investigation of the Impact of Composition Medium on the Quality of Scores From the TOEFL® Writing Section: A Report From the Broad-Based Study

Author(s):
Wolfe, Edward. W.; Manalo, Jonathan. R.
Publication Year:
2005
Report Number:
RR-04-29
TOEFL-RR-72
Source:
Document Type:
Subject/Key Words:
Composition medium computer-based testing direct writing assessment English proficiency English as a foreign language English as a second language TOEFL writing test

Abstract

This study examined scores from 133,906 operationally scored TOEFL® test (Test of English as a Foreign Language™) essays to determine whether the choice of composition medium has any impact on score quality for subgroups of test takers. Results of analyses demonstrate that (a) scores assigned to word-processed essays are slightly more reliable than scores assigned to handwritten essays and exhibit higher correlations with TOEFL multiple-choice subscores; (b) female test takers, examinees whose native language is not based on a Roman/Cyrillic alphabet, and examinees with lower English proficiency are more likely to choose the handwriting medium; (c) the probability of choosing handwriting as the composition medium increases with age for Asian examinees, but decreases with age for most European examinees; and (d) examinees with lower TOEFL multiple-choice scores tend to have higher handwritten than word-processed essay scores, while examinees with higher TOEFL multiple-choice scores tend to have similar scores in either medium.

Read More