An Investigation of the Impact of Composition Medium on the Quality of Scores From the TOEFL® Writing Section: A Report From the Broad-Based Study
- Author(s):
- Wolfe, Edward. W.; Manalo, Jonathan. R.
- Publication Year:
- 2005
- Report Number:
- RR-04-29
- TOEFL-RR-72
- Source:
- Document Type:
- Subject/Key Words:
- Composition medium computer-based testing direct writing assessment English proficiency English as a foreign language English as a second language TOEFL writing test
Abstract
This study examined scores from 133,906 operationally scored TOEFL® test (Test of English as a Foreign Language™) essays to determine whether the choice of composition medium has any impact on score quality for subgroups of test takers. Results of analyses demonstrate that (a) scores assigned to word-processed essays are slightly more reliable than scores assigned to handwritten essays and exhibit higher correlations with TOEFL multiple-choice subscores; (b) female test takers, examinees whose native language is not based on a Roman/Cyrillic alphabet, and examinees with lower English proficiency are more likely to choose the handwriting medium; (c) the probability of choosing handwriting as the composition medium increases with age for Asian examinees, but decreases with age for most European examinees; and (d) examinees with lower TOEFL multiple-choice scores tend to have higher handwritten than word-processed essay scores, while examinees with higher TOEFL multiple-choice scores tend to have similar scores in either medium.