Automated Scoring of Spontaneous Speech Using SpeechRater v1.0 TOEFL TPO
Williamson, David M.
- Publication Year:
- Report Number:
ETS Research Report
- Document Type:
- Page Count:
- Subject/Key Words:
Test of English as a Foreign Language (TOEFL),
TOEFL Practice Online (TPO),
Automated Scoring and Natural Language Processing
This report presents the results of a research and development effort for SpeechRater Version 1.0 (v1.0), an automated scoring system for the spontaneous speech of English language learners used operationally in the Test of English as a Foreign Language (TOEFL) Practice Online assessment (TPO). The report includes a summary of the validity considerations and analyses that drive both the development and the evaluation of the quality of automated scoring. These considerations include perspectives on the construct of interest, the context of use, and the empirical performance of the SpeechRater in relation to both the human scores and the intended use of the scores. The outcomes of this work have implications for short- and long-term goals for iterative improvements to SpeechRater scoring.