Analyzing and Comparing Reading Stimulus Materials Across the TOEFL Family of Assessments
- Chen, Jing; Sheehan, Kathleen M.
- Publication Year:
- Report Number:
- RR-15-08, TOEFLiBT-26
- ETS Research Report
- Document Type:
- Page Count:
- Subject/Key Words:
- Test of English as a Foreign Language (TOEFL) Text Analysis Text Complexity TextEvaluator TOEFL iBT TOEFL Junior TOEFL Primary
The TOEFL family of assessments includes the TOEFL Primary, TOEFL Junior, and TOEFL iBT tests. The linguistic complexity of stimulus passages in the reading sections of the TOEFL family of assessments is expected to differ across the test levels. This study evaluates the linguistic complexity of each passage in a corpus of TOEFL stimulus passages. The analysis was conducted using the TextEvaluator, a comprehensive text analysis system developed at Educational Testing Service (ETS). For each TOEFL reading passage, TextEvaluator provides an overall complexity score and 8 component scores that measure text complexity in specific domains. The results suggest that the overall complexity scores of the reading passages selected for use at different test levels are significantly different. According to the analysis of the component scores, the TOEFL Primary, TOEFL Junior, and TOEFL iBT passages are similar with respect to some aspects of text variation and distinct with respect to others. Score ranges based on the distributions of the overall complexity scores and the distributions of the component scores of all the passages at each test level can be used as guidelines to develop or select new passages.