skip to main content skip to footer

The Effectiveness of the TOEFL Essentials Test for Distinguishing English Proficiency Levels TOEFL MST

Author(s):
Norris, John M.; Lee, Jeremy
Publication Year:
2023
Report Number:
RM-23-07
Source:
ETS Research Memorandum
Document Type:
Report
Page Count:
43
Subject/Key Words:
TOEFL Essentials, English Language Proficiency, Validity Argument, Evaluation Methods, Test Reliability, Criterion Validity, Admissions to Higher Education, Test Design, Multistage Testing (MST), Test Scores

Abstract

The TOEFL Essentials test, launched in 2021, is an assessment of English language proficiency for use in informing higher education admissions decisions as well as for other purposes that call for an overall estimation of a learner’s English proficiency in daily life and academic settings. The test design combines innovative item types and multistage adaptive testing to emphasize both efficiency of measurement and effectiveness in estimating learners’ proficiency levels, from beginning to advanced users of English. A basic assumption underlying the validity of interpretations and uses for the test is that scores will consistently distinguish English learners at varying levels of proficiency. To investigate this assumption, test performance data were collected from foreign language learners of English enrolled in university English language programs in Canada and the United States. Learners’ English proficiency levels were estimated a priori based on course placements within the program curricula, and teacher judgments of learner proficiency levels were also collected. We found that test scores exhibited high reliability estimates and learners’ scores were spread across a wide range of score bands. Comparisons of test scores between a priori groupings by low, medium, and high abilities indicated that the test consistently distinguished between the groups. While strong positive correlations were found between teacher ratings of learners’ proficiencies and corresponding test scores, teachers estimated the proficiency levels of learners slightly higher than test scores in relation to an external language proficiency framework. Learners also exhibited generally uneven profiles of ability across the four skills tested, raising questions about the extent to which course placements can serve as a reliable criterion variable for representing holistic proficiency levels. Implications for the interpretation of TOEFL Essentials test scores and directions for future validity research are discussed.

Read More