skip to main content skip to footer

Statistical Analyses for the Updated TOEIC Listening and Reading Test DIF TOEIC

Author(s):
Cid, Jaime; Wei, Youhua; Kim, Sooyeon; Hauck, Claudia
Publication Year:
2017
Report Number:
RM-17-05
Source:
ETS Research Memorandum
Document Type:
Report
Page Count:
22
Subject/Key Words:
Statistical Analysis, Test of English for International Communication (TOEIC), TOEIC Listening and Reading Tests, Workplace Skills, Global Trends, Item Analysis, Statistical Report, Pilot Test, Item Difficulty, Differential Item Functioning (DIF), Test Reliability

Abstract

SUMMARY: To ensure that tests continue to meet the needs of test takers and score users, it is important that testing programs periodically revisit their assessments. For this reason, in order to keep up with the continuously changing use of English and the ways in which individuals commonly communicate in the global workplace and everyday life, an updated TOEIC Listening and Reading test was designed and first launched in May 2016. The update was intended to benefit both test takers and scores users. Not only would test takers be better able to demonstrate the relevant skills needed to communicate effectively in today’s global workplace, but also score users would continue to be confident that test scores reflect the range of skills necessary for success in their workplaces. Although the update to the test included some new item types, the overall quality and difficulty of the TOEIC Listening and Reading test was intended to remain the same, with no change in total testing time, number of items, test difficulty or score scale. This paper reports the results of a pilot study that contributed to ensuring that the TOEIC Listening and Reading test continues to be a fair, valid and reliable assessment of everyday and workplace English. The results of the pilot study also helped test developers make the appropriate adjustments to the test before it was launched operationally. Since the updated test was launched, the difficulty and discrimination of the items of the updated test, reliability of its scores and scaled score values have also been closely monitored. The operational results presented in this report also suggest that that the updated TOEIC Listening and Reading test continues to have the same psychometric quality of the pre-updated TOEIC Listening and Reading test.

Read More