angle-up angle-right angle-down angle-left close user menu open menu closed search globe bars phone store

Justifying the Construct Definition for a New Language Proficiency Assessment: The Redesigned TOEIC Bridge Tests—Framework Paper

Author(s):
Schmidgall, Jonathan; Oliveri, Maria Elena; Duke, Trina; Carter Grissom, Elizabeth
Publication Year:
2019
Report Number:
RR-19-30
Source:
ETS Research Report
Document Type:
Report
Page Count:
20
Subject/Key Words:
Adults Common European Framework of Reference (CEFR) Conceptual Assessment Framework (CAF) Domain Analysis Domain Modeling English Language Proficiency Evidence-Centered Design (ECD) Knowledge Skills and Abilities (KSA) Redesign Test Development Theory of Action TOEIC Bridge Test

Abstract

This paper describes the motivations behind the design of the redesigned TOEIC Bridge assessments to measure all four communication skills, the purposes the assessments, and how we defined English language listening, reading, speaking, and writing proficiency in everyday contexts for basic to intermediate learners. This information provides a basis for test development and subsequent validity research. One of the most critical steps in the test development process is defining the construct, or the knowledge, skills, or abilities, to be assessed. This foundational step provides the basis for initial assumptions about the meaning of test scores and serves as a reference for subsequent validity research. In this paper, we describe the purpose of the redesigned TOEIC Bridge assessments to measure all four communication skills and elaborate the theoretical basis of its construct definition. We also describe how an evidence-centered design (ECD) approach was used to develop the redesigned TOEIC Bridge assessments and the first stage of that approach, the domain analysis. The domain analysis begins by elaborating a clearer definition of the context in which language is evaluated by the redesigned TOEIC Bridge assessments, "everyday adult life." Next, we review research literature and relevant language proficiency standards to highlight the knowledge, skills, and abilities relevant to basic to intermediate English proficiency for everyday adult life. This information is synthesized in the construct definitions for reading, listening, speaking, and writing ability for basic to intermediate levels of general English proficiency in the context of everyday adult life.

Read More