skip to main content skip to footer

Exploring the Potential of a Video‐Mediated Interactive Speaking Assessment ELL CBA

Author(s):
Ockey, Gary J.; Timpe-Laughlin, Veronika; Davis, Larry; Gu, Lin
Publication Year:
2019
Report Number:
RR-19-05
Source:
ETS Research Report
Document Type:
Report
Page Count:
29
Subject/Key Words:
Speaking Assessments, Interactive Video, Skype (Video Conferencing Service), Interaction Competence, Speaking Tasks, Social Interaction, Oral Communication, Communication Skills, Dialogues, English Language Learners (ELL), Computer Based Assessment (CBA), China

Abstract

The construct of oral ability is multifaceted, but due to technological and practical constraints, the majority of computer‐delivered speaking assessments are designed to measure only certain aspects of this ability. Most notably, interactional competence, which we define as the ability to actively structure dialogic speech in real time, is often not assessed. Innovations in technology, namely, computer‐mediated video, make it possible for test takers in different locations to see and speak with others in real time and may make it achievable for computer‐based tests to assess more aspects of oral communication, including interactional competence. This report gives the findings from a study that explored to what extent computer‐mediated video, namely, Skype, could function in conjunction with a platform designed to present four innovative speaking tasks that could conceivably assess a broad construct of oral ability. The overarching goals of this project were twofold. First, we aimed to determine (a) the degree to which current computer video‐mediated technology can be used effectively to deliver assessments remotely and (b) the extent to which participants felt that four specific tasks could assess speaking ability by means of this technology. The speaking tasks included giving short responses to an interlocutor's questions, summarizing a proposal, defending a position in a group discussion, and giving a prepared presentation and responding to questions from other participants. Two data collections were conducted: one with all 72 participants located in the United States and one with all 74 participants located in China. The findings provide preliminary evidence that the stability of computer video‐mediated technology varies considerably, with technical disruptions being relatively few in the U.S. trial but very frequent in the China context. Moreover, the findings suggest that participants viewed the tasks as generally representing interactive speaking activities that they encounter in the oral language use domain, affording them the opportunity to demonstrate their oral abilities, and that these tasks can be effectively completed in a computer video‐mediated environment when technology cooperates.

Read More