skip to main content skip to footer

A Study of Characteristics of the SPEAK Test SPEAK TSE

Author(s):
Cascallar, Eduardo; MacCallum, Robert; Sarwark, Susan M.; Smith, Jan
Publication Year:
1995
Report Number:
RR-94-47
Source:
ETS Research Report
Document Type:
Report
Page Count:
71
Subject/Key Words:
English (Second Language), Interrater Reliability, Speaking Proficiency English Assessment Kit (SPEAK), Test Reliability, Test Validity, Test of Spoken English (TSE)

Abstract

The purpose of this project was to investigate issues of reliability and validity arising from the local administration and scoring of the Spoken Proficiency English Assessment Kit (SPEAK), the "off-the-shelf" version of the Test of Spoken English (TSE), and to assess the validity of the SPEAK test as a predictor of classroom English ability for international teaching assistants (ITAs). The reliability issues addressed included interrater consistency within and across institutions, score comparability across institutions and Educational Testing Service (ETS), and differential weightings by individual raters, institutions, and TSE raters of the three diagnostic language factors in assessing the overall comprehensibility of spoken English. SPEAK tests were administered to 119 international teaching assistants at three participating institutions, arranged in a counterblanaced design with 12 ETS-scored tests of the same form, and scored by raters at each institution. For the predictive validity phase of the study, the SPEAK scores of the 59 ITAs who held classroom teaching responsibilities were used as predictive variables in multiple regression analyses. Criterion variables were student ratings of the ITA's spoken language use in the classroom and other instructional contexts as well as ratings of the ITA on several dimensions of teaching effectiveness from the Student Instructional Report (SIR), a standardized instructor/course rating instrument. Results of the study indicate that the locally administered and scored SPEAK test, like TSE, is a reasonably reliable instrument for local screening and an appropriate measure of the general communicative proficiency in spoken English of ITAs in U.S. instructional settings.

Read More