TOEFL® COE 2015 Research Program

Preliminary applications described below should be submitted via email to toeflrc@ets.org by April 14, 2014.

Purpose

TOEFL® Committee of Examiners (COE) research grants are intended to support research on topics related to the TOEFL® test, both the TOEFL iBT® test and the TOEFL ITP® Assessment Series, and promote high-quality language assessment research as judged by the COE and Educational Testing Service (ETS).

Award Value

Studies involving the collection of new empirical data can normally be funded up to US$125,000. This amount includes institutional overhead.

Research Topics

1. CONCURRENT VALIDITY

1.1 Investigate how TOEFL iBT and/or TOEFL ITP total test scores and one or more section sub-scores (reading, writing, listening and/or speaking) relate to communicative success in English-medium academic content programs, and/or language support programs, especially in regions of the world where English is not used as the primary language. Of particular interest are studies that incorporate one or more of the following features:

  • Investigate expectations for and indicators of academic English-language proficiency (beyond course grades and grade point averages), for example by developing or using criterion measures for communicative success
  • Investigate TOEFL iBT or TOEFL ITP scores as reflective of threshold levels of academic English proficiency for particular instructional settings

1.2 Investigate the relationship between TOEFL iBT or TOEFL ITP test scores and performance on high-priority language use tasks in the target instructional settings by test takers representing a broad range of proficiency levels.

2. INTEGRATED TASKS

2.1 Investigate the use of meta-cognitive, cognitive, affective, and/or social processes involved in responding to integrated language skills tasks, including but not limited to TOEFL iBT test tasks. For example, of interest are studies that:

  • Explore processes and strategies involved in responding to reading to write; reading and listening to write; listening to speak; or reading and listening to speak tasks, focusing on both general processes and strategies involved in integrated tasks and those strategies and processes that are specific to a communication goal such as summarization, synthesis, or comparison/contrast
  • Investigate the socio-cognitive challenges associated with distinct task types and/or design factors
  • Examine the role of audience, purpose, and other factors in influencing the processes and strategies involved in integrated task performance
  • Investigate and model the relationship between proficiency levels and the processes and strategies involved in integrated task performance

2.2 Investigate teaching practices that involve integration of two or more skills and their relationship to students’ performance on TOEFL iBT integrated tasks. The focus of the teaching should be on preparation for using English for academic purposes rather than test preparation for the TOEFL iBT test. A university preparatory ESL program would be particularly suitable as a setting for this study.

3. DOMAIN ANALYSIS

3.1 Identify and examine specific domains and contexts of English-medium instruction around the world, and consider the implications for the design of TOEFL iBT tasks. For example, of interest are studies that:

  • Identify and examine the English communication demands typical of technology-mediated learning environments
  • Identify and examine the English communication demands typical of content courses in English-medium universities in regions where English is not the primary language
  • Characterize the use of English in English-medium instructional contexts from the perspective of English as a Lingua Franca

4. TEST USE

4.1 Investigate the uses of TOEFL iBT and/or TOEFL ITP test scores or section scores (reading, writing, listening and/or speaking), possibly combined with other information, for one or more of the following decision-making purposes: (a) admissions, (b) placement in and/or exiting a program, (c) financial support, and (d) other purposes.

5. INNOVATION AND DEVELOPMENT

5.1 Investigate the design and implementation of innovative technology-mediated task types for assessing academic English proficiency. The task may involve a single modality (listening, speaking, reading, or writing) or the integration of two or more modalities. Examples of suitable task types are scenario-based tasks, simulations, gaming-based tasks, or live synchronous interaction tasks. (Note: The focus of such a study would be on the design and prototyping of the task, rather than programming for operational delivery.)

6. OTHER TOEFL-RELATED RESEARCH

Consideration will be given to research on other issues related to the TOEFL iBT or TOEFL ITP tests.

TOEFL iBT test dataset and research instrument available to COE-funded researchers

The dataset and research instrument described below will be made available to principal investigators funded by the TOEFL COE research program if needed.

TOEFL iBT Public Use Dataset

Three separate datasets from each of two test forms are available and include scores accompanied by demographic information about test takers (age, gender, native language, etc.) and copies of relevant test materials:

  • Set 1. Item-level scores on all sections of the test for a random sample of 1,000 examinees from each form (2,000 examinees total).
  • Set 2. Speech samples and final scores for each of six speaking tasks for 240 examinees from each form, stratified by quartiles (480 examinees total).
  • Set 3. Writing samples and final scores for each of two writing tasks for 240 examinees from each form, stratified by quartiles (480 examinees total).

TOEFL iBT Form Creator software

Software that creates a standalone application which administers TOEFL iBT test items/sections/forms as specified by the researcher — up to two complete forms. The forms can be customized in a variety of ways (e.g., omitting items/sections, inserting pauses, changing timing) and can be administered on a local PC. For each test taker, the form produces as output a scoring file as well as speaking samples (in .wav format) and/or writing samples (in .txt format) when appropriate.

Eligibility and Conditions

Applications are invited from faculty or staff affiliated with not-for-profit organizations and institutions (e.g., universities) with expertise in English language learning and assessment research. Proposals from members of the TOEFL COE and for-profit organizations are not eligible. These studies are not intended to serve as the basis for student theses or dissertations.

Each awardee will be required to submit interim and final reports and to appropriately acknowledge the support of the TOEFL program when disseminating the results of the funded work.

Application Process

Interested applicants should submit an application in accordance with the instructions below. Invitations to submit a full proposal will be issued to selected applicants based on the quality and cost effectiveness of the initial application.

An application must include

  1. A cover page specifying:
    1. Title of proposed study
    2. The name, mailing address, email address and telephone number of the Principal Investigator
    3. Date of submission
  2. A précis of no more than 1,000 words and no less than 700 words including a description of the topic the researchers plan to address, a brief rationale, the research questions and design including the data to be collected (target population, sample size, etc., if relevant), the instruments to be used, and the analyses to be conducted. The application should include a time line and a one-line estimate of the total cost, including institutional overhead. (A full budget is not required at the précis stage).
  3. A one-page description of the Principal Investigator’s qualifications (in short C.V. format).
  4. The names and email addresses of two people who are familiar with the Principal Investigator’s work in this area and are willing to act as references.

Evaluation Criteria

Invitations to submit a full proposal will be issued to selected applicants based on the quality of the précis. Précis will be evaluated for their relevance to the identified topics, the feasibility and quality of the proposed research, the qualifications of the Principal Investigator, organizational capacity to conduct the research, and cost effectiveness. Priority will be given to submissions from individuals not affiliated with ETS. Evaluative comments on précis will be sent only to applicants who receive an invitation to submit a full proposal.

Schedule

  • April 14, 2014 — Deadline for submission of applications and précis
  • May 16, 2014 — Responses to précis mailed
  • July 15, 2014 — Deadline for submission of full proposals
  • October 31, 2014 — Responses to proposals mailed

See also: