TOEFL® COE 2016 Research Program

ETS is pleased to announce the TOEFL® Committee of Examiners (COE) 2016 Research Program. Preliminary applications described below should be submitted via email to toeflrc@ets.org by April 15, 2015.

Purpose

TOEFL® Committee of Examiners (COE) research grants are intended to support research on topics related to the TOEFL® test, both the TOEFL iBT® test and the TOEFL ITP® Assessment Series, and promote high-quality language assessment research as judged by the COE and Educational Testing Service (ETS).

Award Value

Studies involving the collection of new empirical data can normally be funded up to US$125,000. This amount includes institutional overhead.

Research Topics

1. CHARACTERIZING THE GROWTH PATTERNS OF ACADEMIC ENGLISH PROFICENCY

1.1 Develop and validate descriptors of different levels of English proficiency achieved through study in intensive academic English-language programs/pre-sessional academic English-language programs and/or English-medium academic content programs.

1.2 Conduct linguistic analyses of longitudinal TOEFL iBT writing and/or speaking test data and/or classroom/naturalistic academic language production data and characterize the growth patterns of academic English proficiency.

2. INTEGRATED TASKS

2.1 Design contextualized integrated tasks that specify the goal of communication and the intended audience and examine the role of audience, purpose, and other factors in influencing the processes and strategies involved in integrated task performance.

3. DOMAIN ANALYSIS

3.1 Identify and examine specific domains and contexts of English-medium instruction around the world, and consider the implications for the design of TOEFL iBT tasks. For example, of interest are studies that:

  • Identify and examine the English communication demands typical of technology-mediated learning environments (such as flipped/hybrid classrooms, web-based distance coursework, high-tech classrooms)
  • Identify and examine the English communication demands typical of content courses in English-medium universities in regions where English is not the primary language
  • Characterize the use of English in English-medium instructional contexts from the perspective of English as a Lingua Franca

4. TEST USE

4.1 Investigate the uses of TOEFL iBT and/or TOEFL ITP test scores or section scores (reading, writing, listening and/or speaking), possibly combined with other information, for one or more of the following decision-making purposes: (a) admissions, (b) placement in and/or exiting a program, (c) financial support, and (d) other educational purposes. This may involve document analysis and survey data to obtain evidence of how the scores are interpreted by the users; what other factors influence the relevant decisions; how the test takers perceive the decision-making process; and what the consequences of these test uses are.

5. INNOVATION AND DEVELOPMENT

5.1 Investigate the design and implementation of innovative technology-mediated task types for assessing academic English proficiency. The task may involve a single modality (listening, speaking, reading, or writing) or the integration of two or more modalities. Examples of suitable task types are scenario-based tasks that specify the context including the communication goal, the setting and the intended audience, simulations, gaming-based tasks, or live synchronous interaction tasks. (Note: The focus of such a study would be on the design and prototyping of the task, rather than programming for operational delivery.)

6. OTHER TOEFL-RELATED RESEARCH

Consideration will be given to research on other issues related to the TOEFL iBT or TOEFL ITP tests.

TOEFL iBT test dataset and research instrument available to COE-funded researchers

The dataset and research instrument described below will be made available to principal investigators funded by the TOEFL COE research program if needed.

TOEFL iBT Longitudinal Datasets

ETS will provide upon request datasets containing spoken and written responses to two TOEFL iBT research test forms. Students were enrolled in English-language programs outside the United States and took the research version of the test twice.

  • Set 1. Spoken responses from 100 students (with Korean or Spanish as their L1) enrolled in 16-week English language courses, for a total of 1,200 spoken responses (six responses for each of two different test forms).
  • Set 2. Written responses from 240 students (with Chinese, Japanese, Korean or Spanish as their L1) enrolled in English-language courses for 15 or 16 weeks, for a total of 960 essays (two essays per test form, two test forms).

TOEFL iBT Public Use Dataset

Three separate datasets from each of two test forms are available and include scores accompanied by demographic information about test takers (age, gender, native language, etc.) and copies of relevant test materials:

  • Set 1. Item-level scores on all sections of the test for a random sample of 1,000 examinees from each form (2,000 examinees total).
  • Set 2. Speech samples and final scores for each of six speaking tasks for 240 examinees from each form, stratified by quartiles (480 examinees total).
  • Set 3. Writing samples and final scores for each of two writing tasks for 240 examinees from each form, stratified by quartiles (480 examinees total).

TOEFL iBT Form Creator Software

Software that creates a standalone application which administers TOEFL iBT test items/sections/forms as specified by the researcher — up to two complete forms. The forms can be customized in a variety of ways (e.g., omitting items/sections, inserting pauses, changing timing) and can be administered on a local PC. For each test taker, the form produces as output a scoring file as well as speaking samples (in .wav format) and/or writing samples (in .txt format) when appropriate.

Eligibility and Conditions

Applications are invited from faculty or staff affiliated with not-for-profit organizations and institutions (e.g., universities) with expertise in English language learning and assessment research. Proposals from members of the TOEFL COE and for-profit organizations are not eligible. These studies are not intended to serve as the basis for student theses or dissertations.

Each awardee will be required to submit interim and final reports and to appropriately acknowledge the support of the TOEFL program when disseminating the results of the funded work.

Application Process

Interested applicants should submit an application in accordance with the instructions below. Invitations to submit a full proposal will be issued to selected applicants based on the quality and cost effectiveness of the initial application.

An application must include

  1. A cover page specifying:
    1. Title of proposed study
    2. The name, mailing address, email address and telephone number of the Principal Investigator
    3. Date of submission
  2. A précis of no more than 1,000 words and no less than 700 words including a description of the topic the researchers plan to address, a brief rationale, the research questions and design including the data to be collected (target population, sample size, etc., if relevant), the instruments to be used, and the analyses to be conducted. The application should include a timeline and a one-line estimate of the total cost, including institutional overhead. (A full budget is not required at the précis stage).
  3. A one-page description of the Principal Investigator’s qualifications (in short C.V. format).
  4. The names and email addresses of two people who are familiar with the Principal Investigator’s work in this area and are willing to act as references.

Evaluation Criteria

Invitations to submit a full proposal will be issued to selected applicants based on the quality of the précis. Précis will be evaluated for their relevance to the identified topics, the feasibility and quality of the proposed research, the qualifications of the Principal Investigator, organizational capacity to conduct the research, and cost effectiveness. Priority will be given to submissions from individuals not affiliated with ETS. Evaluative comments on précis will be sent only to applicants who receive an invitation to submit a full proposal.

Schedule

  • April 15, 2015 — Deadline for submission of applications and précis
  • May 15, 2015 — Responses to précis mailed
  • July 15, 2015 — Deadline for submission of full proposals
  • October 30, 2015 — Responses to proposals mailed

See also: