angle-up angle-right angle-down angle-left close user menu open menu closed search globe bars phone store

TOEFL® Committee of Examiners 2019 Research Program

ETS is pleased to announce the TOEFL Committee of Examiners (COE) 2019 Research Program. Applications described below should be submitted via email to TOEFLRC@ets.org by July 13, 2018.

Purpose

TOEFL COE research grants are intended to support research on topics related to the TOEFL iBT® test and the TOEFL ITP® Assessment Series, promoting high-quality language assessment research as judged by the COE and Educational Testing Service (ETS). Applications that employ diverse and innovative research methodologies are encouraged, as are those focused on the TOEFL ITP Assessment Series.

Award Value

Studies involving the collection of new empirical data can normally be funded up to US$100,000. This amount includes institutional overhead. ETS prefers to select bids which respect its nonprofit status. We therefore request that the overhead rate not exceed 15 percent.

Research Topics

1. Exploring the predictive validity of TOEFL iBT and/or TOEFL ITP® tests within countries where English is not the dominant language

1.1 Use test score data from the TOEFL iBT or TOEFL ITP tests, as well as other measurements or data, such as background variables, student demographic information, home/family metrics or other variables deemed appropriate, to predict performance at universities or colleges. For example, performance may be measured according to grades, writing sample scores, teacher evaluations, etc. A variety of research designs may be considered, including retrospective longitudinal designs and/or studies that have enough data points to employ, with appropriate levels of power, structural equation modeling or a repeated measures design.

1.2 Conduct a content analysis of the TOEFL iBT and/or TOEFL ITP tests by comparing aspects of the exam (or the exam's preparation materials) with the in-country/English-medium-university curricular content and related English-language demands. Strongly preferred are studies that include an analysis that considers the country's national English-language learning standards, and also preferred are studies that look at multiple in-country curricula and data sources.

2. Investigating the use of mobile-assisted technologies in language testing

2.1 Investigate the use of mobile phones and/or other handheld devices for integrated-skills testing within 1 or more of the following populations: English language learners (a) of varying proficiency, (b) of different L1 backgrounds, and (c) across diverse geographical regions.

2.2 Investigate, through surveys, focus groups, or other qualitative methods, English-language teachers' and students' uses of, and needs for mobile-assisted technologies in language learning, including the roles played by assessment.

3. Test use/policy implications

3.1 Investigate the interpretation and uses of TOEFL iBT and/or TOEFL ITP total and/or section scores, possibly combined with other information, for 1 or more of the following decision-making processes: (a) admissions, (b) placement into and/or exiting from a program, (c) awarding financial support, and (d) other educational or professional purposes. Also investigate test takers' perceptions of the decision-making process and the consequences of these decisions.

3.2 Conduct studies to establish the appropriate TOEFL iBT and/or TOEFL ITP test score for different test takers and purposes (e.g., validating cut scores for specific uses).

4. Investigating the 4-skill profiles (proficiency test scores on listening, reading, speaking and writing) of English language learners in relation to background factors (L1, country of origin, etc.) and skill-specific requirements in English for Academic Purposes (EAP) settings

4.1 Conduct studies to evaluate the language-skill profiles of EAP students relative to their background variables or independent variables, such as L1, country of origin, age, length of time spent studying English, etc. Other factors or backgrounds are welcome to be suggested.

4.2 Investigate learner groups with variable language-skill profiles to examine progress in English academic programs, or whether different groups have unique challenges or circumstances that contribute to varying rates of success.

4.3 Explore stakeholder perceptions of the relationship between learners' EAP profiles and their academic performance.

5. Other

Consideration will be given to research on other issues or innovations related to the TOEFL iBT or TOEFL ITP tests.

TOEFL iBT test dataset and research instrument available to COE-funded researchers.

The dataset and research instrument described below will be made available to principal investigators funded by the TOEFL COE research program, if needed.

TOEFL iBT Public Use Dataset

3 separate datasets from each of 2 test forms are available and include scores accompanied by demographic information about test takers (age, gender, native language, etc.) and copies of relevant test materials:

  • Set 1. Item-level scores on all sections of the test for a random sample of 1,000 examinees from each form (2,000 examinees total).
  • Set 2. Transcripts of spoken responses and final scores for each of the 6 speaking tasks for 240 examinees from each form, stratified by quartiles (480 examinees total).
  • Set 3. Writing samples and final holistic scores for each of the 2 writing tasks for 240 examinees from each form, stratified by quartiles (480 examinees total).

TOEFL iBT Form Creator Software

Software that creates a stand-alone application which administers TOEFL iBT test items/sections/forms as specified by the researcher — up to 2 complete forms. The forms can be customized in a variety of ways (e.g., omitting items/sections, inserting pauses, changing timing) and can be administered on a local PC. For each test taker, the form produces an output of a scoring file, as well as speaking samples (in .wav format) and/or writing samples (in .txt format) when appropriate.

Eligibility and Conditions

Applications are invited from faculty or staff affiliated with nonprofit organizations and institutions (e.g., universities) with expertise in English language learning and assessment research. Proposals from members of the TOEFL COE and for-profit organizations are not eligible. These studies are not intended to serve as the basis for student theses or dissertations.

Each awardee will be required to submit interim and final reports, in addition to appropriately acknowledging the support of the TOEFL program when disseminating the results of the funded work.

Letter of Institutional Commitment

Proposals must include a letter committing the applicant's institution to the proposal and stating its validity for a period of at least 90 days from the proposal's deadline date. The original letter must be signed by a representative of the institution's contracting office with commitment authority, and must be submitted with the proposal. Proposals will not be reviewed for consideration until such a letter has been received. The contracting officer may obtain a sample ETS grant agreement by submitting a request to pmills@ets.org.

Optional Letter of Intent

Interested researchers are strongly encouraged, but not required, to submit a brief (maximum 1-page) letter of intent by May 1, 2018. The letter should be submitted electronically to TOEFLRC@ets.org, should have "Intent to apply" in the subject line, and should contain the following information: (1) The name and address of the principal investigator's higher education or research institution, and the names and affiliations of any co-investigators; (2) the research topic that the proposal addresses; and 3) a very brief description (maximum 2 paragraphs) of the proposed study. Applicants who submit letters of intent will receive feedback regarding their proposed study. In addition, receiving letters of intent from potential applicants informs ETS about the approximate number of applications that may be received and allows for more efficient planning of the proposal review process. Applicants who do not send this letter may still submit an application by the July 13, 2018, deadline.

Application Process

Proposal materials must be submitted via email to TOEFLRC@ets.org by 11:59 p.m. U.S. Eastern Time on Friday, July 13, 2018.

Submissions must include the following documents:

Proposal Content Requirements

The non-blind copy of your proposal must include all of the following key components:

  1. A cover page, which must include:
    1. the name and affiliation of the principal investigator and any co-investigators
    2. the email address and phone number of the principal investigator
    3. the title of the project
    4. identification of the research topic(s) addressed by the proposal
    5. the date of submission
  2. An abstract page, which must include:
    1. the title of the project
    2. a statement of the problem, including a brief description of the project proposed and a brief discussion of the implications or utility of the anticipated outcomes (250 words maximum)
    3. identification of the research topic(s) addressed by the proposal
  3. The body of the proposal should be approximately between 2,000 and 2,500 words and must include each of the following sections:
    1. Rationale (statement of the problem or the issue and relevance to 1 or more of the research topics above)
    2. Literature review (focused summary of related research)
    3. Research questions (specific question(s) or research objectives to be addressed by the project)
    4. Methodology (explanation of the proposed research methodology, including descriptions of the following):
      • Study context and participants
      • Instruments to be used
      • Data collection procedures
      • Identification and coding of variables
      • Analyses to be conducted
    5. Implications (a brief discussion of the implications or utility of anticipated or possible outcomes and relevance to practice or theory)
    6. Limitations (a brief discussion of the limitations of the proposed study)
  4. References
  5. Budget (See our sample budget for needed specifications.)
  6. Schedule (list of the major activities of the project and the period in which each will be conducted; please assume March 1, 2019, as the start date for the project)
  7. Proposed data management plan (1-paragraph description of the type of data to be collected, the type of permissions requested for collecting and storing the data (IRB/ethical approval), and processes for storing de-identified data)
  8. Proposed dissemination plan (state in how many and what type of publications you intend to disseminate the findings; for instance, 1 ETS TOEFL iBT Research Report, or 1–2 journal articles)

The blind copy of your proposal should be the same as the non-blind copy, except it must not include:

  • the cover page
  • the author(s)' identifying information (including names and affiliations) anywhere in the abstract page, body, references, budget and schedule

Proposal Format Requirements

Please use the following format:

  • Microsoft Word® 2003 or later version (.doc or .docx)
  • Times New Roman font size 12
  • 8.5" x 11" (U.S. "Letter") page size
  • 1-inch (1.0") margins on all sides
  • Double (2.0) spaced
  • For page numbering, paragraph indentation, references, graphs and tables, please follow APA style (American Psychological Association, 6th Edition)

Proposal Evaluation Process

Each proposal will be evaluated according to the following criteria:

  • The proposal is relevant to the research topics identified above.
  • The rationale for conducting the research is sufficient.
  • The background or literature review is adequate and appropriate.
  • The research questions are clearly stated.
  • The data collection procedures are clearly explained.
  • The methods of data analysis are sufficiently detailed and appropriate.
  • The authors discuss the anticipated results and implications for the field of language learning and assessment.
  • The authors specify the limitations of the proposed study.
  • The principal investigator has the appropriate qualifications and background to conduct the proposed study.

The review will take into account each of the criteria above. To view our review form, see TOEFL COE Proposal Review Form.

Applicants who are invited to revise and resubmit proposals will receive constructive comments. An invitation to the next step does not guarantee that applicants will receive a grant.

Reporting Requirements and Payment Schedule

All grant winners will be required to submit 2 to 3 progress reports during their funded study. The exact due dates for the progress reports depend upon the individual project schedule. Please indicate the dates by which you intend to submit your progress reports in your project schedule.

Grant funds are generally disbursed in several payments, aligned to study milestones. The timing and amount of grant disbursements is flexible, depending on the project's needs. Please estimate in your budget and project schedule the specific amounts you will need, for specific purposes, at specific dates. For each payment that is disbursed, ETS must receive an invoice that itemizes the relevant project deliverables for which grant funds are requested.

Dissemination Plan

Authors are expected to submit a publishable manuscript as the final deliverable of the grant. Typically, the research can be published either as an ETS Research Report or as an article in a scholarly journal. Please specify which publication venue you plan to pursue, and be aware that the manuscript will have to pass ETS technical review before being submitted for publication, whether as an ETS Research Report or an article in a scholarly journal.

Grant recipients are required to appropriately acknowledge the support of the TOEFL program when disseminating the results of the funded work. The following standard disclaimer should be included in any publications, including conference presentations:

"This research was funded by the Educational Testing Service (ETS) under a Committee of Examiners and the Test of English as a Foreign Language research grant. ETS does not discount or endorse the methodology, results, implications or opinions presented by the researcher(s)."

Contact

If you have additional questions about the application process or the grant in general, please contact us at TOEFLRC@ets.org.

Schedule

  • July 13, 2018 — Deadline for submission of full proposals
  • October 31, 2018 — Responses to proposals mailed

See also: