Video Title: How ETS® Makes the TOEFL® Test Fair - Video transcript
People in this video
Pablo Garcia Gomez - Assessment Specialist, ETS
On-screen: How ETS® Makes the TOEFL® Test Fair
Pablo Garcia Gomez (Assessment Specialist, ETS): The TOEFL Program is very interesting because one of the assets of the program is we have people who come from different countries, like me. I come from Argentina. I did my education in Spanish, and I learned English as a foreign language. So did some of my colleagues here who are from many different countries like Korea, the Czech Republic, Nigeria, Surinam and many others.
On-screen: The TOEFL® Test is accepted by 8,000 institutions in over 130 countries
Pablo Garcia Gomez: We also have members of our team who were born in English-speaking countries but who lived and worked in many different countries, and having different people with different background, actually makes the review process very, very rich and we get different points of view. So, definitely this is not an American test or North American test. The TOEFL test is used by thousands of institutions around the world.
On-screen: Creating and Reviewing TOEFL® Test Questions
- Test Framework
- Content Review
Pablo Garcia Gomez: The creation and review of the questions that would go into the TOEFL test is a very rigorous and thorough process. We have reference documents that we use. We have the frameworks that tell us what should be tested, key documents because there’s a lot of research behind them, and we have specifications, very clear and detailed instructions on how to write the questions. After that, there are a series of reviews, we call "content reviews" and "fairness reviews". During the content review process, we pay a lot of attention to the clarity and accessibility of the language that we are using, and for all questions, we want to make sure that they are comparable, that these questions that we are putting today in the tests are comparable to other questions in the test and other questions in other test forms.
On-screen: Fairness Review
Pablo Garcia Gomez: The content review is only part of the review process and ETS standards for assessment mandates that every single question in the test receives a fairness review. We have special fairness reviewers who are trained and calibrated. Because they don’t work in the TOEFL program, the review is more objective. The assessment specialists and the fairness reviewers take a look at what we call "construct irrelevant" variance aspects that are not part of what we want to measure, that may interfere with the test-taker's response, and we don’t want the test takers to be distracted by those things.
On-screen: Pilot Testing & Statistical Review
Pablo Garcia Gomez: There’s another very important piece of information that we use. That is the statistical information. So, we try our questions and see how they perform before they actually count for test-taker's scores. If they perform well, we include them in the test. Even after administration, we look at the statistical result.
The effort is ongoing. It never stops. To do that, we have to look at how a test did on this administration and how it did on the following administration. It goes on and on and on. As someone who has learned English as a foreign language and someone who has helped other people learn English as a foreign language, you want to ensure that when someone goes to take the test that the situation and the test itself is as fair as possible. It makes me feel really proud of what we do here.
Total length of video: 3:36