Five Things You Didn’t Know About Test Security at ETS
by Ray Nicosia
| Media Contact: | Kristen Mitchell |
|---|---|
| Email: | mediacontacts@ets.org |
Princeton, NJ, USA (June 3, 2022) – Maintaining the security of our assessments at ETS has been a decades-long endeavor. As testing has moved from paper and pencil to computer based, and from test center to at home, the transformation of the testing industry has been profound. With this evolution comes the need to envision security in new and different ways and that we are continuously doing everything necessary, and then some, to stay ahead of those who attempt to cheat and harm the integrity of our tests.
As complex as the test security field can be, here are five things you may not know (but should) about test security at ETS:
- ETS’s Office of Testing Integrity (OTI) has been combatting test fraud and cheating for over 64 years.
- ETS has invested billions of dollars over the last 6 decades in test security and technology infrastructure to combat cheating.
- ETS can, and does, cancel test scores — even after they are released to test takers and institutions.
- Cheating on standardized assessments is not unique to ETS.
- At home assessments have contributed to an increase in cheating attempts among test takers.
For 64 years, ETS has staffed a dedicated OTI office comprised of investigators, experts and analysts who ensure the validity and integrity of our exams and whose work it is to defend those test takers who test honestly. In coordination with highly trained proctors, AI technologies and data analysis, the team works to analyze scores and testing sessions that are suspected of having been earned or conducted unfairly and, as a result, cancel test scores, should evidence warrant doing so. As an industry leader in the space, our OTI office recognizes the need to do more to combat cheating around the globe.
ETS has invested tens of millions of dollars each year, for the last number of years, on test security and technology infrastructure — billions over the last 6 decades. In the coming months, we plan to invest tens of millions of additional dollars to further our technological capabilities and improve test security measures to catch even the most minute forms of cheating. These capabilities and improvements include but are certainly not limited to: continued remote access software (RAS) prevention using ETS proprietary expertise and capabilities coupled with new, integrated technology to sustain world class test security measures; enhancing machine learning models to continue to detect dishonest test-taking behaviors at a high rate; the development of new and refinement of existing statistical evaluation models that support continued efforts to identify and shut down cheating rings and cheaters; and continued research on new item types and assessment design approaches specific to combatting cheating. The investments we make are in addition to our rigorous security protocols and quality control procedures in place to investigate issues, cancel scores and hold individuals accountable who violate our testing policies.
When ETS has evidence that a test taker’s scores may have been earned unfairly — a violation of our testing policy — OTI can hold scores from being released to further evaluate the testing session and score earned to determine whether those scores should be canceled. In some cases, that means canceling scores even when they were already released to test takers and/or institutions if evidence is collected post-release that determines that scores were earned due to an unfair advantage. In the most extreme cases, ETS bans test takers who are in violation of our testing policies.
Cheating is an industrywide issue which has been spotlighted by the boom of the remote testing industry over the last 2 years. Although it is not frequently discussed within the education sector, we owe it to our stakeholders to keep them informed of what we are seeing, how we are addressing concerns and our plans for the future. We know that test security is one of the most crucial pillars of the end-to-end testing process that assures scores are valid, reliable and trustworthy, and as the organization leading the field in test security and remote testing, we take seriously our responsibility to protect the test takers who test honestly. We are equally committed to serving institutions around the world relying on scores for high-stakes decisions. Unreliable scores have implications for unprepared students who cannot meet the rigor of higher education programs and in turn have negative impacts on hard-earned institutional reputations. Score cancellations are a sign of good enforcement by testing organizations and the lack of such cancellations should give pause to institutions and stakeholders alike.
With the new delivery model of at home testing, ETS has seen a more than 200% increase in score cancellations across both TOEFL® and GRE® test administrations combined in our second year of at home testing (FY21) compared to the first year (FY20). Although this increase has occurred, this does not mean we have waned in our vigilance. In fact, quite the opposite is true, which is evidenced by our ability to identify fraudulent activity and cancel scores after thorough analysis and investigations have taken place. While there are more attempts at cheating, by default, more scores are being cancelled. We have seen examples of test takers’ attempts to cheat using remote access software, proxy testing, cellphones to send and receive messages, as well as a combination of these and other cheating techniques. Our OTI team is made up of leading experts on test security and will continue to remain ahead of the curve in ensuring we catch and deter those test takers who attempt to cheat on our assessments.
Ray Nicosia is a Principal, Test Security in the Office of Testing Integrity at ETS.
About ETS
At ETS, we advance quality and equity in education for people worldwide by creating assessments based on rigorous research. ETS serves individuals, educational institutions and government agencies by providing customized solutions for teacher certification, English-language learning, and elementary, secondary and postsecondary education, and by conducting education research, analysis and policy studies. Founded as a nonprofit in 1947, ETS develops, administers and scores more than 50 million tests annually — including the TOEFL® and TOEIC® tests, the GRE® tests and The Praxis Series® assessments — in more than 180 countries, at over 9,000 locations worldwide.