Leading Scientists Convene at ETS to Explore Personalization of Assessment in the Service of Equity

“We must try to measure each individual’s knowledge and skills in ways that are best suited for them.”
- Randy Bennett, ETS Norman O. Frederiksen Chair in Assessment Innovation

In theory, personalized assessment seems so … obvious. We can all agree on what’s needed, but it’s the “how we do it” that has sparked challenges and opportunities that require the assessment community to redefine the future assessment and develop solutions to measure what each individual knows and can do regardless of background, interest, first language or cultural identity.

At ETS, we are living and breathing the future of assessment every single day. And while we are privileged to have some of the leading researchers and innovative minds working on personalized learning, among other key areas of research, we know we cannot alone reshape the current assessment paradigm.

“This event is one of the many steps we take to collectively explore the potential of personalized assessments for the greater good," said Kadriye Ercikan, Vice President of Research at ETS.

The event ignited a dynamic dialogue, fostering discussions on what it will take to make personalized assessments that serve equity a reality. This was the first of four planned convenings this year aligning with ETS's new research agenda; here’s what we heard at our first event:

Randy Bennett,
Norman O. Frederiksen Chair in Assessment Innovation, ETS

"The U.S., as well as many other countries around the world have experienced unprecedented levels of population change. That change has been in the direction of greater diversity along many dimensions but particularly along racial/ethnic, linguistic, socioeconomic lines. All of those dimensions bring significant levels of cultural heterogeneity. Today’s tests were not designed to accommodate the levels of cultural diversity we, or many other countries, are experiencing. That conflict is driving the need for new conceptions of assessment, including personalization in the service of responding to diversity.

If not done intentionally to achieve equity ends, personalization could as easily produce greater levels of inequity. Who does the development matters, the methods they choose matter, the variables they select to personalize upon matter, the participant samples they employ matter. The demographics of those who develop tests and those who develop AI learning systems don’t look anything like the demographics of public-school populations in the U.S. or in many other countries. That’s a problem in the context of designing personalization for equity.”

Matthew L. Bernacki,
Associate Professor of Learning Sciences and Psychological Studies, University of North Carolina at Chapel Hill, Donald and Justeen Tarbet Distinguished Scholar

“Educational researchers have spent decades researching how instructional designs can benefit specific learners, and there are many psychological theories about learning that can serve as playbooks for teachers and designers to think about how a lesson can accommodate or leverage an asset that a learner brings to a learning experience. Rather than build that knowledge, personalization experts need to bring the existing knowledge base into their design schema so they personalize to the right features of the learner that are known to improve how assessments can impact the learner's experiences and outcomes. ... Adopting personalization as a lens for design can help us build a fuller representation of the individual as we design learning and assessment experiences that provide every individual an equitable opportunity to demonstrate and build their knowledge and skills."

Okan Bulut,
Associate Professor of Measurement and Data Science, The University of Alberta

“As we move from one-size-fits-all assessments to personalized assessments, we will need to rethink the core concepts of educational measurement, such as reliability and validity. With broader and more inclusive definitions for these concepts, we can better support the mass adoption of personalized assessments. Personalization holds the potential to address societal issues by fostering a more equitable and inclusive learning environment. Personalized assessments will empower learners in authentically demonstrating what they know and can do through their own learning experiences and choices."

Neil Heffernan,
Founder and Board President, The ASSISTments Foundation; The William Smith Dean's Professor of Computer Science & Director of the Graduate Program in Learning Science & Technologies, Worcester Polytechnic Institute

"It was great to come join ETS and my peers to talk about personalization in the service of equity. Formative assessments will continue to be important, and I was honored to join this group to explain how my online math platform, ASSISTments.org, is personalizing learning."

Serena Magrogan,
Senior Director, AP^® Curriculum, Instruction, & Assessment The College Board^®

“The key to personalized assessment is prescriptive learning. Prescriptive learning opportunities, with clear output parameters and upfront scoring rubrics, may help students take ownership, accountability and even pride in the output of their work. And in this way, the assessment works for them, not against them.”

Kyndra V. Middleton,
Director of Graduate Studies Professor of Educational Psychology,
Howard University

“Over the past few years, we’ve seen exciting and promising developments in personalization in learning and assessment. Personalization has the potential to benefit every student while also providing equitable access. However, with personalization comes the responsibility to ensure adequate measurement, but currently, this movement is trending in a positive direction!”

Robert J. Mislevy,
Professor Emeritus, University of Maryland, Former Frederic M. Lord Chair in Measurement & Statistics, ETS

“Personalization and Socio-Culturally Responsive Assessment (SCRA) leverage students’ personal, social, cultural and linguistic backgrounds as appropriate to better show what they know and can do. My talk outlines an evidentiary-reasoning framework to support designing, using and analyzing the properties of assessments of various kinds applying these principles for various purposes. I examine designs that provide different tradeoffs between the local validity of score interpretations as for learning and the comparability of scores as for large-scale testing programs. Advances in learning science and technology make possible a range of approaches that increase validity, hence fairness, for targeted score uses.”

Jesse R. Sparks,
Senior Research Scientist, ETS

“There are many challenges and opportunities around personalizing assessment in the service of greater equity. How we enact personalization may differ in extent and quality depending on the assessment use case — for example, there may be greater opportunities to incorporate personalization in assessments designed to support teaching and learning. It is critical for us to understand how and when we should personalize and to what degree personalized support can lead to more equitable access to participate in assessment and contribute to more equitable student outcomes and educational opportunities."

Ye Tong,
Senior Vice President of Assessment Operations, NBME

“Personalized assessments have tremendous potential in the formative space to support learners on their paths to success. However, we must be mindful of the cultural and social aspects of assessment design and implementation during personalization. On the one hand, we need to provide fair assessments that give everyone equitable opportunities to demonstrate what they know and can do regardless of their backgrounds. On the other hand, we need to use assessments as a teaching tool to create awareness for the learners not only of their own culture and values but also of others.”