NAEP Summer Intern Blog Series: Learning about Equity and Fairness in Assessment

December 8, 2021

Throughout the summer, we’re highlighting the experiences of some of our NAEP interns, in their own words. 

This week’s post is from the perspectives of both Janine Jackson and Christopher Terrazas. This summer, Janine is collaborating with ETS senior research scientist and strategic advisor, Catherine Millett in ETS’s Policy Evaluation and Research Center (PERC), on a project exploring future graduate student profiles. Christopher is collaborating with ETS research scientists, Geoffrey Phelps and Cara Cahalan Laitusis, and senior program advisor of the NAEP program, Jhan Doughty-Berry, on a project analyzing the potential of exploring teacher and student perspectives in NAEP using empathy maps. Here’s their latest view about what they’ve learned from our staff about equity and fairness in assessment:

Janine Jackson:

First let me say that I continuously pray for time to slow down because the summer is passing by far too quickly and I don’t want this experience to end. Having the opportunity and space to meet and speak with people who share my passion for all things “measurement” is absolutely inspiring. These last few weeks have been nothing short of amazing, but this past week in particular was spectacular.

My fellow interns and I met with ETS research and development staff members, Michael Walker, Cara Cahalan Laitusis, Randy Bennett and Haley Lemp to learn about and discuss fairness, equity and accessibility in assessment and learning. It is encouraging to note that the issue of fairness and equity in assessment has been a longstanding focus for ETS. According to the panel, an effort has always been made to “do no harm” meaning identify, remove and prevent bias via differential item functioning (DIF) analysis and item content review. In addition to continuously working to do no harm, there is an added push towards creating learning tools and assessment products that contribute to social justice. Moreover, from an AI perspective, there is a lot of work going on at ETS to make sure products are designed from beginning to end with an accessible and fair user experience in mind through the incorporation of insight gained from the Universal Design for Learning (UDL) framework. This type of work requires lots of thought and research and there are efforts in place to make sure that there are a variety of voices and experiences included in the discussion. That was just Monday.

On Tuesday we learned about the “life of an item.” From conceptualization to administration to scoring and reporting, months of work goes in to creating a single test item. Who knew?! As an aspiring psychometrician, learning about each step in detail from an organization known for excellence in assessment development increased my depth of understanding exponentially.

Also this week, we were matched with ETS staffers that share our backgrounds and interests and were given time away from our projects to chat about their path to employment at ETS, their day-to-day work experience and triumphs and challenges that come with the work. I was lucky enough to be matched with Kelsei Thomas from New Product Development, and we had a delightful and informative conversation. Like every staff member that I have met to date, Kelsei listened to my ideas and perspectives and introduced me to other ETS staff members who can provide insight into my research. For that, I am grateful.

To close out the week, we were able to meet with alumni from our individual colleges and universities. It was so exciting to meet fellow Morgan State University alumna Aisha Torres, who shared how her education and experience at the university prepared her for her work at ETS. We spoke in great detail about curriculum alignment, networking and diversity training and at the end of our conversation, I was even more proud to be a graduate student attending the HBCU and national treasure that is Morgan State University.

Each week the experience at ETS gets better. I know that I am contributing to research that may impact others in a meaningful way. My mentors and team members in the Policy Evaluation and Research Center (PERC), Catherine Millett, Marisol Kevelson and Anjali Srivastava have helped me grow in this short time as a writer and data analyst. It feels great to wake up in the morning and be excited about going to work. I’m looking forward to next week.

Christopher Terrazas:

This past week was one where I was able to participate in a seminar that discussed equity and fairness in assessment. This is a topic that is very dear to my heart, because I spent the last three years working as a classroom teacher. In the space, I was able to create and nurture an environment that valued the voice, experience and identity of young people. This was equity at work.

Michael Walker, distinguished presidential appointee and director of ETS’s Center for Research on Validity, Fairness and Equity in Learning and Assessment, led the session, and while doing so, he provided time for each intern to share a little about themselves. By doing so, he created a space that provided all interns access to listen to one another’s voice, experience and lived identity. He created an atmosphere that was caring and powerful. He did exactly what I did as a classroom teacher. Michael also made a powerful comment. He said, “Because one important effort that ETS makes to promote equity is to invite diverse group of interns to work with us and to teach us what you know; to stay in contact and to collaborate with us in the future to push the field in the right direction”. Another ETSer in the call was managing principal research scientist Cara Cahalan Laitusis. Cara mentioned one comment to us all during the session, and I found it to be powerful. She said, “…it is more than having a diverse team, but also empowering people to speak up.”

These types of comments made in our meeting demonstrate the value that these two ETSers and ETS have in increasing equity and fairness in assessment in education. It is apparent that this work is important now more than ever when institutions are being questioned.

Janine Jackson is a NAEP-Summer Pre-Doctoral Research Experience (SPRE) intern who is currently pursuing her doctoral degree in psychometrics at Morgan State University in Baltimore, Maryland. Christopher Terrazas is a NAEP-Summer Pre-Doctoral Research Experience intern who is currently pursuing his PhD at the University of Texas at Austin.